Reads and Writes

§ Binary Values

Beneath the collections, every verb takes a batch, names its collections by identifier, and hands back what its call produced.

use ustore::{Arena, CollectionID, RAMOptions, Database, Durability, Entry, Key, Matching, Modality};

let database = Database::init_ram(&RAMOptions::default())?;
let mut arena = Arena::new()?;

let [ada, grace, kent] = [1u64, 2, 3].map(Key::from);
let people = database.collection_create("people", Modality::Blobs)?;
let entries = [Entry::new(ada, b"Ada"), Entry::new(grace, b"Grace")];
database.head().write(people, &entries, Durability::Buffered)?;

let values = database.head().read(people, &[ada, grace, kent], &mut arena)?;
assert_eq!(values.get(0), Some(b"Ada".as_slice()));
assert_eq!(values.get(2), None);

An Entry pairs a key with borrowed bytes, laid out so a slice of them crosses as it stands: the keys as one strided run, and the values through an accessor over the same entries, so nothing is staged on the way in. A write also takes owned keys beside a stringtape::BytesTape<u32> or BytesTape<u64> of their values, one more offset than keys into one run of bytes, which the library reads as it stands. A read hands back a Values, which addresses the tape in the arena rather than copying out of it, and reports a key the database does not hold as None. Values::tape hands the whole batch out as a Tape over that tape: one more offset than values, and the bytes between them, 32-bit while the bytes fit in 4 GiB, as Arrow lays out binary, and 64-bit beyond, as it lays out large_binary. A missing value is empty there, and Values::presences is the bitmap of the keys found, as Arrow reads a validity buffer.

A write carries values and nothing else, so an empty slice writes an empty value, and On::remove removes keys with their values in a batch of its own. A removal therefore never hides inside a batch of writes.

let entries = [Entry::new(ada, b"rewritten"), Entry::new(kent, b"")];
database.head().write(people, &entries, Durability::Buffered)?;
database.head().remove(people, &[grace], Durability::Buffered)?;

let first = database.head().scan(people, Key::from(0u64), 100, &mut arena)?.keys;
let measured = database.head().measure(people, ada..kent, &mut arena)?;
let largest = KeyRange { direction: ScanDirection::Descending, ..KeyRange::default() };
let last = database.head().scan(people, largest, 10, &mut arena)?.keys;
let headers = database.head().read_slices(people, &[ada, kent], ..64, &mut arena)?;
database.head().remove_range(people, ada..kent, Durability::Buffered)?;

On::remove removes whole documents the same way, while a field goes with a merge that carries a JSON null for it. scan lists up to the count it wants from a start key, sample draws keys at random, and measure bounds the keys and the bytes a range of keys holds. Scans, cursors and measures take a KeyRange: a start key, inclusive, an end key, exclusive, and a ScanDirection, where a key alone walks up from it and start..end walks up between them. read_slices reads one byte range of every value, and remove_range, docs_remove_range and graph_remove_vertices_range remove every key of a range in one atomic call. docs_sample, pages_sample and graph_sample_vertices draw keys from the other modalities, as docs_measure, pages_measure and graph_measure_vertices measure them.

A Cursor walks one collection page by page, each pull landing in the arena it is given, with the values of its keys when it asks for them:

let mut cursor = database.head().on(&executor).cursor(people, Key::from(0u64), Matching::default())?;
loop {
    let page = cursor.next_pairs(1024, &mut arena)?;
    for (index, key) in page.keys().iter().enumerate() {
        println!("{key}: {:?}", page.get(index));
    }
    if page.keys().len() < 1024 { break; }
}

Every pull resumes past the last key the pull before it handed out, so a key written behind the walk is never handed out and one written ahead of it is. next pulls the keys alone, and next_pairs reads their values in the same call, where a key removed at the head between the listing and the read of one pull reads as an empty value. A page shorter than the count it wanted exhausts the walk for good, so every pull after it hands out nothing, even once more keys are written, and a pull that fails leaves the cursor where it was. A cursor opens from a blocking view, sees what that view sees pull by pull, and pulls on its executor; it holds the database open, and the transaction or snapshot it reads through, until it is dropped.

Database::changes_cursor opens a feed of the changes committed to one collection after a CommitStamp, oldest first, pinning that stamp as a snapshot does:

let mut feed = database.changes_cursor(people, stamp)?;
let changes = feed.next_pairs(1024, &mut arena)?;
for (index, (key, kind)) in changes.keys().iter().zip(changes.kinds()).enumerate() {
    println!("{key} {kind} at {:?}: {:?}", changes.stamps()[index], changes.value(index));
}

Every pull resumes past the last change the one before it handed out, and an empty one reached the newest commit. A ChangeKind is a write, a removal or a partial change, and a range removal comes as the removal of every key it removed. A feed whose history is already gone is refused with Status::WriteConflict, and LevelDB, which keeps no stamps, refuses it with Status::Unsupported.