Reads and Writes

§ Binary Values

write, set, read, get, contains and remove address binary values by key, while scan, sample and measure walk them, each in the blobs collection its collection= names. A write takes one bytes-like value per key, one value broadcast to every key, or a tape, and borrows every value for the length of the call rather than copying it:

posters = database.collection_create("posters", modality=ustore.Modality.BLOBS)
database.write(keys, [b"a", b"bb", b"ccc"], collection=posters)  # one value per key, read by an accessor
database.write(keys, b"same", collection=posters)       # one value every key repeats
database.write(keys, pyarrow.array([b"a", b"bb", b"ccc"]), collection=posters)  # an Arrow binary array, a u32 tape
database.write(keys, (data, np.array([0, 1, 3, 6], dtype=np.uint64)), collection=posters)  # bytes and offsets, u64
database.write(other_keys, database.read(keys, arena, collection=posters), collection=posters)  # a read, as a tape

A tape is an Arrow binary, large_binary, string or large_string array, or a (data, offsets) tuple of buffers whose offsets are unsigned 32-bit or 64-bit integers, one more than the values, climbing within the data. It reaches the C ABI as the u32 or u64 tape it already is, so nothing is copied or gathered, and an Arrow null is refused, since remove removes keys. A read answers the tape the engine wrote, with 32-bit offsets while its bytes fit in 4 GiB and 64-bit ones beyond, and Values exports it through the Arrow C data interface as binary or large_binary to match.

scan(start, wanted, arena) lists up to wanted keys from start upwards, and end= stops it before a key. direction=ScanDirection.DESCENDING walks down from start instead, largest key first, and a start of None begins at the first key in the direction:

database.scan(10, 8, arena, collection=posters, end=30)                          # keys in [10, 30)
database.scan(None, 8, arena, collection=posters, direction=ScanDirection.DESCENDING)  # the 8 largest keys
database.scan(30, 8, arena, collection=posters, end=10, direction=ScanDirection.DESCENDING)  # keys in (10, 30]

scan_batch(starts, wanted, arena) runs one scan per start in one call, and sample_batch(count, wanted, arena) draws count samples, all from the one collection collection= names, every one held to the same wanted, which is what the C ABI takes; scan and sample remain the single forms. measure(start, end, arena) answers a Measurement of the bounds on the entries, the bytes of their values and the space they take, without walking the range, which stops before end and takes the same direction=. docs_measure, pages_measure and graph_measure_vertices measure the other modalities, as docs_sample, pages_sample and graph_sample_vertices sample them. read(keys, arena, offset=16, limit=64) reads up to 64 bytes of every value from its 16th, where an offset past the end reads empty. remove(keys, ends=ends) removes every key from each of keys up to its end in one atomic call, and so do docs_remove, graph_remove_vertices and pages_remove, whose ranges remove whole pages.

cursor(start=None, *, collection, end=None, direction=ScanDirection.ASCENDING) opens a walk over the keys of one collection, and its next(wanted, arena, *, values=False) pulls the next page into the arena that pull names, with the values of its keys in the same call when asked:

arena = ustore.Arena()
with database.cursor(collection="movies") as cursor:   # reads nothing until its first pull
    while page := cursor.next(4096, arena, values=True):
        for key, poster in zip(page, page.values):      # UUIDs and memoryviews over the arena
            consume(key, poster)

A pull resumes past the last key the one before it handed out, so a key written behind the walk never comes and one written ahead of it does. A page shorter than wanted exhausts the walk for good: every later pull answers an empty page, which is falsy, whatever is written since. A key removed at the head between the listing and the read of one pull reads as an empty value, and a pull that fails or is cancelled leaves the cursor where it was. The cursor holds its database open, and the transaction or snapshot it reads through, until it closes, so a snapshot may close before the walk ends. Every pull blocks, on the receiver’s executor or the calling thread, so a deferred view opens no cursor, and a cursor serves one pull at a time. ustore.keys.scan_keys(view, collection, modality=...) yields every key through the scans of that modality, scan, docs_scan or graph_scan_vertices, and one arena it keeps for the walk, which the NetworkX and Pandas views build on.