Batches and Arenas

§ Arenas

Shorthand and verbs trade convenience for reuse, as in every binding:

• Shorthand makes a one-off arena. blobs[key].value() calls ustore_arena_init, reads into that arena, and hands it to its answer, which frees it on destruction, so nothing is copied and no handle keeps state. A scalar answer, such as a presence or a degree, is copied out and the arena freed at once. The price is one arena per call, so a loop reads through a verb instead.
• Verbs reuse a caller’s arena. arena_t::make(executor) places one where its executor computes, or on host pages for calls on the calling thread, and a loop passing it to every call allocates once. Every call rewinds it, so its views last until the next call given that arena; to keep them longer, pass that call another arena.

auto [arena, arena_status] = arena_t::make(executor);
for (std::span<ustore_key_t const> batch : batches) {
    auto [values, read_status] = db.on(executor).find("blobs")->read(batch, arena);
    if (read_status.failed()) break;
    consume(values); // valid until the next call given `arena`
}

An owned_value_t derives from the value_view_t it holds, so it compares and converts like one, and view() slices that plain view off. Collections, transactions and snapshots hold no arena, so threads share one handle, each passing its own arena.

Walks split the same way: the keys, pairs and prefixed streams own one arena each, while blobs_cursor_t and strings_cursor_t take the caller’s on every pull, so a loop reuses one arena it keeps:

auto [cursor, cursor_status] = blobs_cursor_t::make(db.receiver(), blobs);
std::size_t const wanted = 256;
for (std::size_t pulled = wanted; cursor_status.succeeded() && pulled == wanted;) {
    auto [page, page_status] = cursor.next_pairs(wanted, arena);
    if (page_status.failed()) break;
    consume(page.values); // valid until the next call given `arena`
    pulled = page.keys.size();
}

make opens the walk at the view its receiver reads and from an optional start key, next pulls keys alone and next_pairs keys with their values, and a page shorter than wanted exhausts the walk for good. strings_cursor_t::make takes a prefix and an optional start string, and its next pulls strings in the order of their text.

§ Receivers and Deferred Calls

The receiver chooses where a call runs, and the verb stays. db.on(executor), transaction->on(executor) and snapshot->on(executor) hand out the same collections, proxies and operators, whose calls block on the executor:

auto [executor, executor_status] = executor_handle_t::for_ram({.threads = 4});
blobs_collection_t on_executor = *db.on(executor).find("blobs");
status = on_executor[key_from_integer(42)].assign("purpose of life");

deferred(executor) hands out the deferred twin of every collection, whose verbs mirror the C calls and answer a deferred<verb> once submitted. find<graph_collection_t>(name) picks the twin by modality, read-only at a snapshot, while the deferred database’s [] takes a binary collection by the identifier create or find resolved, so addressing it never blocks. status() polls without blocking, std::move(call).wait() answers what the call produced, and a deferred dropped unwaited waits in its destructor:

deferred_database_t later = db.deferred(executor);
auto [arena, arena_status] = arena_t::make(executor);
std::array<ustore_key_t, 2> const keys {key_from_integer(1), key_from_integer(2)};
std::array<sz::string_view_t, 2> const texts {R"({"person": "Alice"})", R"({"person": "Bob"})"};

sz::expected<values_tape_t> values = sz::strings_view_t(texts).gather();
if (!values) return;
deferred<blobs_write_t> writing = later[blobs].write(keys, std::move(values.value));
status = std::move(writing).wait();
deferred<blobs_read_t> reading = later[blobs].read(keys, arena);
expected<values_view_t> read = std::move(reading).wait();
for (value_view_t found : *read)
    std::printf("%zu bytes\n", found.size());

Documents are written and read whole or by field, under any modification, and a gather answers a table together with the rows and columns it views outside the arena:

deferred_docs_collection_t people = *later.find<docs_collection_t>("people");
sz::expected<values_tape_t> documents = sz::strings_view_t(texts).gather();
if (!documents) return;
status = people.write(keys, std::move(documents.value), {}, ustore_docs_modify_merge_k).wait();

std::array<sz::string_view_t, 1> const columns {"person"};
sz::expected<values_tape_t> fields = sz::strings_view_t(columns).gather();
if (!fields) return;
auto gathered = people.gather(keys, std::move(fields.value), std::array {ustore_docs_field_type_string_k}, arena).wait();
auto names = gathered->table.column(0).as<std::string_view>();

Graphs read edges and degrees for one role of every vertex, and write and remove edges and vertices:

deferred_graph_collection_t graph = *later.find<graph_collection_t>("graph");
std::array<edge_t, 2> const chain {edge_t {key_from_integer(1), key_from_integer(2)}, edge_t {key_from_integer(2), key_from_integer(3)}};
status = graph.write_edges(chain).wait();
expected<edges_view_t> leaving = graph.read_edges(keys, arena, ustore_vertex_role_source_k).wait();
expected<std::span<ustore_vertex_degree_t const>> degrees = graph.degrees(keys, arena).wait();

Strings are written and read in batches, and a scan answers a page of strings for every prefix:

deferred_strings_collection_t words = *later.find<strings_collection_t>("words");
std::array<sz::string_view_t, 2> const spellings {"apple", "apricot"};
sz::expected<values_tape_t> strings = sz::strings_view_t(spellings).gather();
sz::expected<values_tape_t> meanings = sz::strings_view_t(texts).gather();
sz::expected<values_tape_t> prefixes = sz::strings_view_t(std::span {spellings}.first(1)).gather();
if (!strings || !meanings || !prefixes) return;
status = words.write(std::move(strings.value), std::move(meanings.value)).wait();
expected<strings_scan_t::finished_t> scanned = words.scan(std::move(prefixes.value), 16, arena).wait();

A transaction defers its own verbs too, reset among them:

deferred_transaction_t deferred_transaction = transaction->deferred(executor);
status = deferred_transaction.find("blobs")->remove(keys).wait();
expected<read_stamp_t> staged = deferred_transaction.stage().wait();
expected<commit_stamp_t> stamp = deferred_transaction.commit().wait();

The deferred database also creates, drops and lists collections, and describes, clears and compacts itself, while the deferred snapshot saves itself through save. Opening blocks instead, as every constructor does: init_ram, open_ram, open_rocksdb, view_rocksdb and open_leveldb take the executor to block on as their last argument, while open_nvme blocks on the executor its options name and reads how to move bytes from it once:

expected<database_t> opened = database_t::open_rocksdb({.directory = bytes_of("/var/lib/ustore")}, executor);

Either way the database runs its own calls on the calling thread until a receiver names an executor.

• Deferred inputs are owned, arenas borrowed. Keys, vertices, edges and column types come as a std::array or an st::vector, and values, strings and fields as a values_tape_t, StringZilla’s sz::strings_t that sz::strings_view_t::gather fills, so a borrowed span or view does not compile. The arena comes as an arena_t&, borrowed until the call is final, which is safe because freeing an arena waits for the deferred calls naming it, while a temporary arena does not compile.
• Submission copies the arguments. A deferred<verb> keeps one heap block that never moves, holding the outcome, the owned inputs and the outputs the call writes through, while the C argument structure dies on the stack it was filled on. status() polls through ustore_outcome_status, and wait() and the destructor wait.
• Answers view the arena. A finished read answers the views a blocking one does, valid until the next call given that arena, and a finished gather owns its rows, columns and field names beside a table whose cells stay in the arena.
• A transaction serves one call at a time. Wait on one deferred call through it before submitting the next, which otherwise finishes at once with status_code_t::contended_k.
• The wrappers have no life of their own. They start no threads, keep no counts and hold no hidden arena: freeing a database, transaction or snapshot waits in C for the deferred calls naming it.