Platform Notes

§ Layout

• lib.c: the module glue of ustore.lib.
• lib.h: declarations every translation unit shares, with the arena and ragged-input rules.
• types.c: failures, enums, arenas, ragged inputs and result tapes, executors and deferred calls.
• db.c: databases, collections, snapshots, transactions and views.
• blobs.c: binary values and their cursors.
• docs.c: JSON documents and typed column gathers.
• graph.c: graph edges and vertices.
• arrow.c: Apache Arrow streams: collection exports, imports and ingests.
• strings.c: collection names, resolved onto identifiers and listed in text order.
• collection.c: collections addressed by name, and the mapping views each answers.
• CMakeLists.txt: the ustore_python target, installed into the ustore package.
• ustore/__init__.py: the package, building the enums and re-exporting ustore.lib.
• ustore/lib.pyi: type stubs for the extension.
• ustore/keys.py: walks every key of a collection through a cursor.
• ustore/graph_view.py: NetworkX-compatible graphs over the graph and document verbs.
• ustore/table_view.py: a Pandas table view over document collections.
• ustore/py.typed: the PEP 561 marker that ships the stubs.

Each modality joins as one C file beside lib.c, with its verbs declared in lib.h and stubbed in ustore/lib.pyi. The suites live in test/ as <topic>.py beside the C++ ones, and benchmarks in bench/.

§ What This Binding Does Not Expose

• The strings cursor, which export_arrow drives.
• Deferred constructors: an open takes executor= and blocks on it, so a database exists once it answers.
• Deferred cursors: every pull blocks, so a deferred view opens none.
• ustore_status_name and the other _name and _from_name functions: the module builds every enum through them at import, so Status["WRITE_CONFLICT"] parses a name and no string ever reaches a verb.

§ Design

The binding is a thin layer over the C ABI in c/ustore/, and every rule below follows from that.

• Blocking calls stay as simple as the language allows. db.read(keys, arena) takes no executor, completion or view argument and returns its result; nothing about asynchrony reaches a caller who never asks for it.

• The receiver chooses, the verb stays. What every C argument struct puts before its inputs, the view, the executor and the completion, is chosen by the object a verb is called on: db.on(executor), db.deferred(executor), snapshot, txn.unwatched(). A deferred view offers every verb a blocking view does, with the same arguments, and answers a ustore.Deferred instead of the result. Opening has no database yet, so the constructors take executor= and block on it, and the NVMe engine, whose init keeps the executor it is handed, opens on the calling thread alone.

• Every verb is one C call, so every verb defers. Through db.deferred(executor), these answer a Deferred:

  • read, get, contains, write, set, remove, scan, scan_batch, sample, sample_batch and measure;
  • docs_read, docs_write, docs_gist, docs_gather, docs_scan, docs_sample, docs_measure and docs_remove;
  • pages_scan, pages_sample, pages_measure and pages_remove;
  • graph_scan_vertices, graph_sample_vertices, graph_measure_vertices, graph_read_edges, graph_degrees, graph_write_edges, graph_remove_edges, graph_write_vertices and graph_remove_vertices;
  • collection_find, collection_list, commit, stage and a snapshot’s save;
  • clear, compact and describe of a database;
  • view[name].blobs[key], view[name].docs[key], view[name].docs_json[key], view[name].scan, view[name].sample_keys, and has_node and read_edges of view[name].graph.

  What Python answers itself, or what takes several dependent calls, cannot be one Deferred. So in, an assignment, del, view[name].sample, view[name].table and every other method of view[name].graph raise TypeError naming the verb to call instead. A cursor’s pulls each block, so a deferred view opens none.

• Deferred calls use the language’s own idioms. status polls through ustore_outcome_status, wait blocks through ustore_outcome_wait, and await hands wait() to the running event loop’s default executor.

• Ragged inputs keep their layout. An Arrow array or a pair of NumPy buffers passes as the u32 or u64 tape it is, and anything else is borrowed into views the call pins at submission, which the C accessor reads on the executor’s threads.

• Arenas are lent and reused, never hidden. Every output whose length scales with the call lands in an arena the caller passes, and no handle keeps one of its own. An exported buffer pins its arena until released, and a view over a rewound one raises UninitializedError.

• Shorthand is slower, verbs reuse an arena. Syntax with no place for an arena makes a one-off arena per call and copies out of it, which costs an allocation and a copy that a verb handed a reused arena never pays.

• Deferral costs nothing until used. A blocking call allocates no handle and keeps its call on the stack; a deferred call allocates one Deferred that owns every input, output slot and arena reference the library reads until the call finishes.

• The binding has no life of its own. It starts no threads, keeps no queues and counts no deferred calls, transactions, snapshots or cursors. The C executor owns every thread, freeing a database, transaction, snapshot or arena waits in C for the deferred calls naming it, a transaction, snapshot or cursor holds its database open in C, so db.close() never refuses, and await borrows the event loop’s own executor. A progress callable runs on whichever thread the library reports from, attaching the interpreter there itself.

• Statuses and modes are enums. ustore.Status, ustore.Mood, ustore.Watch, ustore.Durability and the rest mirror the C enumerators value for value, so nothing is ever compared as a string.

• Language idioms over parallel APIs. Mapping sugar, context managers, the buffer protocol, struct sequences and Arrow export work on every receiver that can honour them.