# Platform Notes

> UStore for Python

### Layout

- `lib.c`: the module glue of `ustore.lib`.
- `lib.h`: declarations every translation unit shares, with the arena and ragged-input rules.
- `types.c`: failures, enums, arenas, ragged inputs and result tapes, executors and deferred calls.
- `db.c`: databases, collections, snapshots, transactions and views.
- `blobs.c`: binary values and their cursors.
- `docs.c`: JSON documents and typed column gathers.
- `graph.c`: graph edges and vertices.
- `arrow.c`: Apache Arrow streams: collection exports, imports and ingests.
- `strings.c`: collection names, resolved onto identifiers and listed in text order.
- `collection.c`: collections addressed by name, and the mapping views each answers.
- `CMakeLists.txt`: the `ustore_python` target, installed into the `ustore` package.
- `ustore/__init__.py`: the package, building the enums and re-exporting `ustore.lib`.
- `ustore/lib.pyi`: type stubs for the extension.
- `ustore/keys.py`: walks every key of a collection through a cursor.
- `ustore/graph_view.py`: NetworkX-compatible graphs over the graph and document verbs.
- `ustore/table_view.py`: a Pandas table view over document collections.
- `ustore/py.typed`: the PEP 561 marker that ships the stubs.

Each modality joins as one C file beside `lib.c`, with its verbs declared in `lib.h` and stubbed in `ustore/lib.pyi`.
The suites live in `test/` as `<topic>.py` beside the C++ ones, and benchmarks in `bench/`.

### What This Binding Does Not Expose

- The strings cursor, which `export_arrow` drives.
- Deferred constructors: an open takes `executor=` and blocks on it, so a database exists once it answers.
- Deferred cursors: every pull blocks, so a deferred view opens none.
- `ustore_status_name` and the other `_name` and `_from_name` functions: the module builds every enum through them at import, so `Status["WRITE_CONFLICT"]` parses a name and no string ever reaches a verb.

### Design

The binding is a thin layer over the C ABI in `c/ustore/`, and every rule below follows from that.

- __Blocking calls stay as simple as the language allows.__
  `db.read(keys, arena)` takes no executor, completion or view argument and returns its result; nothing about asynchrony reaches a caller who never asks for it.
- __The receiver chooses, the verb stays.__
  What every C argument struct puts before its inputs, the view, the executor and the completion, is chosen by the object a verb is called on: `db.on(executor)`, `db.deferred(executor)`, `snapshot`, `txn.unwatched()`.
  A deferred view offers every verb a blocking view does, with the same arguments, and answers a `ustore.Deferred` instead of the result.
  Opening has no database yet, so the constructors take `executor=` and block on it, and the NVMe engine, whose init keeps the executor it is handed, opens on the calling thread alone.
- __Every verb is one C call, so every verb defers.__
  Through `db.deferred(executor)`, these answer a `Deferred`:
  - `read`, `get`, `contains`, `write`, `set`, `remove`, `scan`, `scan_batch`, `sample`, `sample_batch` and `measure`;
  - `docs_read`, `docs_write`, `docs_gist`, `docs_gather`, `docs_scan`, `docs_sample`, `docs_measure` and `docs_remove`;
  - `pages_scan`, `pages_sample`, `pages_measure` and `pages_remove`;
  - `graph_scan_vertices`, `graph_sample_vertices`, `graph_measure_vertices`, `graph_read_edges`, `graph_degrees`, `graph_write_edges`, `graph_remove_edges`, `graph_write_vertices` and `graph_remove_vertices`;
  - `collection_find`, `collection_list`, `commit`, `stage` and a snapshot's `save`;
  - `clear`, `compact` and `describe` of a database;
  - `view[name].blobs[key]`, `view[name].docs[key]`, `view[name].docs_json[key]`, `view[name].scan`, `view[name].sample_keys`, and `has_node` and `read_edges` of `view[name].graph`.

  What Python answers itself, or what takes several dependent calls, cannot be one `Deferred`.
  So `in`, an assignment, `del`, `view[name].sample`, `view[name].table` and every other method of `view[name].graph` raise `TypeError` naming the verb to call instead.
  A cursor's pulls each block, so a deferred view opens none.
- __Deferred calls use the language's own idioms.__
  `status` polls through `ustore_outcome_status`, `wait` blocks through `ustore_outcome_wait`, and `await` hands `wait()` to the running event loop's default executor.
- __Ragged inputs keep their layout.__
  An Arrow array or a pair of NumPy buffers passes as the u32 or u64 tape it is, and anything else is borrowed into views the call pins at submission, which the C accessor reads on the executor's threads.
- __Arenas are lent and reused, never hidden.__
  Every output whose length scales with the call lands in an arena the caller passes, and no handle keeps one of its own.
  An exported buffer pins its arena until released, and a view over a rewound one raises `UninitializedError`.
- __Shorthand is slower, verbs reuse an arena.__
  Syntax with no place for an arena makes a one-off arena per call and copies out of it, which costs an allocation and a copy that a verb handed a reused arena never pays.
- __Deferral costs nothing until used.__
  A blocking call allocates no handle and keeps its call on the stack; a deferred call allocates one `Deferred` that owns every input, output slot and arena reference the library reads until the call finishes.
- __The binding has no life of its own.__
  It starts no threads, keeps no queues and counts no deferred calls, transactions, snapshots or cursors.
  The C executor owns every thread, freeing a database, transaction, snapshot or arena waits in C for the deferred calls naming it, a transaction, snapshot or cursor holds its database open in C, so `db.close()` never refuses, and `await` borrows the event loop's own executor.
  A progress callable runs on whichever thread the library reports from, attaching the interpreter there itself.
- __Statuses and modes are enums.__
  `ustore.Status`, `ustore.Mood`, `ustore.Watch`, `ustore.Durability` and the rest mirror the C enumerators value for value, so nothing is ever compared as a string.
- __Language idioms over parallel APIs.__
  Mapping sugar, context managers, the buffer protocol, struct sequences and Arrow export work on every receiver that can honour them.

`import ustore`

## PageEncoding

How a page or a fragment is spelled on import or rendered on read, valued as `ustore_page_encoding_t`.

```python
class PageEncoding(IntEnum)
```

- `HTML` = `0`
- `XML` = `1`
- `MARKDOWN` = `2`
- `TEXT` = `3`
- `JSON` = `4`