# Batches and Arenas

> UStore for Mojo

### Receivers & Deferred Calls

The receiver chooses where a verb runs, what it sees and how it completes, and the verb stays the same.
A database, transaction or snapshot runs a verb on the calling thread, `handle.on(executor)` blocks on an executor, and `handle.deferred(executor)` submits it there and answers a `Deferred`:

```mojo
db.write(collections, keys, values)                       # the head, on the calling thread
db.on(executor).write(collections, keys, values)          # blocking, on the executor
var call = db.deferred(executor).read(collections, keys, arena)  # Deferred[...]
print(call.wait().present(0))
```

`On` and `DeferredOn` offer the same verbs with the same arguments, and the blocking verbs are written once, in the `BlobsVerbs`, `StringsVerbs`, `DocsVerbs`, `GraphVerbs`, `PagesVerbs` and `DatabaseVerbs` traits every blocking receiver conforms to.
The verb names follow the C ones: blobs verbs are unprefixed, as `write` and `scan`, and the other modalities carry theirs, as `strings_scan`, `docs_gather`, `graph_read_edges` and `pages_insert`.
Reads see the head of a database, a transaction, or a snapshot; writes join the receiver's transaction, recording their keys unless it came from `txn.unwatched()`, and a snapshot's receiver, which has none, writes at the head.
`stage`, `commit`, `rollback` and `reset` act on the receiver's transaction and `save` on its snapshot, and the engine answers `Status.UNINITIALIZED` for a receiver without one.

A `Deferred` owns its outcome and outputs until the call is final, and borrows the arena and every input it was handed, which Mojo keeps alive for as long as the `Deferred` is; freeing the database, the executor or a transaction waits for the calls naming them.
Mojo has no runtime of its own to await on, so `wait` is the whole interface: it blocks in `ustore_outcome_wait` until the call is final, then answers the result or raises the failure, as many times as it is asked.
`status` reads the outcome without blocking, and destroying an unfinished `Deferred` waits for it.

### Arenas

Every output whose length only the call knows, values, keys, offsets, bounds, columns and edges, lands in an `Arena` you pass, and the verb answers a view over it without copying.
`Arena(executor)` places it where that executor computes, decided once, and `Arena(library)` makes host pages for the calling thread.
Each call rewinds its arena, so the previous call's views die and a steady loop allocates once; reading a stale one raises `Status.INVALID_ARGUMENT`.
To keep a result past the next call, pass that call another arena, and give every deferred call its own.

### Progress & Cancellation

Scans take their start keys and how many keys every scan keeps, plus an optional config for the end keys, the direction, the patterns or filter, the predicate and the progress handler:

```mojo
def stop_at_once(finished: Int, total: Int, seen: Pointer[Int, MutAnyOrigin]) -> Continuation:
    seen[] = finished
    return Continuation.CANCEL

def admit_even(keys: Span[Key, MutAnyOrigin], admissions: Span[Admission, MutAnyOrigin], state: Pointer[Int, MutAnyOrigin]):
    for index in range(len(keys)):
        admissions[index] = Admission.ADMIT if keys[index].uuid.bytes[15] % 2 == 0 else Admission.REJECT

var seen = 0
var config = BlobsScanConfig(predicate=Predicate.of[admit_even](Pointer(to=seen)))
var runs = db.scan(CollectionsView.repeating(people), KeysView.repeating(start), 100, arena, config)
_ = db.scan(collections, starts, 100, arena, BlobsScanConfig(progress=Progress.of[stop_at_once](Pointer(to=seen))))  # raises Status.CANCELLED
```

Handlers are compile-time parameters, since a capturing closure cannot cross into C, and their state crosses beside them as a pointer that must outlive the call.
Progress handlers return a value, which Mojo 1.2 is confirmed to pass back on Linux x86-64; Linux AArch64 and macOS rest on the CI matrix.

`from ustore import On, DeferredOn, Completion, …`

## On

A database, transaction or snapshot bound to the executor its verbs block on. Binding-only.

```mojo
struct On[origin: ImmOrigin]
```

- `origin` (comptime): The handle and the executor, which this receiver borrows.

- `scope` `Scope`: What every verb addresses.

Conforms to: `BlobsVerbs`, `Copyable`, `DatabaseVerbs`, `Deinitable`, `DocsVerbs`, `GraphVerbs`, `ImplicitlyCopyable`, `Movable`, `PagesVerbs`, `StringsVerbs`

## DeferredOn

A database, transaction or snapshot bound to the executor its verbs are submitted to: every verb answers a [`Deferred`](#Deferred) at once, borrowing its arena and inputs until final. Binding-only.

```mojo
struct DeferredOn[origin: ImmOrigin]
```

The verbs and their arguments are those of the blocking receivers.

- `origin` (comptime): The handle and the executor, which this receiver borrows.

- `scope` `Scope`: What every verb addresses.

Conforms to: `BlobsDeferredVerbs`, `Copyable`, `DatabaseDeferredVerbs`, `Deinitable`, `DocsDeferredVerbs`, `GraphDeferredVerbs`, `ImplicitlyCopyable`, `Movable`, `PagesDeferredVerbs`, `StringsDeferredVerbs`

## Completion

Whether a verb returns with its final status or once submitted, `ustore_completion_t`.

```mojo
struct Completion
```

The receiver picks it: a handle and `handle.on(executor)` pass [`BLOCKING`](#BLOCKING), and
`handle.deferred(executor)` passes [`DEFERRED`](https://www.unum.cloud/docs/ustore/mojo/errors.md#DEFERRED), so no verb takes it as an argument.

- `BLOCKING` = `Completion(Int32(0))`: Return with the final status stored and every output readable.
- `DEFERRED` = `Completion(Int32(1))`: Return once submitted; the status reads [`DEFERRED`](https://www.unum.cloud/docs/ustore/mojo/errors.md#DEFERRED) until final.
- `code` `c_int`: The raw `ustore_completion_t` value the ABI exchanges.

Conforms to: `Copyable`, `Deinitable`, `Equatable`, `ImplicitlyCopyable`, `Movable`, `RegisterPassable`, `TrivialRegisterPassable`

## Mood

What an executor's threads or rings do once it goes idle, `ustore_mood_t`.

```mojo
struct Mood
```

- `PARKED` = `Mood(Int32(0))`: Parked, so the next call pays a wake-up.
- `SPINNING` = `Mood(Int32(1))`: Spinning, trading power for the next call's latency.
- `code` `c_int`: The raw `ustore_mood_t` value the ABI exchanges.

Conforms to: `Copyable`, `Deinitable`, `Equatable`, `ImplicitlyCopyable`, `Movable`, `RegisterPassable`, `TrivialRegisterPassable`

## CPUClass

The CPU generation a database or an executor confines itself to, `ustore_cpu_class_t`.

```mojo
struct CPUClass
```

- `UNKNOWN` = `CPUClass(Int32(0))`: Whatever the running CPU offers.
- `SERIAL` = `CPUClass(Int32(1))`: The portable floor every CPU runs.
- `INTEL_HASWELL` = `CPUClass(Int32(2))`: x86 cores from Intel Haswell on.
- `INTEL_SIERRA` = `CPUClass(Int32(3))`: x86 efficiency cores from Intel Sierra Forest on.
- `INTEL_SKYLAKE` = `CPUClass(Int32(4))`: x86 server cores from Intel Skylake on.
- `INTEL_ICELAKE` = `CPUClass(Int32(5))`: x86 server cores from Intel Ice Lake on.
- `AWS_G2` = `CPUClass(Int32(6))`: Arm cores from AWS Graviton 2 on.
- `APPLE_M2` = `CPUClass(Int32(7))`: Arm cores from Apple M2 on.
- `AWS_G5` = `CPUClass(Int32(8))`: Arm server cores from AWS Graviton 5 on.
- `ALIBABA_C930` = `CPUClass(Int32(9))`: RISC-V cores with RVV, Zawrs and Zacas, from Alibaba's C930 on.
- `MONOVERSIONED` = `CPUClass(Int32(10))`: The one class of a monoversioned build.
- `code` `c_int`: The raw `ustore_cpu_class_t` value the ABI exchanges.

Conforms to: `Copyable`, `Deinitable`, `Equatable`, `ImplicitlyCopyable`, `Movable`, `RegisterPassable`, `TrivialRegisterPassable`

## Transport

How the NVMe engine moves bytes between its files and memory, `ustore_transport_t`.

```mojo
struct Transport
```

- `AUTOMATIC` = `Transport(Int32(0))`: io_uring where the kernel offers it, Linux AIO where it offers only that, and blocking calls elsewhere.
- `BLOCKING` = `Transport(Int32(1))`: Positioned reads and writes that block the polling thread.
- `LINUX_URING` = `Transport(Int32(2))`: io_uring on unbuffered files.
- `LINUX_AIO` = `Transport(Int32(3))`: Linux AIO on unbuffered files.
- `LINUX_SPDK` = `Transport(Int32(4))`: SPDK's user-space driver on NVMe devices the process owns.
- `code` `c_int`: The raw `ustore_transport_t` value the ABI exchanges.

Conforms to: `Copyable`, `Deinitable`, `Equatable`, `ImplicitlyCopyable`, `Movable`, `RegisterPassable`, `TrivialRegisterPassable`

## Gds

How a VRAM executor moves bytes between the drives and its device, `ustore_gds_t`.

```mojo
struct Gds
```

- `AUTOMATIC` = `Gds(Int32(0))`: Direct transfers where the driver is loaded, and the compatibility mode elsewhere.
- `DIRECT` = `Gds(Int32(1))`: Direct transfers, refused with [`DEVICE`](https://www.unum.cloud/docs/ustore/mojo/errors.md#DEVICE) where the driver is absent.
- `COMPATIBILITY` = `Gds(Int32(2))`: The compatibility mode, which bounces through host pages it manages.
- `code` `c_int`: The raw `ustore_gds_t` value the ABI exchanges.

Conforms to: `Copyable`, `Deinitable`, `Equatable`, `ImplicitlyCopyable`, `Movable`, `RegisterPassable`, `TrivialRegisterPassable`

## GPUVendor

The GPU vendors one binary drives at once, `ustore_gpu_vendor_t`.

```mojo
struct GPUVendor
```

- `UNKNOWN` = `GPUVendor(Int32(0))`: No vendor.
- `NVIDIA` = `GPUVendor(Int32(1))`: NVIDIA, through the CUDA driver.
- `AMD` = `GPUVendor(Int32(2))`: AMD, through the HIP runtime.
- `code` `c_int`: The raw `ustore_gpu_vendor_t` value the ABI exchanges.

Conforms to: `Copyable`, `Deinitable`, `Equatable`, `ImplicitlyCopyable`, `Movable`, `RegisterPassable`, `TrivialRegisterPassable`

## GPUClass

The GPU generations UStore tells apart, `ustore_gpu_class_t`.

```mojo
struct GPUClass
```

- `UNKNOWN` = `GPUClass(Int32(0))`: No class.
- `NVIDIA_AMPERE` = `GPUClass(Int32(1))`: Ampere-class NVIDIA and every NVIDIA GPU without a class of its own.
- `NVIDIA_ADA` = `GPUClass(Int32(2))`: Ada-class NVIDIA.
- `NVIDIA_HOPPER` = `GPUClass(Int32(3))`: Hopper-class NVIDIA.
- `NVIDIA_BLACKWELL` = `GPUClass(Int32(4))`: Blackwell-class NVIDIA datacenter parts.
- `NVIDIA_BLACKWELLRTX` = `GPUClass(Int32(5))`: Consumer Blackwell NVIDIA.
- `AMD_CDNA3` = `GPUClass(Int32(6))`: CDNA3 AMD, gfx942.
- `AMD_CDNA4` = `GPUClass(Int32(7))`: CDNA4 AMD, gfx950.
- `AMD_CDNA5` = `GPUClass(Int32(8))`: CDNA5 AMD, gfx1250.
- `code` `c_int`: The raw `ustore_gpu_class_t` value the ABI exchanges.

Conforms to: `Copyable`, `Deinitable`, `Equatable`, `ImplicitlyCopyable`, `Movable`, `RegisterPassable`, `TrivialRegisterPassable`

## GPUDevice

One GPU as UStore numbers it, `ustore_gpu_device_t`: its position in [`Library.gpu_devices`](https://www.unum.cloud/docs/ustore/mojo/platform-notes.md#gpu_devices) is the ordinal a VRAM executor names it by.

```mojo
struct GPUDevice
```

- `vendor` `GPUVendor`: The vendor whose runtime drives the device.
- `vendor_ordinal` `UInt32`: The device's ordinal in its vendor's runtime.
- `gpu_class` `GPUClass`: The class of the device.
- `compute_units` `UInt32`: Streaming multiprocessors or compute units.
- `memory_bytes` `UInt64`: Device memory in bytes.

Conforms to: `Copyable`, `Defaultable`, `Deinitable`, `Movable`

## __init__

An empty entry for the listing to fill.

```mojo
def __init__(out self)
```

## name

The vendor's own name for the device.

```mojo
def name(self) -> String
```

## Continuation

Whether the call a progress handler is watching carries on or unwinds, `ustore_continuation_t`.

```mojo
struct Continuation
```

- `CONTINUE` = `Continuation(Int32(0))`: Carry on to the next unit of work.
- `CANCEL` = `Continuation(Int32(1))`: Stop, publish nothing, and finish the call with [`CANCELLED`](https://www.unum.cloud/docs/ustore/mojo/errors.md#CANCELLED).
- `code` `c_int`: The raw `ustore_continuation_t` value the ABI exchanges.

Conforms to: `Copyable`, `Deinitable`, `Equatable`, `ImplicitlyCopyable`, `Movable`, `RegisterPassable`, `TrivialRegisterPassable`

## Progress

How a long call reports its advance: the `progress` and `progress_state` pair of every verb that takes one. The handler never runs concurrently with itself, nor after its call is final.

```mojo
struct Progress
```

- `callback` `Address`: The handler's C entry point; `NULL_ADDRESS` skips every report.
- `state` `Address`: Forwarded into every report; it must outlive the call.

Conforms to: `Copyable`, `Defaultable`, `Deinitable`, `ImplicitlyCopyable`, `Movable`

## __init__

No handler: the call reports nothing.

```mojo
def __init__(out self)
```

## of

Reports to `handler` with `state`, which returns [`Continuation.CANCEL`](#CANCEL) to stop the call.

```mojo
def of[T: AnyType, origin: MutOrigin, //, handler: def(Int, Int, Pointer[T, MutAnyOrigin]) thin -> Continuation](
    state: Pointer[T, origin]
) -> Self
```

- `T` (comptime): What `state` points at, inferred from it.
- `origin` (comptime): The state's origin, inferred with it.
- `handler` (comptime): Called with the units finished, the units in total, and `state`.
- `state`: Forwarded into every report; it must outlive the call.

## Executor

The threads or rings a verb runs on, freed once it and every device arena it made go out of scope.

```mojo
struct Executor
```

`Executor(library)` owns nothing: every verb handed it runs on the calling thread. Calls on one
executor start in submission order, and a blocking call runs alone.

- `library` `Library`: The loaded engine, held so its symbol table outlives every call made through the handle.
- `address` `Address`: The opaque `ustore_executor_t` every verb hands back to the engine; `NULL_ADDRESS` runs inline.
- `vram` `Bool`: Whether it drives a GPU, so the arenas it makes hold device memory.

Conforms to: `Deinitable`, `Movable`

## __init__

The caller-thread executor, which every verb accepts.

```mojo
def __init__(out self, library: Library)
```

```mojo
def __init__(out self, *, library: Library, address: Int, vram: Bool)
```

Adopts a handle that is already spawned, which is how the factories hand one back.

- `library`: The loaded engine, kept so the shared object stays mapped.

## for_ram

Spawns host threads for the RAM, RocksDB and LevelDB engines. Calls `ustore_executor_for_ram`.

```mojo
def for_ram[ram_origin: ImmOrigin, //](
    library: Library,
    ram: RamOptions[ram_origin] = RamOptions(),
    threads: Int = Int(0),
    deferred_limit: Int = Int(0),
    cpu_class: CPUClass = CPUClass.UNKNOWN
) -> Self
```

- `library`: The loaded engine, kept so the shared object stays mapped.
- `ram`: The memory domains it serves; copied, not borrowed.
- `threads`: Threads to spawn; zero takes 7/8 of the cores those domains carry.
- `deferred_limit`: Deferred calls it holds at once; zero takes a default.
- `cpu_class`: The class its threads confine their waits and atomics to; [`UNKNOWN`](https://www.unum.cloud/docs/ustore/mojo/graphs.md#UNKNOWN) derives.

Raises: [`Error`](https://www.unum.cloud/docs/ustore/mojo/errors.md#Error) if the domains cannot be resolved or the OS refuses the threads.

## on_pool

Schedules onto a borrowed ForkUnion pool, so several libraries in one process share one set of threads. Calls `ustore_executor_on_pool`.

```mojo
def on_pool[ram_origin: ImmOrigin, //](
    library: Library,
    pool: Int,
    ram: RamOptions[ram_origin] = RamOptions(),
    deferred_limit: Int = Int(0)
) -> Self
```

- `library`: The loaded engine, kept so the shared object stays mapped.
- `pool`: A live ForkUnion `fu_pool_t`, which must outlive the executor.
- `ram`: The memory domains it serves; copied, not borrowed.
- `deferred_limit`: Deferred calls it holds at once; zero takes a default.

Raises: [`Error`](https://www.unum.cloud/docs/ustore/mojo/errors.md#Error) if the pool or the domains are refused.

## for_nvme

Spawns threads for the NVMe engine, each submitting and polling its own I/O ring. Calls `ustore_executor_for_nvme`.

```mojo
def for_nvme(
    library: Library,
    nvme: NvmeOptions = NvmeOptions(),
    threads: Int = Int(0),
    deferred_limit: Int = Int(0),
    cpu_class: CPUClass = CPUClass.UNKNOWN
) -> Self
```

- `library`: The loaded engine, kept so the shared object stays mapped.
- `nvme`: How the databases it opens move bytes; copied, not borrowed.
- `threads`: Threads to spawn; zero derives them from the cores.
- `deferred_limit`: Deferred calls it holds at once; zero takes a default.
- `cpu_class`: The class its threads confine their waits and atomics to; [`UNKNOWN`](https://www.unum.cloud/docs/ustore/mojo/graphs.md#UNKNOWN) derives.

Raises: [`Error`](https://www.unum.cloud/docs/ustore/mojo/errors.md#Error) if the transport is unknown or the OS refuses the threads.

## for_vram

Opens one submission stream on `device`, or borrows the caller's, driven by one host thread. Calls `ustore_executor_for_vram`.

```mojo
def for_vram(
    library: Library,
    device: Int = Int(0),
    stream: Int = Int(0),
    scratch_limit: Int = Int(0),
    gds: Gds = Gds.AUTOMATIC,
    deferred_limit: Int = Int(0)
) -> Self
```

- `library`: The loaded engine, kept so the shared object stays mapped.
- `device`: The UStore ordinal of the device, its position in [`Library.gpu_devices`](https://www.unum.cloud/docs/ustore/mojo/platform-notes.md#gpu_devices).
- `stream`: A borrowed stream on that device, which must outlive the executor; null creates one.
- `scratch_limit`: Ceiling on the device scratch a call takes; zero lets it grow.
- `gds`: How reads and compactions move bytes between the drives and the device.
- `deferred_limit`: Deferred calls it holds at once; zero takes a default.

Raises: [`Error`](https://www.unum.cloud/docs/ustore/mojo/errors.md#Error) with [`DEVICE`](https://www.unum.cloud/docs/ustore/mojo/errors.md#DEVICE) if no device carries the ordinal.

## drain

Waits for every call submitted before the drain, then adopts `mood` whenever it next idles. Calls `ustore_executor_drain`.

```mojo
def drain(self, mood: Mood = Mood.PARKED)
```

Raises: [`Error`](https://www.unum.cloud/docs/ustore/mojo/errors.md#Error) for a failure of the executor itself, never one of a drained call.

## Residency

Where an arena's outputs live, `ustore_residency_t`.

```mojo
struct Residency
```

- `UNKNOWN` = `Residency(Int32(0))`: No residency, which only a zeroed description reads as.
- `RAM` = `Residency(Int32(1))`: Host pages, which the calling thread reads in place.
- `VRAM` = `Residency(Int32(2))`: One device's memory, which only that device's code reads.
- `code` `c_int`: The raw `ustore_residency_t` value the ABI exchanges.

Conforms to: `Copyable`, `Deinitable`, `Equatable`, `ImplicitlyCopyable`, `Movable`, `RegisterPassable`, `TrivialRegisterPassable`

## Arena

The memory every output whose length scales with a call lands in, freed when it goes out of scope.

```mojo
struct Arena
```

Made where an executor computes, decided once; nothing is allocated until a call writes into it.
Every call handed it rewinds and refills it, which invalidates the previous call's outputs, so a
steady loop allocates once. One arena serves one call at a time; a second concurrent use is
refused by the engine as [`CONTENDED`](https://www.unum.cloud/docs/ustore/mojo/errors.md#CONTENDED), which leaves the holder's results readable.

A result refuses every read with [`INVALID_ARGUMENT`](https://www.unum.cloud/docs/ustore/mojo/errors.md#INVALID_ARGUMENT) once a later call rewinds its arena, and a
result over a device arena refuses every host read with [`DEVICE`](https://www.unum.cloud/docs/ustore/mojo/errors.md#DEVICE).

Conforms to: `Deinitable`, `Movable`

## __init__

Host pages for calls on the calling thread. Calls `ustore_arena_init`.

```mojo
def __init__(out self, library: Library)
```

```mojo
def __init__(out self, executor: Executor)
```

Memory where `executor` computes, serving the calls of its kind. Calls `ustore_arena_init`.

## __deinit__

Frees the arena and every output in it, then lets a device arena's executor go. Calls `ustore_arena_free`.

```mojo
def __deinit__(deinit self)
```

The executor and the loader reference are surrendered only after the call, for the reason
`_ExecutorHandle` records.

## vram

Whether its memory is a GPU's, which host reads refuse. Binding-only.

```mojo
def vram(self) -> Bool
```

## describe

Where the arena's outputs live, as the library reports it. Calls `ustore_arena_describe`.

```mojo
def describe(self) -> ArenaDescription
```

## Deferred

A call submitted to an executor, which owns its outcome and outputs until it is final. Binding-only.

```mojo
struct Deferred[C: Call]
```

[`status`](#status) reads [`DEFERRED`](https://www.unum.cloud/docs/ustore/mojo/errors.md#DEFERRED) until then, without blocking, and [`wait`](#wait) blocks until the call is
final and answers its result or raises its failure, as many times as it is asked. The result is
the view a blocking call answers, borrowing the same arena. Dropping an unfinished one waits the
same way, so nothing the engine still writes is ever freed under it. The crossing's origins keep
the arena and every borrowed input alive for as long as this is, and freeing the database, the
executor or a transaction waits for the calls naming them.

- `C` (comptime): The verb's crossing, which names the result.

- `library` `Library`: The loaded engine, held so the wait can still reach `ustore_outcome_wait`.
- `slots` `OwnedPointer[CallSlots[C.Outputs]]`: The outcome and outputs the engine writes, on the heap since they outlive the verb.
- `call` `Optional[C]`: What the call lends the engine, until a wait concludes it.
- `answer` `Optional[C.Result]`: The result, once a wait found the call succeeded.
- `failure` `Optional[Error]`: The failure, once a wait found the call failed.
- `generation` `Int`: The stamp `served` answered at submission, so a result waited on late still reads as stale.

Conforms to: `Deinitable`, `Movable`

## __init__

Submits `call` deferred, on the receiver's executor.

```mojo
def __init__(out self, scope: Scope, var call: C)
```

## __deinit__

Waits until the call is final, then frees what it lent. Calls `ustore_outcome_wait`.

```mojo
def __deinit__(deinit self)
```

## status

[`DEFERRED`](https://www.unum.cloud/docs/ustore/mojo/errors.md#DEFERRED) until the call is final, then its final status; never blocks. Calls `ustore_outcome_status`.

```mojo
def status(self) -> Status
```

## wait

Blocks until the call is final, then answers its result or raises its failure, every time it is asked. Calls `ustore_outcome_wait`.

```mojo
def wait(mut self) -> ref[self] C.Result
```

Raises: [`Error`](https://www.unum.cloud/docs/ustore/mojo/errors.md#Error) carrying the call's failure.