Inputs and Outputs

§ Arenas

Every output whose length only the call knows, such as matches, rows, products, monoliths and cursor or stream batches, lands in an Arena you pass, and the verb answers a view over it without copying. Arena(executor) places it where that executor computes, decided once, and Arena(library) makes host pages for the calling thread. Each call rewinds its arena, so the previous call’s views die and a steady loop allocates once. To keep a result past the next call, or to read one while making the next, pass that call another arena. An arena serves one call at a time, so each deferred call needs its own. A view’s origin keeps its arena alive, and reading a stale one raises Status.UNINITIALIZED. An arena from a VRAM executor holds device memory, which host accessors refuse with Status.DEVICE, and serves only that executor’s calls, keeping the executor alive until the arena goes.

var arena = Arena(executor)  # Arena(library) makes host pages for the calling thread
for position in range(len(batches)):
    var matches = index.on(executor).search(batches[position], 10, arena)  # rewinds: the last batch's matches are gone
    total += matches.query_distances(0)[0]

var found = Arena(executor)
var fetched = Arena(executor)  # two live results, two arenas
var hits = index.on(executor).search(query, 10, found)
for key in hits.query_keys(0):
    rerank(key, index.on(executor).get[DType.float32](key, fetched))

var east_executor = Executor.for_ram(library, threads=4)
var west_executor = Executor.for_ram(library, threads=4)  # one executor and one arena per submitted call
var east_arena = Arena(east_executor)
var west_arena = Arena(west_executor)
var east = east_shard.deferred(east_executor).search(queries, 10, east_arena)
var west = west_shard.deferred(west_executor).search(queries, 10, west_arena)
print(east.wait().query_keys(0)[0], west.wait().query_keys(0)[0])

The arena is the verb’s last required argument, before the optional config, on every receiver. A call that finds its arena serving another call finishes with Status.CONTENDED and rewinds nothing, so the holder’s views stay readable. A call that fails for any other reason still rewinds it.

§ Rows & Scalars

Every verb is a batch over strided rows, so a memory-mapped dataset reaches the index with no reshape and no copy, and a single vector is a one-row view. A RowsView borrows a caller-owned run as rows of equal width, and it is the shape every batch argument takes. RowsView.of derives the row count and the byte pitch from a packed run, and the scalar kind from its element type.

var vectors = RowsView.of(embeddings, 768)  # Float32 elements, so ScalarKind.F32

The library converts on ingest and on export, so the scalar kind of the input is independent of the index’s own quantization, and a Float32 query searches an I8 index without a conversion pass of your own. Sub-byte kinds count logical dimensions rather than bytes and name their kind explicitly, so 128 dimensions of U1X8 occupy sixteen bytes per row:

var packed = List[UInt8](unsafe_uninit_length=rows * 16)  # filled before the call
var bits = RowsView.of(packed, 128, ScalarKind.U1X8)

A dataset whose rows are padded to a wider pitch builds the view field by field, with its own stride in bytes.

§ Search

Every verb with optional knobs takes a config named after its C argument structure, such as DenseInsertConfig, DenseSearchConfig or DenseMultiplyConfig, where every field left out derives what the C interface derives. So the bare call takes none, and naming a field reaches a knob:

var stats = DenseTraversalStats()
_ = index.on(executor).search(queries, 10, arena, DenseSearchConfig(expansion_factor=128, stats=Pointer(to=stats)))
print(stats.computed_distances, "distances computed")

A stats slot is overwritten whole by every call it is lent to, so a reused one reads the last call’s figures rather than a running total.

A search answers Matches: a queries-by-wanted grid of keys, the distances paired with them, and one match count per query, all written into the arena. query_keys and query_distances borrow one query’s results, nearest first, trimmed to the count that query reported:

var matches = index.search(queries, 10, arena)
for query in range(len(matches)):
    for rank in range(len(matches.query_keys(query))):
        print(query, matches.query_keys(query)[rank], matches.query_distances(query)[rank])

The breadth of the candidates rides each call as expansion_factor rather than the handle, so a recall sweep runs against one immutable index with no mutation between points. DenseSearchConfig(traversal=Traversal.SCAN) scores every entry instead of walking, on RAM and NVMe handles, which gives that sweep its exact ground truth.