Inputs and Outputs

§ Arenas

Every output whose length only the call knows, such as matches, rows, products, monoliths and cursor or stream batches, lands in an Arena you pass, and the verb answers a view over it without copying. Arena::new_in(&executor) places it where that executor computes, decided once, and Arena::new() makes host pages for the calling thread. Each call rewinds its arena, so the previous call’s views die and a steady loop allocates once. To keep a result past the next call, or to read one while making the next, pass that call another arena. An arena serves one call at a time, so each deferred call needs its own. Every call, blocking or deferred, borrows the arena as &mut Arena and its result borrows it in turn, so the borrow checker refuses a stale view at compile time and keeps a call’s inputs out of the arena it rewinds. An arena from a VRAM executor holds device memory, which host accessors refuse with Status::Device. It grows, fills and frees on that executor’s first stream, so it serves only calls on that executor, any other refused with Status::Device, and holds the executor until it drops.

let mut arena = Arena::new_in(&executor)?;                           // Arena::new() is host pages for the calling thread
for batch in &batches {
    let matches = index.on(&executor).search(batch, 10, &mut arena)?; // rewinds: the last batch's matches are gone
    total += matches.query(0)?.distances.iter().sum::<f64>();
}

let (mut found, mut fetched) = (Arena::new()?, Arena::new()?);       // two live results, two arenas
for &key in index.search(&query, 10, &mut found)?.query(0)?.keys {
    rerank(key, index.get::<f32>(key, &mut fetched)?.view()?);       // rewinds fetched, never found
}

let (mut executors, mut arenas) = (Vec::new(), Vec::new());          // one executor and one arena per submitted call
for _ in &shards {
    executors.push(Executor::for_ram(&[], None, 8)?);
}
for executor in &executors {
    arenas.push(Arena::new_in(executor)?);
}
let mut calls = Vec::new();
for ((shard, executor), arena) in shards.iter().zip(&executors).zip(&mut arenas) {
    calls.push(shard.deferred(executor).search(queries.clone().expect("room for a copy"), 10, arena)?);
}
for call in calls {
    consume(call.await?);                                            // or: call.wait()?
}

The library never frees an arena under a call: dropping one waits for the calls writing into it, and a call naming an arena that serves another call finishes with Status::Contended and rewinds nothing, so even a leaked Deferred leaves it safe. An arena’s describe() says where it lives, and on which device. Fixed-size reports, such as stats, descriptions and required, stay plain values, and so does state you carry between calls, like the attention carry.

§ Rows & Scalars

rows borrows a packed slab; a dataset whose rows are padded to a wider pitch passes MatrixView::from_raw_parts instead, which takes the byte stride directly. The scalar type of the input is independent of the stored quantization, so the f32 slab of the Quickstart lands in an i8 index with nothing converted beforehand. The scalar types are NumKong’s, so nothing needs re-wrapping to move between the two crates, f16 and bf16 included, emulated where the CPU has no native support.

Initializing an index with a packed quantization and inserting floating-point vectors quantizes on ingest; the u1x8 type also lets you pass packed bit-vectors directly.

use numkong::u1x8;

let options = DenseOptions {
    dimensions: 8,
    metric: DenseMetric::Hamming,
    quantization: Scalar::U1x8,
    ..DenseOptions::default()
};
let index = DenseIndex::init_ram(&options, &[])?;
let slab = [u1x8(0b0000_1111), u1x8(0b1111_0000)];
index.insert(&keys, &rows(&slab, 8)?)?;

An insert batch is all-or-nothing and becomes visible at once, and it overlaps safely with concurrent searches on the same handle.

§ Search

Each configurable verb’s config converts into its Dense*Config struct, whose other fields are named and default to what the C interface derives. So the bare value is the whole call, as in search(&queries, 10, &mut arena), insert(&keys, &vectors) or save(path), and ..DenseSearchConfig::from(10) reaches every knob.

let mut arena = Arena::new_in(&executor)?;
let mut stats = DenseTraversalStats::default();
let matches = index.on(&executor).search(&rows(&queries, 768)?, DenseSearchConfig {
    expansion_factor: 128,
    stats: Some(&mut stats),
    ..DenseSearchConfig::from(10)
}, &mut arena)?;
println!("{} distances computed", stats.computed_distances);

query pairs one query’s keys with its distances, nearest first and trimmed to the cells the call wrote, and iter walks every query in order.

for query in matches.iter()? {
    for (key, distance) in query.keys.iter().zip(query.distances) {
        println!("{key} at {distance}");
}
}

The breadth of the candidates rides the call rather than the handle, as expansion_factor on every config that walks the graph, so a recall sweep is a loop over ef values against one immutable index, with no mutation and no synchronization between points. traversal: Traversal::Scan scores every entry instead of walking, on RAM and NVMe handles, which gives that sweep its exact ground truth; a VRAM handle refuses it with Status::Unsupported. DenseTraversalStats reports the members visited, the distances computed and the starvation restarts, a graph-connectivity signal rather than a cost, overwritten by every call it is passed to.