Inputs and Outputs

§ Arenas

Every output whose length only the call knows, such as matches, rows, products, monoliths and cursor or stream batches, lands in an Arena you pass, and the verb answers a view over it without copying. There is one verb per operation: no allocating twin, no caller buffers, and no default arena. Arena(on: executor) places it where that executor computes, decided once, and Arena() makes host pages for the calling thread. Each call rewinds its arena, so the previous call’s views die and a steady loop allocates once it reaches its high-water mark. To keep a result past the next call, or to read one while making the next, pass that call another arena. An arena serves one call at a time, so each deferred call needs its own. Views stay on the arena’s thread, which the compiler checks, and reading a stale one traps as an out-of-bounds index does. A view cannot be ~Escapable while its lifetime annotation is still behind an experimental feature flag as of Swift 6.4, so it checks its arena at runtime instead. An arena from a VRAM executor holds device memory, which host accessors refuse with a trap, and serves only that executor’s calls, keeping the executor alive until the arena goes.

let arena = try Arena(on: executor)                                              // Arena() is host pages for the calling thread
for batch in batches {
    let matches = try index.on(executor).search(batch, wanted: 10, arena: arena) // rewinds: last batch's matches are gone
    total += matches.joined().reduce(0) { $0 + $1.distance }
}
let found = try Arena(), fetched = try Arena()                                   // two live results, two arenas
for match in try index.search(query, wanted: 10, arena: found)[0] {
    rerank(match.key, try index.get(key: match.key, as: Float.self, arena: fetched))
}
let calls = try shards.indices.map { at in
    try shards[at].deferred(executors[at]).search(queries, wanted: 10, arena: arenas[at]) // one executor and one arena per submitted call
}
var results: [Matches] = []
for call in calls { results.append(try await call.value) }                       // or: try call.wait()

A result keeps the arena it reads alive, so it outlives the index and the caller’s reference alike.

§ Rows & Scalars

A RowsView borrows a NumKong MatrixView, or keeps a NumKong Tensor alive. A matrix view borrows and never owns, so it stays valid only for as long as the storage it points into, which is why a call over one sits inside a withUnsafeBufferPointer.

let padded: [Float] = load()                                               // 768 dimensions pitched at 1024
try padded.withUnsafeBufferPointer { buffer in
    let rows = MatrixView(baseAddress: buffer.baseAddress!, rows: 1_000, columns: 768, rowStrideBytes: 1024 * 4)
    try index.on(executor).insert(keys: keys, vectors: RowsView(rows))
}

The library checks every row view’s width against the index geometry itself, so a mismatch surfaces as its own shapeMismatch. Sub-byte kinds count logical dimensions rather than stored values, so a U1x8 view one value wide carries eight dimensions.

The scalar type of what you hand the index is independent of the scalar type it stores: the library converts on ingest and on export. Double, Float, Int8, and UInt8 are native, and the low-precision encodings come from NumKong: BFloat16, E5M2, E4M3, E3M2, E2M3 and U1x8, each of which converts from a Float rather than only naming a bit pattern.

let row = [BFloat16(float: 0.3), BFloat16(float: 0.5), BFloat16(float: 1.2)]

Scalar, DenseMetric and Mood spell themselves through the library’s own name tables, and Scalar(name: "bf16") reads one back.

§ Search

A search writes up to wanted matches per query into its arena, ranked nearest first, and answers Matches reading them in place: matches[query] is one query’s QueryMatches, so neither level copies a match out. A query’s own count stops at the matches the call wrote, however many were wanted. Both levels are RandomAccessCollection, so a query walks, maps and slices like any other sequence.

let stats = DenseTraversalStats()
let matches = try index.on(executor).search(queries, wanted: 10, arena: arena, config: DenseSearchConfig(
    expansionFactor: 64, stats: stats))
for query in 0 ..< matches.count {
    for match in matches[query] { print(match.key, match.distance) }
}
print(stats.visitedMembers, stats.computedDistances, stats.restarts)

expansionFactor is the breadth of the candidates the walk explores, and zero derives it from the geometry. The insert, refine, search, maxsim, multiply and attend configs take a DenseTraversalStats, and every call overwrites the stats it is handed rather than accumulating, so a sweep sums them itself. Insertions and searches overlap freely on one handle: searches read only entries published behind the visibility watermark, while insertions publish in order.