Executors

An index says where its payload lives; an Executor says who walks it. One executor serves any number of handles, so several categories of index share one machine’s cores or devices instead of each claiming a private set. Its kind is fixed by the factory that spawned it and must match every handle it drives; a mismatch reports device.

let hostThreads = try Executor.forRam(memoryDomains: [0, 1], threads: 0)
let submissionRings = try Executor.forNvme(paths: ["/mnt/nvme0"], threads: 8)
let deviceStreams = try Executor.forVram(devices: [0, 1], streams: nil, scratchLimit: 1 << 30)

Each factory takes every argument its C constructor has, and only those. forVram can borrow your CUDA streams, one per device, or cap per-device scratch. A thread count of zero derives from the hardware: 7/8 of the cores those memory domains carry, since full occupancy starves the ordered publication path, and for NVMe the devices’ queue capabilities. forVram takes no thread count at all: one host thread drives one stream per device, so there is no count to name.

One call holds an executor at a time, and index handles stay freely shareable. Executor.onPool(memoryDomains:cpuClass:pool:) borrows an existing ForkUnion pool instead of spawning threads, and drain(mood:) waits until no call holds the executor and picks whether its threads park or spin once idle.

§ Receivers & Deferred Calls

The receiver chooses where a verb runs and how it completes, and the verb stays the same. Called on the index, a verb runs on the calling thread; on index.on(executor) it blocks on that executor, refused while another call holds it; on index.deferred(executor) it submits there and answers a Deferred at once. DenseIndex, DenseIndexOn and DenseIndexDeferredOn take each verb with the same arguments, and a blocking call takes no executor or completion, with a default config deriving every knob.

let here = try index.search(queries, wanted: 10, arena: arena)                  // the calling thread
let blocking = try index.on(executor).search(queries, wanted: 10, arena: arena) // blocking, on the executor
let call = try index.deferred(executor).search(queries, wanted: 10, arena: arena) // Deferred<Matches>
let matches = try await call.value  // or: try call.wait()

A Deferred owns its outcome and its outputs, and keeps alive every object whose memory the call reads, until the call is final. It borrows the index, the executor and the arena as a blocking call does, since releasing any of them waits for the calls naming it, and its waited result is the same view a blocking call answers. A blocking call lends its inputs in place and frees its outcome, slots and callbacks on return; a deferred one copies only the keys and paths it borrows. status reads deferred until then, through the outcome word with acquire ordering, and wait() blocks and answers the result or throws the failure, the same every time it is asked. value waits on a Dispatch thread and then reads the result in the caller’s isolation, so an async caller suspends rather than blocks, and the cooperative pool keeps its threads. Releasing an unfinished one waits for it first.

let call = try index.deferred(executor).search(queries, wanted: 10, arena: arena)
if call.status == .deferred {
    // other work
}
let matches = try await call.value  // or: try call.wait()

A failure this binding catches before the call crosses throws at submission, and one the library catches arrives through status and wait. One call holds an executor at a time, so a second call finding it held finishes at once with contended, and nothing queues. A deferred call on a caller-exclusive pool runs on the pool’s thread 0, so polling status alone finishes it, and its progress handler and predicate run on the pool’s threads; a single thread or a VRAM executor runs it at submission. Calls depending on each other are ordered by waiting on the first before submitting the second. Rows built over a Tensor stay alive with the call, while memory borrowed through a MatrixView or a softmax carry must outlive it. The constructors that the C interface lets fan out take an executor in their config and run on the calling thread without one.