Executors

An executor owns host threads or device streams; an index owns entries. Keeping them apart is what lets one pool serve several indexes, and what makes the cost of spawning visible at the line that spawns.

One factory per C constructor, each taking only what its hardware has: ForVram has no thread count because a single host thread drives one stream per device.

using Executor threads = Executor.ForRam(threads: 8);             // Zero derives 7/8 of the cores
using Executor rings = Executor.ForNvme(threads: 4);              // One submission ring apiece
using Executor streams = Executor.ForVram(scratchLimit: 1 << 30); // One stream per device
using Executor shared = Executor.OnPool(forkUnionPool);           // Borrows another library's threads

threads.Drain(Mood.Spinning);                                     // Waits out the holding call, then keeps spinning

Each factory also names where its shards go, which is how a dual-socket or multi-GPU box is reached.

using Executor domains = Executor.ForRam(16, memoryDomains: [0, 1]);  // One shard per NUMA node
using Executor devices = Executor.ForVram([0, 1], cudaStreams);        // Borrowed streams, one per device
using Executor striped = Executor.ForNvme(paths: ["/mnt/nvme0", "/mnt/nvme1"]);

One call holds an executor at a time, while handles stay freely concurrent, so a caller wanting genuine concurrency gives each thread its own executor.

§ Receivers & Deferred Calls

The receiver chooses where a verb runs, and the verb stays the same. Called on the index, a verb runs on the caller’s own thread, single-threaded on RAM and on the handle’s device and default stream on VRAM. On index.On(executor) it blocks on that executor, refused while another call holds it. On index.Deferred(executor) it submits there and answers a Deferred<TResult> at once.

Matches local = index.Search(queries, wanted: 10, arena);                 // The caller's thread
Matches pooled = index.On(pool).Search(queries, wanted: 10, arena);       // Blocking, on the pool
Matches awaited = await index.Deferred(pool).Search(queries, 10, arena);  // Or: .Wait() in place of await

A Deferred<TResult> owns its outcome and the pins on every input it lends the library until the call is final, and borrows its index and arena exactly as a blocking call does, so Wait() answers the same view a blocking call would, over the same arena. Status reads Status.Deferred until then, Wait() blocks and answers the result or raises the failure, and await runs that wait on the thread pool. Verbs that produce nothing answer a plain Deferred, as Task stands beside Task<TResult>.

using Deferred<Matches> call = index.Deferred(pool).Search(queries, wanted: 10, arena);
if (call.Status == Status.Deferred)
{
    // other work
}
Matches matches = await call;                                       // Or: call.Wait()

Waiting again answers the same result or raises the same failure. Disposing a call nobody waited for waits for it, discards what it produced, and lets its arena serve the next call. One left unfinished and unreachable keeps its outputs and pins alive until a collection finds it final, rather than freeing memory the library still writes. Disposing an index or an arena that a deferred call names waits for that call first. One call holds an executor at a time, so a second call finding it held finishes at once with Status.Contended, and nothing queues. A deferred call on a caller-exclusive pool runs on the pool’s thread 0, so polling alone finishes it, and its progress handler and predicate run on the pool’s threads; a single thread or a VRAM executor runs it at submission. Calls depending on each other are ordered by waiting on the first before submitting the second.