Executors

An executor owns the host threads or device streams a fanned-out verb runs on; an index owns entries. Keeping them apart is what lets one pool serve several indexes, and what makes the cost of spawning visible at the line that spawns. Its residency must match the residency of every handle it drives. The index itself runs a call on the calling thread instead: single-threaded on RAM, on the handle’s device and default stream on VRAM.

There is one factory per C constructor, and each takes every argument its C twin takes.

Executor.forRam(new int[] {0, 1}, null, 16);                        // two NUMA nodes, sixteen workers
Executor.forRam(new int[0], CPUClass.INTEL_HASWELL, 0);             // waiters and atomics of one class
Executor.onPool(forkUnionPool, new int[0], null);                   // a thread pool the caller already runs
Executor.forVram(new int[] {0, 1, 2, 3}, null, new MemorySegment[0], 0); // four devices, own streams
Executor.forNvme(new String[] {"/mnt/a", "/mnt/b"}, null, 8);       // two drives, eight submission rings

forVram takes no thread count: one host thread keeps every device fed, so the width of that pool is the device list itself. It borrows one CUDA stream per device when given them, and caps scratch bytes per device at a non-zero scratchLimit. Naming no hardware derives it: an empty domain list spreads over every memory domain the process may allocate on, an empty device list opens one free device, and an empty path list leaves every shard where its manifest already points.

try (Executor pool = Executor.forRam(new int[0], null, 16)) {
    DenseIndex.On receiver = index.on(pool);
    receiver.insert(keys, vectors);
    Matches matches = receiver.search(queries, 10, arena);
    receiver.save("index.safegraphs");
    pool.drain(Mood.PARKED);
}

One call holds an executor at a time, and handles themselves stay freely concurrent. drain waits until no call holds the executor, then sets what it does once idle: Mood.PARKED lets its threads sleep, Mood.SPINNING keeps them hot for the next call’s latency.

§ Concurrency

A handle may be shared across threads. On a RAM handle, insertions and searches overlap freely: searches read only entries published behind the visibility watermark and never write to the graph, while insertions synchronize through per-node locks and publish in order. Serialization is the one exception, demanding a mutation-free window, though concurrent readers keep searching throughout.

Every call leases the index it names through the FFM scope its handle lives in, so closing an index while another thread’s blocking call still runs on it raises IllegalStateException rather than freeing the handle under that call; deferred calls are waited for by the close itself. A pull leases its cursor or stream in place of the index, and a call leases one scope only, so an executor, or the index under a pull, is closed once no other thread still calls through it. An index, executor, arena or deferred call the program drops without closing is released once it is unreachable, and never while a call naming it is unfinished.

§ Receivers & Deferred Calls

The receiver chooses how a call completes, and the verb stays the same. DenseIndex runs a verb on the caller’s thread, index.on(executor) answers an On whose verbs block on that executor, and index.deferred(executor) answers a DeferredOn whose verbs submit there and answer a Deferred. A Deferred owns its outcome, its argument structure and its inputs until the call is final; status polls without blocking, and join blocks, then answers or raises. It borrows the index, executor and arena it names exactly as a blocking call does, and join answers the same view a blocking call would, over that arena.

try (Arena arena = new Arena(pool);
        Deferred<Matches> call = index.deferred(pool).search(queries, 10, arena)) {
    if (call.status() == Status.DEFERRED) {
        // other work
}
    Matches matches = call.join();    // valid until the next call given `arena`
}

The first join frees the outcome and keeps what it answered or raised, so a later status or join repeats it; only a closed Deferred raises IllegalStateException. Closing an unfinished Deferred waits for the call before freeing what it lent, so a native slab lent to one must stay open until then. Dropping one never blocks the collector: what it lent stays reachable until the call is final, and only then is it freed. One call holds an executor at a time, so a second call finding it held finishes with Status.CONTENDED, and nothing queues. A deferred call on a caller-exclusive pool runs on the pool’s thread 0, so polling status alone finishes it, and its progress handler and predicate run on the pool’s threads; a single thread or a VRAM executor runs it at submission. Calls depending on each other are ordered by joining the first before submitting the second, since no call carries an ordering field.