An index owns entries; an executor owns the threads or device streams a verb runs on, so one executor serves several indexes and the cost of spawning shows at the line that spawns it.
There is a constructor per C one, each taking every argument its own hardware has:
usearch.ExecutorForRam(memoryDomains, cpuClass, threads)usearch.ExecutorOnPool(memoryDomains, cpuClass, forkUnionPool) // shares the threads another library spawnedusearch.ExecutorForVram(devices, gpuClass, streams, scratchLimit) // one host thread, a stream per deviceusearch.ExecutorForNvme(paths, cpuClass, threads) // one submission ring per thread
Zero for any count derives it, and zero threads in ExecutorForRam takes 7/8 of the logical cores, since full occupancy starves the ordered publication path.
A placement names hardware and nothing else, and an executor’s residency must match that of the indexes it drives, or the engine reports StatusDevice.
A nil *Executor owns nothing, and every call handed it runs on the calling goroutine.
One call holds an executor at a time, so goroutines that genuinely overlap take one apiece. Drain(mood) waits until no call holds the executor and picks whether the workers park or spin once idle, and Close releases them.
Constructors address no index yet, so no receiver offers them; the ones the C interface lets fan out take an Executor in their config, and run on the calling goroutine without one.
A DenseIndex may be shared across goroutines with no lock of your own.
Searches read only entries published behind the visibility watermark and never write to the graph, while insertions synchronize through per-node locks and publish in order, so the two overlap freely.
var work sync.WaitGroupfor worker := range workers { work.Add(1) go func() { defer work.Done() if _, err := index.On(executors[worker]).Search(queries, 10, arenas[worker], nil); err != nil { log.Print(err)} }()}work.Wait()
Serialization is the one exception to the free overlap: saving demands a mutation-free window, though concurrent readers keep searching throughout.
The receiver chooses where a verb runs and how it completes, and the verb stays the same.
The index runs it on the calling goroutine, index.On(executor) blocks on an executor, and index.Deferred(executor) submits it there and answers a *usearch.Deferred:
matches, err := index.Search(queries, 10, arena, nil) // the calling goroutinematches, err = index.On(executor).Search(queries, 10, arena, nil) // blocking, holding the executor until it returnscall := index.Deferred(executor).Search(queries, 10, arena, nil) // *usearch.Deferred[usearch.MatchesView]
DenseIndexOn and DenseIndexDeferredOn offer the same verbs with the same arguments; only the blocking ones add Contains, Cursor and SearchStream, whose every pull blocks.
A Deferred holds what the library reads or writes until its call is final: the outcome and the pinned inputs.
call := index.Deferred(executor).Search(queries, 10, arena, nil)if call.Status() == usearch.StatusDeferred { // other work}matches, err := call.Wait() // a view borrowing arena, as a blocking call's does
Go needs nothing more: a goroutine that wants to overlap the wait simply calls Wait there, and a select waits on a channel that goroutine closes. Status reads the outcome without blocking, and Wait blocks until the call is final and hands back what it produced, as many times as it is asked. Close waits the same way and frees whatever Wait has not handed out, so a call no one reads is closed rather than waited on; it is idempotent and safe after Wait.
A failure this binding catches before the call crosses arrives through the same Status and Wait, as the engine’s own refusals do.
A Deferred dropped without Wait or Close is released by the collector once its call is final, which checks it again at every later collection rather than blocking on it.
One call holds an executor at a time, so a second call finding it held finishes at once with StatusContended, and nothing queues.
A deferred call on a caller-exclusive pool runs on the pool’s thread 0, so polling Status alone finishes it, and its progress handler and predicate run on the pool’s threads; a single thread or a VRAM executor runs it at submission.
Calls depending on each other are ordered by waiting on the first before submitting the second.
The slices a deferred verb is handed must not change until it is final, since the library reads them until then.
API
StructDeferredDeferred is a submitted call, which holds its outcome and inputs until it is final.3
type Deferred[Result any] struct {
// contains filtered or unexported fields
}
Status polls without blocking, Wait blocks until the call is final and hands back what it produced, and Close waits the same way and frees what no one took. A Deferred dropped without Wait or Close is released by the collector once the call is final, never blocking on it: until then every later collection checks it again.
A failure the binding catches before the call crosses is reported here too, like any other.
CloseClose blocks until the call is final, releases its pins, frees whatever it produced that Wait has not handed out, and reports the call's failure.
WaitWait blocks until the call is final, releases its pins, then hands back what it produced, as many times as it is asked; one goroutine waits at a time.
StructDenseIndexDeferredOnDenseIndexDeferredOn is the receiver whose verbs submit their calls to an executor and answer a Deferred, offering every verb of DenseIndexOn with the same arguments but the blocking-only sugar, cursors and streams.2
type DenseIndexDeferredOn struct {
// contains filtered or unexported fields
}
Binding-only.
The library reads a deferred call's inputs until the call is final, so the slices a verb is
handed must not change until then; its arena is borrowed, as a blocking verb borrows it.
StructDenseIndexOnDenseIndexOn is the receiver whose verbs block until their call is final: on the calling goroutine when embedded in a DenseIndex, and on an executor when DenseIndex.On makes it.20
type DenseIndexOn struct {
// contains filtered or unexported fields
}
Binding-only.
AbsorbAbsorb moves every donor into this index, one after another, keeping their graphs rather than re-inserting their vectors: each donor's entries append shard by shard, then a refinement links them across the appended part.
Donors must share the index's residency, scalar, metric and geometry. Readers of a RAM index keep searching throughout; a VRAM one admits no overlapping search.
Each absorbed donor is consumed, and every later verb on it reports StatusUninitialized. A stop
between donors, or a failure, leaves every donor not yet absorbed a live handle the caller still
owns and closes; until the call is final every donor reports StatusUninitialized.
AttendAttend is softmax attention of every query row over every indexed key its walk scores, folded as Flash Attention folds key tiles, with no top-k cut, writing one row per query into arena, already divided by the partition.
The index holds one layer's keys for one key-value head and no values, so lookup resolves every
scored key to its value row. Runs on an inner-product index alone, since a softmax needs logits,
and on RAM and NVMe handles, since a VRAM walk needs a device lookup no Go function can be.
See https://arxiv.org/abs/2205.14135 - Flash Attention, whose recurrence this reuses.
ClearClear erases all entries, keeping the reserved capacity and the frozen geometry.
Without starts it yields every entry as stored; with them, the start keys' own entries first in order, then the entries those link to breadth-first, each once.
DescribeDescribe reports the frozen metadata and the current counters in one call.
GetGet exports every vector stored under a key into arena, one row per entry sharing the key, in insertion order, converted to the outputsScalar kind - one call, however many there turn out to be.
Overlaps safely with concurrent searches on the same handle.
The input scalar type is independent of the stored quantization; the library converts on
ingest.
A batch is all-or-nothing and becomes visible at once, since every fallible step runs before
the first entry identity is claimed. A VRAM call that fails after its first launch publishes
none of its entries, and leaves the handle refusing every later mutation.
MaxsimMaxsim scores multi-row queries against multi-vector documents by late interaction: every query row takes its best match within a key, and those maxima sum to that key's score.
A document is one key holding its token rows, which is what ValuesPerKey above one exists
for. rowsPerQuery is the token count each query occupies, queries holds them end to end, and
wanted counts documents rather than rows, zero keeping every key a query scored. Each query's
keys land in arena as one run, largest score first. Runs on an inner-product index alone.
See https://arxiv.org/abs/2004.12832 - ColBERT, which introduced this scoring.
MultiplyMultiply is the approximate sparse product of a matrix against the indexed vectors - the linear algebra sibling of Search, reporting every pair the traversal scored rather than only the nearest.
Pairs arrive in traversal order rather than sorted, and the pairs the walk never reaches are the low and negative products. Runs on an inner-product index alone.
How many pairs a walk scores is known only once it finishes, so the products, keys and offsets
land in arena, as Get's rows do.
RefineRefine re-selects the neighbors of every stored entry against the finished graph, improving edges chosen when the graph around them was still incomplete.
A build step rather than a repair: a cheap insertion pass followed by one refinement reaches
a better graph, in less total time, than a single expensive insertion pass - so a wide
expansion factor belongs here rather than in Insert. Reports the entries re-selected and
changed, so a caller loops until Changed falls below whatever rate they consider settled.
On a RAM handle searches may run throughout; insertions may not. A stop is a time budget
rather than a failure, so it reports no status.
ReserveReserve claims capacity for a number of incoming vectors ahead of insertion.
func (on DenseIndexOn) Reserve(capacity uint) error
The capacity is the desired total, including the current size.
SampleSample draws up to wanted keys around the config's Focus, or across the whole index, each hanging from a shown or earlier draw and placed in three dimensions beside the shown ones, so a view grows one connected sample as it zooms; a larger wanted draws a superset.
SaveSave writes the index under a path, in the layout its extension picks - one .safegraphs monolith, or a .safegraphs.json manifest beside its shards.
SearchSearch resolves a matrix of queries into up to wanted nearest matches apiece, written into arena - the native batch path, parallelized internally, and filtered when the config carries a predicate.
Never writes to the graph, so any number of these may overlap with concurrent insertions. A
zero wanted walks nothing and lands only the counts, every one zero.
SearchStreamSearchStream opens a resumable search for a one-row query, so callers that post-filter aggressively pull until satisfied instead of guessing the wanted count upfront.
No residency implements the stream yet, so the engine answers StatusUnsupported.
SplitSplit hands this index to the receivers, the inverse of Absorb: graph-local parts keeping their edges, one per receiver, leaving this index empty but live.
type Executor struct {
// contains filtered or unexported fields
}
An executor owns workers; an index owns entries. Keeping them apart is what lets one pool serve
several indexes, and what makes the cost of spawning visible at the line that spawns it. Its
kind is fixed by the constructor that produced it and must match the residency of the handles
it drives, or the engine reports StatusDevice.
A nil *Executor is the null executor: it owns nothing, and every call handed it runs on the
calling goroutine - single-threaded on RAM, on the handle's device and default stream on VRAM.
Calls on one executor run in submission order, while handles stay freely concurrent - it is the
executor that orders.
Every constructor takes a deferredLimit sizing its table of deferred calls, where zero takes the
default; a deferred submission finding the table full finishes StatusContended.
ExecutorForNvmeExecutorForNvme spawns host workers for NVMe-resident indexes, one submission ring apiece, across the compute domains local to the devices paths names.
One ring per thread is what keeps submission lock-free: a thread owns its queue outright.
Zero threads derives from those devices' queue capabilities.
ExecutorForRamExecutorForRam spawns host workers for RAM-resident indexes, interleaved across the compute domains local to memoryDomains - the same list the indexes were initialized with, which is what keeps compute and memory domains in agreement.
Zero threads takes seven eighths of the cores those domains carry, since full occupancy starves
the ordered publication path. A pinned cpuClass narrows the workers' waiters and atomics to that
class's.
ExecutorForVramExecutorForVram opens one submission stream on every named device, or borrows the caller's, all of them driven by the one host thread that calls the verbs - which is why it takes no thread count.
Streams is empty, or holds one live CUDA stream per device in devices order, each outliving the
executor. Zero scratchLimit lets scratch grow as geometry asks. A pinned gpuClass the devices do
not run is refused with StatusUnsupported.
ExecutorOnPoolExecutorOnPool creates a RAM executor over a borrowed ForkUnion fu_pool_t, so several libraries in one process share one set of threads while each keeps its own scratch.