# Executors

> USearch for Rust

An executor owns host threads or device streams, and the traversal scratch each of them walks with, grown to a high-water mark.
One executor serves any number of indexes, and its residency must match the one their handles were built for.
There is a constructor per residency, each taking only what its own hardware has, and the placement lists are the ones the indexes were built with, which keeps compute and memory domains in agreement.

```rust
let threads = Executor::for_ram(&[], None, 8)?;                        // eight workers, where zero takes 7/8 of the cores
let domains = Executor::for_ram(&[0, 1], None, 0)?;                    // threads interleaved across two NUMA nodes
let pinned = Executor::for_ram(&[], Some(CPUClass::IntelHaswell), 0)?; // waiters and atomics of one class
let rings = Executor::for_nvme(&["/mnt/nvme0"], None, 4)?;             // one submission ring apiece
let streams = unsafe { Executor::for_vram(&[0, 1], None, &[], 0)? };   // one stream per device, scratch grown as asked
let shared = unsafe { Executor::on_pool(&[], None, pool)? };           // shares the threads another library spawned
let local = Executor::default();                                       // owns nothing, runs on the calling thread

threads.drain(Mood::Spinning)?;                                        // waits out the holding call, then keeps spinning
```

`for_ram` and `for_nvme` take a thread count, where zero derives one.
`for_vram` takes none, since one host thread drives one stream per device, and may borrow your CUDA streams, one per device.
`for_vram` and `on_pool` are `unsafe`, since the streams or the pool they borrow must outlive the executor; the engine wires neither borrowing yet and answers `Status::Unsupported`.
`Executor::default()` owns nothing and runs every call handed to it on the calling thread.
One call holds an executor at a time, so two threads that genuinely overlap take one apiece; an executor is `Send` but not `Sync`, while index handles stay freely shared.
`drain` waits until no call holds the executor and sets the `Mood` idle workers adopt, and dropping an executor drains it, unless a GPU arena it made still holds it, which then does so on its own drop.

### Receivers & Deferred Calls

The receiver decides where a verb runs and how it completes, while the verb and its arguments stay the same:

- `index.search(..)` runs on the calling thread, through `&DenseIndex`.
- `index.on(&executor).search(..)` blocks on the executor.
- `index.deferred(&executor).search(..)` submits the call and answers a `Deferred`.

`DenseVerbs` writes every verb once for all three.
A blocking verb returns its `Result` directly, so nothing about asynchrony reaches a caller who never asks for it.
An `On` borrows the index and the executor, so neither it nor a `Deferred` it submits can outlive them.

```rust
use numkong::Matrix;

let queries = Matrix::<f32>::from_slice(&slab, &[2, 4]).expect("8 values fill 2 × 4");
let call = index.deferred(&executor).search(queries, 10, &mut arena)?;   // Deferred<'_, DenseSearch<..>>
if call.status() == Err(Status::Deferred) {
    // other work
}
let matches = call.await?;                                               // or: call.wait()?
```

A `Deferred` owns the inputs the library reads until the call is final, and borrows the index and the arena, as a blocking call does.
The waited result is the `Matches` a blocking search answers, borrowing the same arena.
`status` answers `Ok(())` once the call succeeded, `Err(Status::Deferred)` while it is not final, and its failure otherwise.
Inside an `async` block, `.await` polls the call's status until it is final, so other tasks run meanwhile, while `wait` blocks the thread without spinning.
The crate starts no thread, so waiting off-thread goes through a scoped thread, as `std::thread::scope(|scope| scope.spawn(move || call.wait()).join())` does.
Dropping an unfinished `Deferred` waits for the call, and a panic a handler raised resurfaces from `wait`, `.await`, or a drop outside unwinding.

The library reads a deferred call's inputs after the verb returns, so a deferred receiver takes owned inputs: keys in a `Vec` or an array, rows in a NumKong `Matrix` or a `'static` `MatrixView`, a `PathBuf`, `String` or `'static` path, and `'static` configs, handlers and value lookups.
Lending `&[Key]` or `&MatrixView` fails to compile with "a deferred receiver needs owned inputs", and a config or view borrowing a local fails as "does not live long enough".
A deferred call allocates one box holding its outcome, argument struct and inputs until it is final.
The box is the crate's rather than yours, since a `Deferred` leaked without a wait would leave the library writing into storage you had reused.

One call holds an executor at a time, so a second call finding it held finishes at once with `Status::Contended`, and nothing queues.
A deferred call on a caller-exclusive pool runs on the pool's thread 0, so polling alone finishes it, and its progress handler and predicate run on the pool's threads; an inclusive pool, a single thread, WebAssembly and a VRAM executor run it at submission.
Calls depending on each other are ordered by waiting on the first before submitting the second, since no call carries an ordering field.

`use usearch::DenseVerbs;`

`use usearch::Blocking;`

`use usearch::Deferred;`

`use usearch::Deferring;`

`use usearch::Executor;`

`use usearch::Mood;`

`use usearch::On;`

`use usearch::Wait;`

## DenseVerbs

Every dense verb, written once for every receiver: `&DenseIndex` blocks on the calling thread, and the [`On`](#On) that [`DenseIndex::on`](https://www.unum.cloud/docs/usearch/rust/metrics.md#on) or [`DenseIndex::deferred`](https://www.unum.cloud/docs/usearch/rust/metrics.md#deferred) makes runs on an executor.

```rust
pub trait DenseVerbs<'index>: Copy
```

A blocking receiver answers what the call produced; a deferring one answers a
[`Deferred`](https://www.unum.cloud/docs/usearch/rust/errors.md#Deferred) that borrows the index. Every configurable verb's config
converts into its `Dense*Config`, so the bare value is the whole call and
`..DenseSearchConfig::from(10)` reaches every knob.
Every verb whose output scales with the call borrows an [`Arena`](https://www.unum.cloud/docs/usearch/rust/inputs-and-outputs.md#Arena) last as `&mut Arena`, blocking
or deferring alike, except [`save_buffer`](https://www.unum.cloud/docs/usearch/rust/metrics.md#save_buffer), whose config holds it, and its result borrows the arena
in turn.

Rust-only.

Implementors: - `impl<'index> [DenseVerbs](#DenseVerbs)<'index> for &'index [DenseIndex](https://www.unum.cloud/docs/usearch/rust/metrics.md#DenseIndex)`
- `impl<'index, Completion: Completion> [DenseVerbs](#DenseVerbs)<'index> for [On](#On)<'index, [DenseIndex](https://www.unum.cloud/docs/usearch/rust/metrics.md#DenseIndex), Completion>`

## Completion

[`Blocking`](#Blocking) or [`Deferring`](#Deferring).

```rust
type Completion: Completion;
```

## *

The index this receiver reaches and the executor its calls run on.

```rust
fn handles(&self) -> Handles
```

## *

Reserves capacity for a number of incoming vectors ahead of insertion, a hint, since insertions grow the index on demand.

```rust
fn reserve(
    self,
    capacity: usize,
) -> <<Self as DenseVerbs<'index>>::Completion as Completion>::Answer<'index, DenseReserve>
```

## *

Reports the frozen metadata and the current counters in one call.

```rust
fn describe(
    self,
) -> <<Self as DenseVerbs<'index>>::Completion as Completion>::Answer<'index, DenseDescribe>
```

## *

The number of entries stored under `key`.

```rust
fn count(
    self,
    key: Key,
) -> <<Self as DenseVerbs<'index>>::Completion as Completion>::Answer<'index, DenseCount>
```

## *

Whether the index holds at least one entry under `key`.

```rust
fn contains(self, key: Key) -> Result<bool>
where
    Self: DenseVerbs<'index, Completion = Blocking>,
```

Rust-only.

## *

Retrieves every vector stored under `key` into `arena`, which the result borrows; an absent key yields no rows.

```rust
fn get<'arena, Value: Element>(
    self,
    key: Key,
    arena: &'arena mut Arena,
) -> <<Self as DenseVerbs<'index>>::Completion as Completion>::Answer<'index, DenseGet<'arena, Value>>
```

## *

Adds a matrix of vectors under their keys - the native batch operation.

```rust
fn insert<'config, Keys: AsRef<[Key]>, Rows: RowsView>(
    self,
    keys: Keys,
    config: impl Into<DenseInsertConfig<'config, Rows>>,
) -> <<Self as DenseVerbs<'index>>::Completion as Completion>::Answer<'index, DenseInsert<'config, Keys, Rows>>
where
    Self::Completion: Accepts<Keys> + Accepts<DenseInsertConfig<'config, Rows>>,
```

A batch is all-or-nothing and becomes visible at once. Every entry is claimed before the first
progress report, so a `Break` is ignored. Overlaps safely with concurrent searches on the
same handle, and the input scalar type is independent of the stored quantization - the
library converts on ingest.

## *

Re-selects the neighbors of every stored entry against the finished graph, answering a [`RefineReport`](https://www.unum.cloud/docs/usearch/rust/refinement.md#RefineReport) of the nodes re-selected and changed and the islands it re-linked.

```rust
fn refine<'config>(
    self,
    config: DenseRefineConfig<'config>,
) -> <<Self as DenseVerbs<'index>>::Completion as Completion>::Answer<'index, DenseRefine<'config>>
where
    Self::Completion: Accepts<DenseRefineConfig<'config>>,
```

A cheap insertion pass followed by refinement beats one expensive pass, so a wide expansion
factor belongs here rather than in [`DenseVerbs::insert`](https://www.unum.cloud/docs/usearch/rust/metrics.md#insert). On a RAM handle searches may run
throughout, on a VRAM one they may not, and insertions never may. A stop is a time budget:
the call still re-links its islands, leaving the graph correct, and answers
[`Status::Cancelled`](https://www.unum.cloud/docs/usearch/rust/errors.md#Cancelled).

## *

Audits how much of the graph a search reaches: forward reachability from as many landmarks as one search draws, swept level by level.

```rust
fn health(
    self,
) -> <<Self as DenseVerbs<'index>>::Completion as Completion>::Answer<'index, DenseHealth>
```

Read-only, so concurrent insertions merely leave fresh entries unaudited.

## *

Resolves a matrix of queries into up to `wanted` nearest matches apiece, writing them into `arena` as [`Matches`](https://www.unum.cloud/docs/usearch/rust/inputs-and-outputs.md#Matches).

```rust
fn search<'config, 'arena, Queries: RowsView>(
    self,
    queries: Queries,
    config: impl Into<DenseSearchConfig<'config>>,
    arena: &'arena mut Arena,
) -> <<Self as DenseVerbs<'index>>::Completion as Completion>::Answer<'index, DenseSearch<'config, 'arena, Queries>>
where
    Self::Completion: Accepts<Queries> + Accepts<DenseSearchConfig<'config>>,
```

Never writes to the graph, so any number of these may overlap with concurrent insertions.

## *

Late-interaction scoring of multi-row queries against multi-row entries, writing up to `wanted` keys per query, largest score first, in `arena` as one ragged run; zero keeps every key a query scored.

```rust
fn maxsim<'config, 'arena, Queries: RowsView>(
    self,
    queries: Queries,
    rows_per_query: usize,
    config: impl Into<DenseMaxsimConfig<'config>>,
    arena: &'arena mut Arena,
) -> <<Self as DenseVerbs<'index>>::Completion as Completion>::Answer<'index, DenseMaxsim<'config, 'arena, Queries>>
where
    Self::Completion: Accepts<Queries> + Accepts<DenseMaxsimConfig<'config>>,
```

Every query row takes its best match within a key, and those maxima sum to that key's
score - so `rows_per_query` is the token count each query occupies, [`queries`](https://www.unum.cloud/docs/usearch/rust/linear-algebra.md#queries) holds them
end to end, and `wanted` counts documents rather than rows. This is what `values_per_key`
above one exists for. Runs on an inner-product index alone.

See [https://arxiv.org/abs/2004.12832](https://arxiv.org/abs/2004.12832) - ColBERT, which introduced this scoring.

## *

Approximate sparse product of a matrix against the indexed vectors, writing its keys, products and offsets in `arena` as one ragged run.

```rust
fn multiply<'config, 'arena, Left: RowsView>(
    self,
    left: Left,
    config: impl Into<DenseMultiplyConfig<'config>>,
    arena: &'arena mut Arena,
) -> <<Self as DenseVerbs<'index>>::Completion as Completion>::Answer<'index, DenseMultiply<'config, 'arena, Left>>
where
    Self::Completion: Accepts<Left> + Accepts<DenseMultiplyConfig<'config>>,
```

The linear-algebra sibling of [`DenseVerbs::search`](https://www.unum.cloud/docs/usearch/rust/metrics.md#search), reporting every pair the traversal
scored rather than only the nearest. Pairs arrive in traversal order rather than sorted.
Runs on an inner-product index alone, since a product is a logit.

## *

Softmax attention of every query row over every indexed key its walk scores, folded as Flash Attention folds key tiles, with no top-k cut, writing into `arena` as rows of the queries' own scalar type.

```rust
fn attend<'config, 'lookup, 'arena, Queries: RowsView, Value: Element>(
    self,
    queries: Queries,
    value_lookup: &'lookup ValueLookup<'lookup, Value>,
    config: impl Into<DenseAttendConfig<'config>>,
    arena: &'arena mut Arena,
) -> <<Self as DenseVerbs<'index>>::Completion as Completion>::Answer<'index, DenseAttend<'config, 'lookup, 'arena, Queries, Value>>
where
    Self::Completion: Accepts<Queries> + Accepts<&'lookup ValueLookup<'lookup, Value>> + Accepts<DenseAttendConfig<'config>>,
```

The index holds one layer's keys for one key-value head and no values: `value_lookup`
resolves every scored key to its value row, as wide as the keys and stored in the index's
own quantization. A key left without a row, or given one of another type or width, refuses the
call with [`Status::ShapeMismatch`](https://www.unum.cloud/docs/usearch/rust/errors.md#ShapeMismatch), since a missing row is a wrong softmax rather than a
smaller one. A VRAM walk resolves values on the device, where no Rust closure runs, so an
arena from a VRAM executor is refused with [`Status::Device`](https://www.unum.cloud/docs/usearch/rust/errors.md#Device). The outputs come back
already divided by the partition. Runs on an inner-product index alone, since a softmax
needs logits.

See [https://arxiv.org/abs/2205.14135](https://arxiv.org/abs/2205.14135) - Flash Attention, whose recurrence this reuses.

## *

Absorbs every donor into this index, one after another, keeping their graphs rather than re-inserting their vectors and linking each in by walks across the appended part.

```rust
fn absorb<'config, Donors: IntoIterator<Item = DenseIndex>>(
    self,
    config: impl Into<DenseAbsorbConfig<'config, Donors>>,
) -> <<Self as DenseVerbs<'index>>::Completion as Completion>::Answer<'index, DenseAbsorb<'config>>
where
    Self::Completion: Accepts<DenseAbsorbConfig<'config, Donors>>,
```

Readers of a RAM index may keep searching throughout, while a VRAM one admits no
overlapping search. The donors are consumed whether or not the absorb succeeds: a stop cuts
only the closing refinement, every donor already moved, and answers [`Status::Cancelled`](https://www.unum.cloud/docs/usearch/rust/errors.md#Cancelled),
while a failure frees the donors it never reached and may leave the failing donor's entries
behind.

## *

Splits this index into the empty receivers, the inverse of [[`DenseVerbs::absorb`](https://www.unum.cloud/docs/usearch/rust/metrics.md#absorb)]: the entries are partitioned along the graph, each part lands in one receiver with its edges kept, and a closing refinement re-links every receiver, leaving this index empty and live.

```rust
fn split<'config>(
    self,
    config: impl Into<DenseSplitConfig<'config>>,
) -> <<Self as DenseVerbs<'index>>::Completion as Completion>::Answer<'index, DenseSplit<'config>>
where
    Self::Completion: Accepts<DenseSplitConfig<'config>>,
```

## *

Draws keys around the focus, or across the whole index, each hanging from a shown or earlier draw and placed in three dimensions beside the shown ones, writing up to `wanted` of them into `arena` as a [`Sample`](https://www.unum.cloud/docs/usearch/rust/retrieval-and-streaming.md#Sample).

```rust
fn sample<'config, 'arena>(
    self,
    wanted: usize,
    arena: &'arena mut Arena,
    config: DenseSampleConfig<'config>,
) -> <<Self as DenseVerbs<'index>>::Completion as Completion>::Answer<'index, DenseSample<'config, 'arena>>
where
    Self::Completion: Accepts<DenseSampleConfig<'config>>,
```

A larger `wanted` under the same seed draws a
superset; only a RAM index samples yet.

## *

Saves the index in the layout the path extension picks - one `.safegraphs` monolith, or a `.safegraphs.json` manifest beside its shards.

```rust
fn save<'config, Location: AsRef<Path>>(
    self,
    config: impl Into<DenseSaveConfig<'config, Location>>,
) -> <<Self as DenseVerbs<'index>>::Completion as Completion>::Answer<'index, DenseSave<'config, Location>>
where
    Self::Completion: Accepts<DenseSaveConfig<'config, Location>>,
```

Demands a mutation-free window; concurrent readers keep searching throughout. A stopped save
leaves leftovers every reader rejects and the caller is left to remove.

## save_*_buffer

Saves the index as one monolith into the arena its config carries.

```rust
fn save__buffer<'config, 'arena>(
    self,
    config: impl Into<DenseSaveBufferConfig<'config, 'arena>>,
) -> <<Self as DenseVerbs<'index>>::Completion as Completion>::Answer<'index, DenseSaveBuffer<'config, 'arena>>
where
    Self::Completion: Accepts<DenseSaveBufferConfig<'config, 'arena>>,
```

## *

Opens a walk over the current contents, pulled blocking on this receiver's executor; each shard's publication watermark is read as the walk enters it, so an entry once passed is never revisited.

```rust
fn cursor(self, starts: &[Key]) -> Result<DenseCursor<'index>>
where
    Self: DenseVerbs<'index, Completion = Blocking>,
```

Without `starts` the walk yields every entry as stored. With them it yields the start keys'
own entries first, in order, then the entries those link to breadth-first, each entry once.

## search_*_stream

Opens a resumable nearest-neighbor stream for a one-row query, pulled blocking on this receiver's executor.

```rust
fn search__stream<Query: RowsView>(
    self,
    config: impl Into<DenseSearchStreamInitConfig<Query>>,
) -> Result<DenseSearchStream<'index>>
where
    Self: DenseVerbs<'index, Completion = Blocking>,
```

Best-first traversal is naturally pausable, so callers that post-filter aggressively pull
until satisfied instead of guessing "k" upfront. The stream keeps walk state rather than a
thread.

No residency implements the stream yet, so the engine answers [`Status::Unsupported`](https://www.unum.cloud/docs/usearch/rust/errors.md#Unsupported).

## Blocking

The completion of a blocking receiver, whose verbs return once their call is final and answer what it produced.

```rust
pub struct Blocking;
```

Rust-only.

Implements: `Debug`

## Deferred

A submitted call, which owns its outcome, argument struct and inputs in one box the library reads until the call is final, and borrows its arena as a blocking call does.

```rust
pub struct Deferred<'index, Verb: Calling>
```

[`Deferred::status`](#status) polls without blocking, and [`Deferred::wait`](#wait) blocks until the call is
final and hands back what it produced, while `.await` polls it from async code. Dropping an
unfinished one waits for it, then frees the box.
A panic a handler raised resurfaces from [`wait`](#wait), `.await`, or a drop outside unwinding.
`'index` is the borrow of the [`On`](#On) that submitted it.

Rust-only.

Implements: `Drop`, `Debug`, `Send`, `IntoFuture`

## status

`Err(Status::Deferred)` until the call is final, then its final status, read without blocking.

```rust
pub fn status(&self) -> Result<(), Status>
```

## wait

Blocks until the call is final, then hands back what it produced.

```rust
pub fn wait(self) -> Result<Verb::Output>
```

## Deferring

The completion of a deferring receiver, whose verbs return once their call is submitted and answer a [`Deferred`](https://www.unum.cloud/docs/usearch/rust/errors.md#Deferred).

```rust
pub struct Deferring;
```

Rust-only.

Implements: `Debug`

## Executor

The host threads or device streams an [`On`](#On) runs its calls on, and the traversal scratch each of them walks with.

```rust
pub struct Executor
```

One executor serves any number of index handles, so separate categories of indexes share one
machine's cores or devices instead of each claiming a private set. Its kind is fixed by the
constructor that produced it and must match the residency of every handle it drives.

[`Executor::default`](#Executor) is the null executor: it owns nothing, and every call
handed it runs on the calling thread.

Calls on one executor start in submission order, so threads wanting to overlap take one apiece;
it is `Send` but not `Sync`.
Every constructor takes a `deferred_limit` sizing its table of deferred calls, where zero takes
the default; a deferred submission finding the table full fails with [`Status::Contended`](https://www.unum.cloud/docs/usearch/rust/errors.md#Contended).
[`Arena::new_in`](https://www.unum.cloud/docs/usearch/rust/inputs-and-outputs.md#new_in) places an arena where it computes: a host arena may outlive it, while a GPU
arena holds it until the arena drops.
Dropping an executor drains it, unless a GPU arena it made still holds it, which then does so.

Implements: `Debug`, `Send`, `Default`

## for_ram

Spawns host workers for RAM-resident indexes, interleaved across the compute domains local to `memory_domains`, which is the same list the indexes were initialized with.

```rust
pub fn for_ram(
    memory_domains: &[i32],
    cpu_class: Option<CPUClass>,
    threads: usize,
    deferred_limit: usize,
) -> Result<Self>
```

Zero `threads` takes 7/8 of the cores those domains carry, since full occupancy starves
the ordered publication path.

A pinned `cpu_class` narrows the threads' waiters and atomics to that class's.

## on_pool

Creates a RAM executor over a borrowed ForkUnion pool, so several libraries in one process share one set of threads while each keeps its own scratch.

```rust
pub unsafe fn on_pool(
    memory_domains: &[i32],
    cpu_class: Option<CPUClass>,
    pool: *mut c_void,
    deferred_limit: usize,
) -> Result<Self>
```

Safety: `pool` must address a live ForkUnion `fu_pool_t` that outlives the executor.

## for_nvme

Spawns host workers for NVMe-resident indexes, one submission ring apiece, across the compute domains local to the devices `paths` names.

```rust
pub fn for_nvme(
    paths: &[impl AsRef<Path>],
    cpu_class: Option<CPUClass>,
    threads: usize,
    deferred_limit: usize,
) -> Result<Self>
```

Zero `threads` derives from those devices' queue capabilities.

A pinned `cpu_class` narrows the threads' waiters and atomics to that class's.

## for_vram

Opens one submission stream on every named device, or borrows the caller's, all of them driven by the one host thread that calls the verbs - which is why this takes no thread count.

```rust
pub unsafe fn for_vram(
    devices: &[i32],
    gpu_class: Option<GPUClass>,
    streams: &[*mut c_void],
    scratch_limit: usize,
    deferred_limit: usize,
) -> Result<Self>
```

`streams` is empty, or holds one CUDA stream per device in `devices` order. Zero
`scratch_limit` lets scratch grow as geometry asks. A pinned `gpu_class` the devices do not
run is refused with [`Status::Unsupported`](https://www.unum.cloud/docs/usearch/rust/errors.md#Unsupported).

Safety: Every stream must be a live CUDA stream on its device that outlives the executor.

## drain

Waits for every call submitted to this executor, then adopts `mood` whenever it next goes idle; a failure is the executor's own, such as a lost device, never one of a drained call.

```rust
pub fn drain(&self, mood: Mood) -> Result<()>
```

## Mood

What an executor does with its threads or streams once it goes idle.

```rust
pub enum Mood
```

- `Parked` = `1`: Idle threads sleep; the next call pays a wake-up.
- `Spinning` = `2`: Idle threads stay hot, trading power for the next call's latency.

Implements: `Clone`, `Copy`, `StructuralPartialEq`, `PartialEq`, `Eq`, `Debug`, `Default`, `Hash`, `Display`, `FromStr`

## On

An index borrowed with the executor its calls run on and how they complete, shared by every modality.

```rust
pub struct On<'index, Index, Completion = Blocking>
```

Neither the receiver nor a [`Deferred`](https://www.unum.cloud/docs/usearch/rust/errors.md#Deferred) it submits outlives the index or the executor.

Rust-only.

Implements: `[DenseVerbs](#DenseVerbs)<'index>`, `Debug`, `Clone`, `Copy`

## Wait

A [`Deferred`](https://www.unum.cloud/docs/usearch/rust/errors.md#Deferred) being awaited: every poll reads its status, and one not yet final asks to be polled again.

```rust
pub struct Wait<'index, Verb: Calling>(_);
```

The library has no wake-up, so an executor re-polls it while other tasks run, spinning only
when nothing else is ready; [`Deferred::wait`](#wait) blocks without spinning.

Rust-only.

Implements: `Future`