# Executors > USearch for C An executor owns the threads or device streams that run calls, the traversal scratch they walk with, and the pinned bounce buffer staging passes through. One executor serves any number of indexes, of any quantization and metric, since its scratch is sized from geometry, grown to the widest asked of it, and kept between calls. A VRAM executor's `scratch_limit` caps each stream's run, which every call's walk scratch and outputs carve from, and a call needing more reports `usearch_status_out_of_memory_k`. One call holds an executor at a time, while a handle admits any number of concurrent callers: the executor serializes, never the index. ```c int32_t const sockets[2] = {0, 1}; usearch_executor_for_ram(&(usearch_executor_for_ram_t) {.executor = &executor, .outcome = &outcome, .ram = {sockets, 2}, // NUMA nodes whose cores it spawns on .threads = 0}); // zero takes 7/8 of those cores ``` Threads interleave across the named domains, so a half-size executor takes half of every socket rather than all of one, and the default stops at 7/8 because full occupancy starves ordered publication. `usearch_executor_for_vram` opens one stream per device, or borrows the caller's, and `usearch_executor_for_nvme` spawns one thread per submission ring. An application with its own ForkUnion pool lends it with `usearch_executor_on_pool` instead, though the engine refuses borrowed pools with `usearch_status_unsupported_k` for now. An executor's kind must match the residency of every handle it drives, and a mismatch reports `usearch_status_device_k`. A `NULL` executor is accepted by every verb and runs the call on the caller's own thread, final when it returns whatever its `completion`, which suits size lookups and other short calls. The exception is a VRAM verb that takes an arena: that arena must be a GPU arena on the index's first device, and it serves only calls on the executor that made it. A second call on a held executor finishes `usearch_status_contended_k`, so callers wanting real parallelism give each thread its own executor. `usearch_executor_drain` waits until no call holds the executor, then sets its threads' mood until the next call: `usearch_mood_parked_k`, the default, lets idle threads sleep in the kernel, `usearch_mood_spinning_k` keeps them hot. A blocking verb called from inside a progress handler or predicate of a call on the same executor reports `usearch_status_contended_k`, and so does a drain on the thread running that call's body; a drain from any other of its threads would wait on itself. `usearch_executor_free` waits out the call holding the executor before it releases the threads; host arenas the executor made stay valid until freed, and GPU arenas it made must be freed before it. ### Completion & Deferred Calls Every call reports through its `outcome` and names its `completion` in the same struct, so no verb has a second, asynchronous twin. `usearch_completion_blocking_k`, the default, returns with the final status stored and every output readable. `usearch_completion_deferred_k` returns once submitted, as an io_uring entry does, and the call finishes on its executor. A thread pool, a GPU stream or a remote server can serve the same verbs without changing a call site. Submission copies the argument struct, but everything it points to stays borrowed until the outcome is final. One call holds an executor at a time, so calls overlap by running on different executors, each with its own outcome and arena: ```c usearch_matches_view_t matches[2]; usearch_outcome_t outcomes[2] = {{0}}; for (size_t shard = 0; shard != 2; ++shard) usearch_dense_search(&(usearch_dense_search_t) { .index = shards[shard], .executor = executors[shard], .outcome = &outcomes[shard], .completion = usearch_completion_deferred_k, .count = 32, .queries = {queries, 128 * sizeof(float), 128, usearch_scalar_f32_k}, .matches = &matches[shard], .arena = arenas[shard], .wanted = 10}); while (usearch_outcome_status(&outcomes[0]) == usearch_status_deferred_k || usearch_outcome_status(&outcomes[1]) == usearch_status_deferred_k) do_other_work(); // polling alone finishes both ``` The status reads `usearch_status_deferred_k` until the call is final, even for a call refused at submission. Then the library stores the message and the final status with release ordering, and touches nothing the struct names after that. The caller reads it with acquire ordering: `usearch_outcome_status` polls and `usearch_outcome_wait` blocks on an atomic wait, both returning the status. Those two are the only safe readers of a deferred outcome, since the status word is atomic only by their contract. Never wait on a call's own outcome from inside its progress handler or predicate, since the call finishes only after they return. A call ends in one order: its arena and the counts of the handles it names, then the pool, then the executor, then the message and the status. So a caller that reads success may resubmit on the same executor or free a handle at once. A call submitted while another holds the executor finishes at once with `usearch_status_contended_k`, and nothing queues, so submitting never allocates. Every `*_free` waits until each executor call naming its handle has ended, so a handle may be dropped at any moment, though never from a callback of a call naming it. Waits, drains and frees only wait: none of them runs another thread's call. ### Inclusive and Exclusive Executors A caller-inclusive pool, a single thread or a WebAssembly build needs its caller to make progress, so a deferred call on it runs at submission and is final when the verb returns. A caller-exclusive pool, which `usearch_executor_for_ram` and `usearch_executor_for_nvme` spawn whenever they spawn more than one OS thread, delegates a deferred call to its own thread 0. That thread runs the call in the caller's place and forks the other threads as the caller would, so the call finishes while the caller does something else, one call per executor at a time. Progress handlers and predicates of a delegated call therefore run on the pool's threads. VRAM executors run every verb at submission, since each verb reads its device results back on the host before it ends. Together these let one manager thread in `int main()` overlap a RAM, an NVMe and a VRAM executor: it submits a deferred call to each and polls their outcomes. `#include ` ## usearch_completion_t Whether a call returns with its final status, or once submitted to its executor. ```c enum usearch_completion_t ``` - `usearch_completion_blocking_k` = `0`: Return with the final status stored and every output readable; the default. - `usearch_completion_deferred_k` = `1`: Return once submitted, the status deferred until final; the caller waits or polls. ## usearch_executor_t Everything a call needs to run but an index does not own: the host threads or device streams that advance the work, the traversal scratch each of them walks with, and the pinned bounce buffer staging passes through. ```c typedef struct usearch_executor_handle_t * usearch_executor_t; ``` One executor serves any number of index handles, so separate categories of indexes share one machine's cores or devices instead of each claiming a private set. Its kind is fixed by the constructor that produced it and must match the residency of every handle it drives; a mismatch reports [`usearch_status_device_k`](https://www.unum.cloud/docs/usearch/c/errors.md#usearch_status_device_k). Scratch is carved one slice per thread or stream, sized from geometry rather than from the scalar type, which is what lets a single executor drive indexes of different quantizations and metrics. It grows to the widest geometry asked of it, is rewound per call, and is kept between calls, so a steady workload allocates once. Calls on one executor start in submission order, and handles stay freely concurrent - it is the executor that orders, never the index. A blocking call runs alone: after every earlier call finishes, and before any later one starts. An executor without threads of its own - caller-inclusive, single-thread, or WebAssembly - runs a deferred call at submission or holds it until its wait, its executor's next drain or blocking call, or the free of a handle it names. Every executor tracks its deferred calls in a fixed table of `deferred_limit` slots, so submitting never allocates. A deferred submission that finds the table full finishes [`usearch_status_contended_k`](https://www.unum.cloud/docs/usearch/c/errors.md#usearch_status_contended_k), and blocking calls take no slot. A blocking verb or [`usearch_executor_drain`](#usearch_executor_drain) called from inside a progress handler or a predicate of a call on the same executor reports [`usearch_status_contended_k`](https://www.unum.cloud/docs/usearch/c/errors.md#usearch_status_contended_k) rather than waiting on itself. Every verb accepts a `NULL` executor and then runs on the caller's thread, blocking whatever its `completion` says: RAM work runs single-threaded, VRAM work runs on the handle's device and default stream and synchronizes before returning, and NVMe reads are synchronous. Its outputs land in any arena of the handle's residency. The library stages only to change layout on the side that computes, or to make pageable memory DMA-able, and never routes device bytes through the host: | Case | Action | |:-|:-| | Device inputs on the executor's device | read in place | | Device inputs needing padding or a cast | a device-to-device copy or a cast kernel | | Pinned or managed host inputs | a direct asynchronous upload | | Pageable host inputs | the pinned bounce buffer, then an upload | | Outputs in an arena on the executor's device | the kernel writes them directly | ## usearch_mood_t What an executor does with its threads or streams once it goes idle. ```c enum usearch_mood_t ``` - `usearch_mood_unknown_k` = `0`: No mood named, which derives parked. - `usearch_mood_parked_k` = `1`: Idle threads sleep; the next call pays a wake-up. - `usearch_mood_spinning_k` = `2`: Idle threads stay hot, trading power for the next call's latency. Spellings: ``` char const * usearch_mood_name(usearch_mood_t mood); ``` ``` usearch_mood_t usearch_mood_from_name(usearch_bytes_view_t name); ``` ## usearch_outcome_wait Blocks on an atomic wait until `outcome` is final, with acquire ordering. ```c usearch_status_t usearch_outcome_wait(usearch_outcome_t *outcome); ``` Never wait on a call's own outcome from inside its progress handler or predicate: the call finishes only after they return. - `outcome` (in): An outcome passed to a call, read until its status leaves [`usearch_status_deferred_k`](https://www.unum.cloud/docs/usearch/c/errors.md#usearch_status_deferred_k). ## usearch_outcome_status Reads the status of `outcome` without waiting, with the acquire ordering its final store pairs with, so a final status also publishes every output of the call. ```c usearch_status_t usearch_outcome_status(usearch_outcome_t *outcome); ``` The one way to poll: the status word is atomic only by this contract, so a caller reading the field directly needs an atomic load its language may not offer on foreign memory. - `outcome` (in): An outcome passed to a call. ## usearch_executor_for_ram Spawns host threads for RAM-resident indexes, interleaved across the compute domains local to the named memory domains, or borrows an existing ForkUnion pool. ```c void usearch_executor_for_ram(usearch_executor_for_ram_t *arguments); ``` Passing the very placement the indexes were built with is what makes compute and memory domains agree, leaving no second place to name hardware and no way for the two to disagree. Interleaving is why a half-size executor takes half of every socket rather than all of one, and the default stops at 7/8 of the cores because full occupancy starves the ordered publication path. A borrowed pool keeps its own threads and its lender's mood, so one machine runs one pool however many libraries schedule onto it. - `executor`: Output for the new executor. - `outcome`: Output for the outcome, final once its status leaves [`usearch_status_deferred_k`](https://www.unum.cloud/docs/usearch/c/errors.md#usearch_status_deferred_k). - `ram`: The memory domains of the indexes it will drive; copied, not borrowed. - `threads`: Threads to spawn; zero takes 7/8 of the cores those domains carry. - `deferred_limit`: Deferred calls it tracks at once, sizing its deferred table; zero takes a default. ## usearch_executor_on_pool Creates a RAM executor over a borrowed ForkUnion pool, so several libraries in one process share one set of threads while each keeps its own scratch. ```c void usearch_executor_on_pool(usearch_executor_on_pool_t *arguments); ``` - `executor`: Output for the new executor. - `outcome`: Output for the outcome, final once its status leaves [`usearch_status_deferred_k`](https://www.unum.cloud/docs/usearch/c/errors.md#usearch_status_deferred_k). - `ram`: The memory domains of the indexes it will drive; copied, not borrowed. - `pool`: The ForkUnion `fu_pool_t` to borrow; it must outlive the executor, and keeps its lender's mood. - `deferred_limit`: Deferred calls it tracks at once, sizing its deferred table; zero takes a default. Statuses: - [`usearch_status_unsupported_k`](https://www.unum.cloud/docs/usearch/c/errors.md#usearch_status_unsupported_k) Always, as borrowed pools are not wired yet. ## usearch_executor_for_vram Opens one submission stream on every distinct device `vram` names, or borrows the caller's, all of them driven by whichever host thread submits. ```c void usearch_executor_for_vram(usearch_executor_for_vram_t *arguments); ``` One stream per device is the whole design: every shard on a device queues on its stream in order, a single host thread keeps every device fed, and adding host threads would buy nothing, since the work has already left for the accelerator by the time one of them would return. A GPU executor never changes process-wide device flags. - `executor`: Output for the new executor. - `outcome`: Output for the outcome, final once its status leaves [`usearch_status_deferred_k`](https://www.unum.cloud/docs/usearch/c/errors.md#usearch_status_deferred_k). - `vram`: The devices of the indexes it will drive; copied, not borrowed. - `streams`: The __optional__ borrowed CUDA streams, one per device in `vram` order; `NULL` creates its own. - `scratch_limit`: The __optional__ ceiling on each stream's run, which every call's scratch and outputs carve from, past which a call reports [`usearch_status_out_of_memory_k`](https://www.unum.cloud/docs/usearch/c/errors.md#usearch_status_out_of_memory_k); zero lets the run grow as the geometry asks. - `deferred_limit`: Deferred calls it tracks at once, sizing its deferred table; zero takes a default. Statuses: - [`usearch_status_unsupported_k`](https://www.unum.cloud/docs/usearch/c/errors.md#usearch_status_unsupported_k) Borrowed `streams`, which are not wired yet. - [`usearch_status_device_k`](https://www.unum.cloud/docs/usearch/c/errors.md#usearch_status_device_k) An ordinal past the listing, or no device named. - [`usearch_status_config_mismatch_k`](https://www.unum.cloud/docs/usearch/c/errors.md#usearch_status_config_mismatch_k) Devices of more than one vendor. ## usearch_executor_for_nvme Spawns host threads for NVMe-resident indexes, one submission ring apiece, interleaved across the compute domains local to the devices the paths name. ```c void usearch_executor_for_nvme(usearch_executor_for_nvme_t *arguments); ``` One ring per thread is what keeps submission lock-free: a thread owns its ring outright, so an external-memory walk never coordinates with a sibling to issue a read. A ring reports its reads only to the thread that owns it, so deferred calls finish on those threads. - `executor`: Output for the new executor. - `outcome`: Output for the outcome, final once its status leaves [`usearch_status_deferred_k`](https://www.unum.cloud/docs/usearch/c/errors.md#usearch_status_deferred_k). - `nvme`: The devices of the indexes it will drive; copied, not borrowed. - `threads`: Threads to spawn, one submission ring apiece; zero derives from the devices' queue capabilities. - `deferred_limit`: Deferred calls it tracks at once, sizing its deferred table; zero takes a default. - `nvme_class`: The I/O engine every ring submits through, opened exactly or refused with the reason its setup gave; [`usearch_nvme_class_unknown_k`](https://www.unum.cloud/docs/usearch/c/introspection.md#usearch_nvme_class_unknown_k) tries io_uring, then AIO, then blocking reads. Statuses: - [`usearch_status_invalid_argument_k`](https://www.unum.cloud/docs/usearch/c/errors.md#usearch_status_invalid_argument_k) `NULL` paths for a nonzero count, or an unknown `nvme_class`. - [`usearch_status_config_mismatch_k`](https://www.unum.cloud/docs/usearch/c/errors.md#usearch_status_config_mismatch_k) Kernel paths mixed with transport identifiers, SPDK pinned over kernel paths or a kernel engine over transport identifiers, or more threads than an SPDK controller granted I/O queue pairs. - [`usearch_status_unsupported_k`](https://www.unum.cloud/docs/usearch/c/errors.md#usearch_status_unsupported_k) Transport identifiers in a build without SPDK, or a pinned engine this build lacks. ## usearch_executor_drain Waits for every call submitted to the executor before the drain began, then sets the mood it adopts whenever it next goes idle. ```c void usearch_executor_drain(usearch_executor_drain_t *arguments); ``` The drain reads the submission ticket on entry and waits up to it, so calls submitted while it waits can't keep it waiting forever. Every call reports its own failure through its own `outcome`, so a drain reports only what befell the executor itself. Parked means futex waits on RAM, blocking-sync events on VRAM, and ring waits on NVMe; spinning polls each of them instead. A borrowed pool keeps its lender's mood. - `executor`: The executor whose submitted calls are awaited. - `outcome`: Output for an executor-level failure, such as a lost device; never one of a drained call. - `mood`: What the executor does whenever it next goes idle; the last drain wins. Statuses: - [`usearch_status_contended_k`](https://www.unum.cloud/docs/usearch/c/errors.md#usearch_status_contended_k) A call running on the executor holds it, as for a drain from that call's own progress handler, predicate or host function. - [`usearch_status_device_k`](https://www.unum.cloud/docs/usearch/c/errors.md#usearch_status_device_k) A GPU executor with no vendor behind it. ## usearch_executor_free Drains every call, releases the scratch and the streams it owns, and frees the executor; `NULL` is safe, and it cannot fail, so takes no outcome. ```c void usearch_executor_free(usearch_executor_t executor); ``` Waits until every deferred call naming the handle is final. Waits until every call running on the executor, blocking or deferred, is final. Never call it from inside a progress handler, a predicate, or a host function the executor runs, since it would drain itself. Host arenas it made stay valid, each keeping its memory until freed; GPU arenas keep the rule [usearch_arena_t](https://www.unum.cloud/docs/usearch/c/inputs-and-outputs.md#usearch_arena_t) notes. - `executor` (inout): The handle to be freed; invalid afterwards.