Wolfram · Executors

Executors

Needs["USearch`"] 3.0.0-rc.1 · 4c3e5023

An executor owns the host threads or device streams; an index owns the entries. One executor serves several indexes, and its residency must match theirs. There is a constructor per C one, each taking every argument its own hardware has as an option, where zero and an empty list derive:

ExecutorForRam["MemoryDomains"->{0, 1}, "Threads"->0]
ExecutorOnPool[poolAddress, "MemoryDomains"->{}]                     (* borrows a fu_pool_t *)
ExecutorForVram["Devices"->{0}, "Streams"->{}, "ScratchLimit"->0]
ExecutorForNvme["Paths"->{"/dev/nvme0n1"}, "Threads"->0]
ExecutorDrain[executor, "parked"]                                    (* or "spinning" *)

ExecutorDrain picks whether the workers sleep or spin once idle, parked when the mood is left out. "Streams" lists borrowed CUDA stream addresses, one per device, and an empty list makes the executor create its own.

§ Receivers

The receiver chooses where a verb runs, and the verb stays the same. A DenseIndex runs it on the kernel’s own thread, and DenseIndexOn[index, executor] fans it out over an executor’s workers:

matches = DenseSearch[index, queries, 10, arena];                            (* the kernel's thread *)
matches = DenseSearch[DenseIndexOn[index, executor], queries, 10, arena];    (* the executor's workers *)

Either way the call blocks and returns once its outcome is final, so a notebook never polls, waits or orders calls. The C interface’s deferred calls are left out of this binding. Constructors address no index yet, so they take no receiver; the ones the C interface lets fan out take an "Executor" option instead.

API

Functions

DenseIndexOnDenseIndexOn[index, executor] is a receiver that runs every verb on executor, blocking until the verb is final; a bare DenseIndex runs verbs on the calling thread.
USearch.wl:14 ↗
DenseIndexOn[index, executor]
ExecutorExecutor[id] owns the threads, streams or submission rings verbs run on, freed once nothing references it.
USearch.wl:15 ↗
Executor[id]
ExecutorForRamExecutorForRam[] spawns host worker threads for RAM-resident indexes.
USearch.wl:18 ↗
ExecutorForRam[OptionsPattern[]]

"MemoryDomains" lists the NUMA nodes to place shards on, {} deriving every node the process may allocate on, and "Threads" is how many threads to spawn, 0 taking 7/8 of the cores those nodes carry.

Parameters

"MemoryDomains"Default: {}
"Threads"Default: 0
ExecutorOnPoolExecutorOnPool[pool] drives RAM-resident indexes on the ForkUnion thread pool at address pool, which the caller owns.
USearch.wl:19 ↗
ExecutorOnPool[pool, OptionsPattern[]]

"MemoryDomains" lists the NUMA nodes to place shards on, {} deriving every node the process may allocate on.

Parameters

"MemoryDomains"Default: {}
ExecutorForVramExecutorForVram[] opens one stream per device for VRAM-resident indexes.
USearch.wl:20 ↗
ExecutorForVram[OptionsPattern[]]

"Devices" lists the device ordinals to place shards on, {} picking device zero; "Streams" lends existing streams, one per device, {} creating its own; and "ScratchLimit" caps the memory each stream carves scratch and outputs from, past which a call fails with "out-of-memory", 0 letting it grow as needed.

Parameters

"Devices"Default: {}
"Streams"Default: {}
"ScratchLimit"Default: 0
ExecutorForNvmeExecutorForNvme[] opens one submission ring per worker for NVMe-resident indexes.
USearch.wl:21 ↗
ExecutorForNvme[OptionsPattern[]]

"Paths" lists the devices or directories to serve shards from, {} leaving every shard where its manifest points, and "Threads" is how many workers to spawn, 0 deriving it from the devices' queues.

Parameters

"Paths"Default: {}
"Threads"Default: 0
ExecutorDrainExecutorDrain[executor, mood] waits for every call submitted before it, then sets what idle workers do: "parked", the default, or "spinning".
USearch.wl:22 ↗
ExecutorDrain[executor, mood]