# Executors > UForm for Rust An executor owns host threads or device streams, the activation scratch, and the pinned buffers staging passes through, grown to a high-water mark. One executor serves any number of models of its residency. ```rust let threads = Executor::for_ram(&[], None, 8)?; // eight threads, where zero takes 7/8 of the cores let domains = Executor::for_ram(&[0, 1], None, 0)?; // threads interleaved across two NUMA nodes let serial = Executor::for_ram(&[], Some(CPUClass::Serial), 0)?; // dispatches as the portable floor let streams = unsafe { Executor::for_vram(&[0, 1], None, &[], None, &[], 0)? }; // one stream per device, scratch grown as asked let shared = unsafe { Executor::on_pool(&[], None, pool, 0)? }; // borrows a ForkUnion pool's threads let local = Executor::default(); // owns nothing, runs on the calling thread threads.drain(Mood::Spinning)?; // waits until no call holds it, then keeps spinning ``` The second argument pins a `CPUClass` or a `GPUClass` to dispatch as, and `None` derives the hardware's own. `gpu_devices(&mut devices)` fills a slice of `GPUDevice` and answers how many GPUs the process can drive, and a device's position there is the ordinal `for_vram` and `load_vram` name it by. `for_vram` and `on_pool` are `unsafe`, since the streams or the pool they borrow must outlive the executor; `on_pool` mirrors the C verb and answers `Status::Unsupported` until borrowing is designed, as `for_vram` does for borrowed streams. An executor runs one call at a time, so two threads that genuinely overlap take one apiece; an executor is `Send` but not `Sync`, while models stay freely shared. Dropping an executor drains it, unless a GPU arena it made still holds it, which then does so on its own drop. `use uform::Mesh;` `use uform::MeshDescription;` `use uform::MeshOptions;` `use uform::MultimemTier;` `use uform::CPUClass;` `use uform::Executor;` `use uform::GPUClass;` `use uform::GPUDevice;` `use uform::GPUVendor;` `use uform::*;` ## Mesh One process's place among the processes of a run: the topology every process agreed on at bootstrap, and the InfiniBand domain joining its NVLink domains, borrowed by every trainer. ```rust pub struct Mesh ``` Mirrors `uform_mesh_t`; dropping it calls `uform_mesh_free`, after every trainer borrowing it. Implements: `Debug`, `Send`, `Sync`, `Drop` ## bootstrap Joins this process to the run's mesh through a one-time rendezvous in `options.directory`; every process of the run calls it with the same count, multicast extent, directory and job, over a VRAM executor whose devices are this process's members. ```rust pub fn bootstrap(executor: &Executor, options: &MeshOptions<'_>) -> Result ``` Calls `uform_mesh_bootstrap`. ## describe Reports the mesh's shape and this process's place in it. ```rust pub fn describe(&self) -> Result ``` Calls `uform_mesh_describe`. ## MeshDescription What a mesh reports about itself. ```rust pub struct MeshDescription ``` Mirrors `uform_mesh_description_t`. - `gpus_per_host_domain` `usize`: GPUs one process drives. - `gpus_per_multimem_domain` `usize`: GPUs one NVLink domain spans. - `gpus_per_colocated_domain` `usize`: GPUs in the whole run. - `host_domain_index` `usize`: This process's index among the run's processes. - `first_gpu_index` `usize`: The run-wide index of this process's first GPU, where its share of a sharded dataset begins. - `multimem_tier` `MultimemTier`: How its NVLink domains reduce. - `gpu_class` `Option`: The class every GPU of the run dispatches as: the executor's pin, or the devices' own. Implements: `Clone`, `Copy`, `StructuralPartialEq`, `PartialEq`, `Eq`, `Debug`, `Hash` ## MeshOptions Where this process sits among the processes training one model, and how they meet. ```rust pub struct MeshOptions<'options> ``` Mirrors `uform_mesh_options_t`. - `host_domain_index` `usize`: This process's index among `host_domain_count`. - `host_domain_count` `usize`: Processes in the run, one per host; zero derives one, a run with no peers. - `gpus_per_multimem_domain` `usize`: GPUs one NVLink multicast domain spans, a fabric property no local probe sees; zero derives the host. - `directory` `&'options Path`: The directory every host sees, where processes exchange their facts; required with peers. - `job` `&'options str`: The run's unique name inside `directory`; required with peers. - `timeout_seconds` `f64`: Seconds any wait for a peer lasts before the call fails; zero derives 120. Implements: `Clone`, `Copy`, `StructuralPartialEq`, `PartialEq`, `Debug`, `Default` ## MultimemTier How a mesh's NVLink domains reduce. ```rust pub enum MultimemTier ``` Mirrors `uform_multimem_tier_t`. - `SwitchReduced` = `0`: The NVLink switch reduces through multicast memory. - `PeerRead` = `1`: Every member reads its peers in a fixed order, where multicast is absent; bit-identical results. Implements: `Clone`, `Copy`, `StructuralPartialEq`, `PartialEq`, `Eq`, `Debug`, `Hash`, `Display`, `FromStr` ## CPUClass The CPU families UForm ships tuned variants for: dispatch reports its pick in this vocabulary, and a RAM placement may pin one. ```rust pub enum CPUClass ``` Mirrors `uform_cpu_class_t`. - `Serial` = `1`: The portable floor: no SIMD assumptions. - `IntelHaswell` = `2`: Haswell-class x86: AVX2 & FMA, the 256-bit floor. - `IntelSierra` = `3`: Sierra Forest-class x86 - the E-core Xeons: AVX-VNNI and its successors without AVX-512. - `IntelSkylake` = `4`: Skylake-class x86: the AVX-512 foundation tier. - `IntelIcelake` = `5`: Icelake-class x86: AVX-512 VNNI & VPOPCNT. - `AmdGenoa` = `6`: Genoa-class x86 - Zen 4 and newer: native bf16 dot products over AVX-512. - `IntelSapphire` = `7`: Sapphire Rapids-class x86: the Genoa bundle plus AVX-512 FP16 and the AMX-BF16 & AMX-INT8 tiles. - `IntelGranite` = `8`: Granite Rapids-class x86: the Sapphire Rapids bundle plus the AMX-FP16 tiles. - `IntelDiamond` = `9`: Diamond Rapids-class x86: the Granite Rapids bundle plus AVX10.2's FP8 conversions. - `AwsG2` = `10`: Graviton2-class Arm - Neoverse N1, the bare NEON floor; phones predating Armv8.6 land here too. - `AppleM2` = `11`: Armv8.6+ NEON: dot products, bf16 & f16 fused multiplies - Apple M2, Cortex-A710 & X2 class silicon. - `AppleM4` = `12`: Apple M4-class Arm: the Armv8.6 NEON bundle plus the SME outer-product tiles. - `AwsG5` = `13`: Graviton5-class Arm - Neoverse V3, Armv9.2. - `NvidiaVera` = `14`: Vera-class Arm - NVIDIA's Armv9.2-generation cores with FP8 dot products. - `AlibabaC930` = `15`: Alibaba C930-class RISC-V - T-Head's XuanTie core, RVA23 vectors. - `Wasm32V128` = `16`: WebAssembly every engine loads: 32-bit, single-threaded, the fixed-width simd128 tier. - `Wasm64Extended` = `17`: WebAssembly with relaxed SIMD, shared memory for real thread pools, and 64-bit addressing. - `Monoversioned` = `18`: The one class of a monoversioned build: every capability its floor implies or its preset vouches for, as the kernel libraries compiled them. Implements: `Clone`, `Copy`, `StructuralPartialEq`, `PartialEq`, `Eq`, `Debug`, `Hash`, `Display`, `FromStr` ## Executor The host threads or device streams an [`On`](https://www.unum.cloud/docs/uform/rust/receivers-and-deferred-calls.md#On) runs its calls on, with the activation scratch and the staging buffers each of them uses. ```rust pub struct Executor ``` One executor serves any number of models of its residency, so several models share one machine's cores or devices instead of each claiming a private set. [`Executor::default`](#Executor) is the C interface's null executor: it owns nothing, and every call handed it runs on the calling thread. Calls on one executor run in submission order, so threads wanting to overlap take one apiece; it is `Send` but not `Sync`. Every constructor takes a `deferred_limit` sizing its table of deferred calls, where zero takes the default; a deferred submission finding the table full fails with [`Status::Contended`](https://www.unum.cloud/docs/uform/rust/errors.md#Contended). [`Arena::new_in`](https://www.unum.cloud/docs/uform/rust/arenas.md#new_in) places an arena where it computes: a host arena may outlive it, while a GPU arena holds it until the arena drops. Mirrors `uform_executor_t`; dropping its last holder calls `uform_executor_free`. Implements: `Debug`, `Send`, `Default` ## for_ram Spawns host threads for RAM-resident models, interleaved across the compute domains local to `memory_domains`, which is the same list the models were loaded with. ```rust pub fn for_ram( memory_domains: &[i32], cpu_class: Option, threads: usize, deferred_limit: usize, ) -> Result ``` Zero `threads` takes 7/8 of the cores those domains carry, and `cpu_class` pins the class its calls dispatch as, where [`None`](https://www.unum.cloud/docs/uform/rust/training.md#None) derives the CPU's own. Calls `uform_executor_for_ram`. ## on_pool Creates a RAM executor over a borrowed ForkUnion pool, so UForm, USearch and UStore in one process share one set of threads while each keeps its own scratch. ```rust pub unsafe fn on_pool( memory_domains: &[i32], cpu_class: Option, pool: *mut c_void, deferred_limit: usize, ) -> Result ``` Calls `uform_executor_on_pool`. #### Safety `pool` must address a live ForkUnion `fu_pool_t` that outlives the executor. ## for_vram Opens one submission stream on every named device, or borrows the caller's, all of them driven by whichever host thread submits - which is why this takes no thread count. ```rust pub unsafe fn for_vram( devices: &[i32], gpu_class: Option, memory_domains: &[i32], cpu_class: Option, streams: &[*mut c_void], scratch_limit: usize, deferred_limit: usize, ) -> Result ``` `devices` are positions in what [`gpu_devices`](#gpu_devices) lists, and [`gpu_class`](#gpu_class) pins the class every one of them dispatches as, where [`None`](https://www.unum.cloud/docs/uform/rust/training.md#None) derives theirs. The host decoders take 7/8 of the cores local to `memory_domains` and dispatch as `cpu_class`, where [`None`](https://www.unum.cloud/docs/uform/rust/training.md#None) derives the CPU's own. `streams` is empty, or holds one CUDA stream per device in `devices` order. Zero `scratch_limit` lets scratch grow as batches ask. Calls `uform_executor_for_vram`. #### Safety Every stream must be a live CUDA stream on its device that outlives the executor. ## drain Waits for every call submitted to this executor before the drain began, then adopts `mood` whenever it next goes idle; a failure is the executor's own, such as a device fault, never one of a drained call. ```rust pub fn drain(&self, mood: Mood) -> Result<()> ``` Calls `uform_executor_drain`. ## GPUClass The GPU families UForm ships tuned variants for, the device twin of [[`CPUClass`](#CPUClass)]: dispatch reports its pick in this vocabulary, and a VRAM placement may pin one. ```rust pub enum GPUClass ``` Mirrors `uform_gpu_class_t`. - `NvidiaAmpere` = `1`: Ampere-class NVIDIA and every GPU without a class of its own - A100, A10 & RTX 30. - `NvidiaAda` = `2`: Ada-class NVIDIA - L4, L40S & RTX 40: the fp8 conversions. - `NvidiaHopper` = `3`: Hopper-class NVIDIA - H100 & H200. - `NvidiaBlackwell` = `4`: Blackwell-class NVIDIA - B200, B300 & GB300. - `NvidiaBlackwellRtx` = `5`: Consumer Blackwell NVIDIA - RTX 50, RTX PRO 6000 & DGX Spark. - `NvidiaRubin` = `6`: Rubin-class NVIDIA - the Vera-Rubin platform's device half. - `AmdCdna3` = `7`: CDNA3 AMD - MI300X & MI325X: gfx942, FNUZ fp8. - `AmdCdna4` = `8`: CDNA4 AMD - MI350X & MI355X: gfx950, OCP fp8. - `AmdCdna5` = `9`: CDNA5 AMD - MI400: gfx1250. - `AppleMetal7` = `10`: Apple7-family GPUs and every Apple GPU without a class of its own - M1 & M2. - `AppleMetal9` = `11`: Apple9-family GPUs - M3 & M4. - `AppleMetal10` = `12`: Apple10-family GPUs - M5. Implements: `Clone`, `Copy`, `StructuralPartialEq`, `PartialEq`, `Eq`, `Debug`, `Hash`, `Display`, `FromStr` ## GPUDevice One GPU as UForm numbers it: its position in what [`gpu_devices`](#gpu_devices) lists is the ordinal every VRAM placement names it by. ```rust pub struct GPUDevice(_); ``` Mirrors `uform_gpu_device_t`. Implements: `Clone`, `Copy`, `Default`, `Debug` ## vendor The vendor whose runtime drives the device. ```rust pub fn vendor(&self) -> Option ``` ## vendor_ordinal The device's ordinal in its vendor's runtime, as `nvidia-smi` or `rocm-smi` list it. ```rust pub fn vendor_ordinal(&self) -> u32 ``` ## gpu_class The class whose kernels run on it. ```rust pub fn gpu_class(&self) -> Option ``` ## compute_units Streaming multiprocessors or compute units. ```rust pub fn compute_units(&self) -> u32 ``` ## memory_bytes Device memory in bytes. ```rust pub fn memory_bytes(&self) -> u64 ``` ## name The vendor's own name for the device, cut before a character the truncation split. ```rust pub fn name(&self) -> &str ``` ## GPUVendor The GPU vendors one binary drives at once. ```rust pub enum GPUVendor ``` Mirrors `uform_gpu_vendor_t`. - `Nvidia` = `1`: NVIDIA, through the CUDA driver. - `Amd` = `2`: AMD, through the HIP runtime. - `Apple` = `3`: Apple, through Metal. Implements: `Clone`, `Copy`, `StructuralPartialEq`, `PartialEq`, `Eq`, `Debug`, `Hash` ## capabilities_compiled Comma-separated hardware capabilities the library was compiled with. ```rust pub fn capabilities_compiled() -> &'static str ``` Calls `uform_capabilities_compiled`. ## capabilities_available Comma-separated hardware capabilities this machine offers at runtime. ```rust pub fn capabilities_available() -> &'static str ``` Calls `uform_capabilities_available`. ## gpu_devices Lists every GPU this process can drive into `devices`, NVIDIA devices first, then AMD, and answers how many there are, which may exceed what `devices` holds. ```rust pub fn gpu_devices(devices: &mut [GPUDevice]) -> Result ``` A vendor this build does not compile, or whose runtime is absent, lists no devices, so a machine without GPUs answers zero. Calls `uform_gpu_devices`.