pub struct MeshMirrors uform_mesh_t; dropping it calls uform_mesh_free, after every trainer borrowing it.
Implements
Debug, Send, Sync, DropAn executor owns host threads or device streams, the activation scratch, and the pinned buffers staging passes through, grown to a high-water mark. One executor serves any number of models of its residency.
let threads = Executor::for_ram(&[], None, 8)?; // eight threads, where zero takes 7/8 of the cores
let domains = Executor::for_ram(&[0, 1], None, 0)?; // threads interleaved across two NUMA nodes
let serial = Executor::for_ram(&[], Some(CPUClass::Serial), 0)?; // dispatches as the portable floor
let streams = unsafe { Executor::for_vram(&[0, 1], None, &[], None, &[], 0)? }; // one stream per device, scratch grown as asked
let shared = unsafe { Executor::on_pool(&[], None, pool, 0)? }; // borrows a ForkUnion pool's threads
let local = Executor::default(); // owns nothing, runs on the calling thread
threads.drain(Mood::Spinning)?; // waits until no call holds it, then keeps spinningThe second argument pins a CPUClass or a GPUClass to dispatch as, and None derives the hardware’s own. gpu_devices(&mut devices) fills a slice of GPUDevice and answers how many GPUs the process can drive, and a device’s position there is the ordinal for_vram and load_vram name it by. for_vram and on_pool are unsafe, since the streams or the pool they borrow must outlive the executor; on_pool mirrors the C verb and answers Status::Unsupported until borrowing is designed, as for_vram does for borrowed streams.
An executor runs one call at a time, so two threads that genuinely overlap take one apiece; an executor is Send but not Sync, while models stay freely shared.
Dropping an executor drains it, unless a GPU arena it made still holds it, which then does so on its own drop.
API
StructMeshOne process's place among the processes of a run: the topology every process agreed on at bootstrap, and the InfiniBand domain joining its NVLink domains, borrowed by every trainer.2
bootstrapJoins this process to the run's mesh through a one-time rendezvous in options.directory; every process of the run calls it with the same count, multicast extent, directory and job, over a VRAM executor whose devices are this process's members.
pub fn bootstrap(executor: &Executor, options: &MeshOptions<'_>) -> Result<Self>Calls uform_mesh_bootstrap.
describeReports the mesh's shape and this process's place in it.
StructMeshDescriptionWhat a mesh reports about itself.
pub struct MeshDescriptionMirrors uform_mesh_description_t.
Fields
gpus_per_host_domain | usize | GPUs one process drives. |
|---|---|---|
gpus_per_multimem_domain | usize | GPUs one NVLink domain spans. |
gpus_per_colocated_domain | usize | GPUs in the whole run. |
host_domain_index | usize | This process's index among the run's processes. |
first_gpu_index | usize | The run-wide index of this process's first GPU, where its share of a sharded dataset begins. |
multimem_tier | MultimemTier | How its NVLink domains reduce. |
gpu_class | Option<GPUClass> | The class every GPU of the run dispatches as: the executor's pin, or the devices' own. |
Implements
Clone, Copy, StructuralPartialEq, PartialEq, Eq, Debug, HashStructMeshOptionsWhere this process sits among the processes training one model, and how they meet.
pub struct MeshOptions<'options>Mirrors uform_mesh_options_t.
Fields
host_domain_index | usize | This process's index among host_domain_count. |
|---|---|---|
host_domain_count | usize | Processes in the run, one per host; zero derives one, a run with no peers. |
gpus_per_multimem_domain | usize | GPUs one NVLink multicast domain spans, a fabric property no local probe sees; zero derives the host. |
directory | &'options Path | The directory every host sees, where processes exchange their facts; required with peers. |
job | &'options str | The run's unique name inside directory; required with peers. |
timeout_seconds | f64 | Seconds any wait for a peer lasts before the call fails; zero derives 120. |
Implements
Clone, Copy, StructuralPartialEq, PartialEq, Debug, DefaultEnumMultimemTierHow a mesh's NVLink domains reduce.
pub enum MultimemTierMirrors uform_multimem_tier_t.
Values
SwitchReduced | 0 | The NVLink switch reduces through multicast memory. |
|---|---|---|
PeerRead | 1 | Every member reads its peers in a fixed order, where multicast is absent; bit-identical results. |
Implements
Clone, Copy, StructuralPartialEq, PartialEq, Eq, Debug, Hash, Display, FromStrEnumCPUClassThe CPU families UForm ships tuned variants for: dispatch reports its pick in this vocabulary, and a RAM placement may pin one.
pub enum CPUClassMirrors uform_cpu_class_t.
Values
Serial | 1 | The portable floor: no SIMD assumptions. |
|---|---|---|
IntelHaswell | 2 | Haswell-class x86: AVX2 & FMA, the 256-bit floor. |
IntelSierra | 3 | Sierra Forest-class x86 - the E-core Xeons: AVX-VNNI and its successors without AVX-512. |
IntelSkylake | 4 | Skylake-class x86: the AVX-512 foundation tier. |
IntelIcelake | 5 | Icelake-class x86: AVX-512 VNNI & VPOPCNT. |
AmdGenoa | 6 | Genoa-class x86 - Zen 4 and newer: native bf16 dot products over AVX-512. |
IntelSapphire | 7 | Sapphire Rapids-class x86: the Genoa bundle plus AVX-512 FP16 and the AMX-BF16 & AMX-INT8 tiles. |
IntelGranite | 8 | Granite Rapids-class x86: the Sapphire Rapids bundle plus the AMX-FP16 tiles. |
IntelDiamond | 9 | Diamond Rapids-class x86: the Granite Rapids bundle plus AVX10.2's FP8 conversions. |
AwsG2 | 10 | Graviton2-class Arm - Neoverse N1, the bare NEON floor; phones predating Armv8.6 land here too. |
AppleM2 | 11 | Armv8.6+ NEON: dot products, bf16 & f16 fused multiplies - Apple M2, Cortex-A710 & X2 class silicon. |
AppleM4 | 12 | Apple M4-class Arm: the Armv8.6 NEON bundle plus the SME outer-product tiles. |
AwsG5 | 13 | Graviton5-class Arm - Neoverse V3, Armv9.2. |
NvidiaVera | 14 | Vera-class Arm - NVIDIA's Armv9.2-generation cores with FP8 dot products. |
AlibabaC930 | 15 | Alibaba C930-class RISC-V - T-Head's XuanTie core, RVA23 vectors. |
Wasm32V128 | 16 | WebAssembly every engine loads: 32-bit, single-threaded, the fixed-width simd128 tier. |
Wasm64Extended | 17 | WebAssembly with relaxed SIMD, shared memory for real thread pools, and 64-bit addressing. |
Monoversioned | 18 | The one class of a monoversioned build: every capability its floor implies or its preset vouches for, as the kernel libraries compiled them. |
Implements
Clone, Copy, StructuralPartialEq, PartialEq, Eq, Debug, Hash, Display, FromStrStructExecutorThe host threads or device streams an On runs its calls on, with the activation scratch and the staging buffers each of them uses.4
pub struct ExecutorOne executor serves any number of models of its residency, so several models share one machine's cores or devices instead of each claiming a private set.
Executor::default is the C interface's null executor: it owns nothing, and every call
handed it runs on the calling thread.
Calls on one executor run in submission order, so threads wanting to overlap take one apiece;
it is Send but not Sync.
Every constructor takes a deferred_limit sizing its table of deferred calls, where zero takes
the default; a deferred submission finding the table full fails with Status::Contended.
Arena::new_in places an arena where it computes: a host arena may outlive it, while a GPU
arena holds it until the arena drops.
Mirrors uform_executor_t; dropping its last holder calls uform_executor_free.
Implements
Debug, Send, Defaultfor_ramSpawns host threads for RAM-resident models, interleaved across the compute domains local to memory_domains, which is the same list the models were loaded with.
pub fn for_ram(
memory_domains: &[i32],
cpu_class: Option<CPUClass>,
threads: usize,
deferred_limit: usize,
) -> Result<Self>Zero threads takes 7/8 of the cores those domains carry, and cpu_class pins the class
its calls dispatch as, where None derives the CPU's own.
Calls uform_executor_for_ram.
on_poolCreates a RAM executor over a borrowed ForkUnion pool, so UForm, USearch and UStore in one process share one set of threads while each keeps its own scratch.
pub unsafe fn on_pool(
memory_domains: &[i32],
cpu_class: Option<CPUClass>,
pool: *mut c_void,
deferred_limit: usize,
) -> Result<Self>Calls uform_executor_on_pool.
Safety pool must address a live ForkUnion fu_pool_t that outlives the executor.
for_vramOpens one submission stream on every named device, or borrows the caller's, all of them driven by whichever host thread submits - which is why this takes no thread count.
pub unsafe fn for_vram(
devices: &[i32],
gpu_class: Option<GPUClass>,
memory_domains: &[i32],
cpu_class: Option<CPUClass>,
streams: &[*mut c_void],
scratch_limit: usize,
deferred_limit: usize,
) -> Result<Self>devices are positions in what gpu_devices lists, and gpu_class pins the class every
one of them dispatches as, where None derives theirs. The host decoders take 7/8 of the
cores local to memory_domains and dispatch as cpu_class, where None derives the CPU's
own. streams is empty, or holds one CUDA stream per device in devices order. Zero
scratch_limit lets scratch grow as batches ask.
Calls uform_executor_for_vram.
Safety Every stream must be a live CUDA stream on its device that outlives the executor.
drainWaits for every call submitted to this executor before the drain began, then adopts mood whenever it next goes idle; a failure is the executor's own, such as a device fault, never one of a drained call.
EnumGPUClassThe GPU families UForm ships tuned variants for, the device twin of [CPUClass]: dispatch reports its pick in this vocabulary, and a VRAM placement may pin one.
pub enum GPUClassMirrors uform_gpu_class_t.
Values
NvidiaAmpere | 1 | Ampere-class NVIDIA and every GPU without a class of its own - A100, A10 & RTX 30. |
|---|---|---|
NvidiaAda | 2 | Ada-class NVIDIA - L4, L40S & RTX 40: the fp8 conversions. |
NvidiaHopper | 3 | Hopper-class NVIDIA - H100 & H200. |
NvidiaBlackwell | 4 | Blackwell-class NVIDIA - B200, B300 & GB300. |
NvidiaBlackwellRtx | 5 | Consumer Blackwell NVIDIA - RTX 50, RTX PRO 6000 & DGX Spark. |
NvidiaRubin | 6 | Rubin-class NVIDIA - the Vera-Rubin platform's device half. |
AmdCdna3 | 7 | CDNA3 AMD - MI300X & MI325X: gfx942, FNUZ fp8. |
AmdCdna4 | 8 | CDNA4 AMD - MI350X & MI355X: gfx950, OCP fp8. |
AmdCdna5 | 9 | CDNA5 AMD - MI400: gfx1250. |
AppleMetal7 | 10 | Apple7-family GPUs and every Apple GPU without a class of its own - M1 & M2. |
AppleMetal9 | 11 | Apple9-family GPUs - M3 & M4. |
AppleMetal10 | 12 | Apple10-family GPUs - M5. |
Implements
Clone, Copy, StructuralPartialEq, PartialEq, Eq, Debug, Hash, Display, FromStrStructGPUDeviceOne GPU as UForm numbers it: its position in what gpu_devices lists is the ordinal every VRAM placement names it by.6
pub struct GPUDevice(_);Mirrors uform_gpu_device_t.
Implements
Clone, Copy, Default, DebugvendorThe vendor whose runtime drives the device.
pub fn vendor(&self) -> Option<GPUVendor>vendor_ordinalThe device's ordinal in its vendor's runtime, as nvidia-smi or rocm-smi list it.
pub fn vendor_ordinal(&self) -> u32gpu_classThe class whose kernels run on it.
pub fn gpu_class(&self) -> Option<GPUClass>compute_unitsStreaming multiprocessors or compute units.
pub fn compute_units(&self) -> u32memory_bytesDevice memory in bytes.
pub fn memory_bytes(&self) -> u64nameThe vendor's own name for the device, cut before a character the truncation split.
pub fn name(&self) -> &strEnumGPUVendorThe GPU vendors one binary drives at once.
pub enum GPUVendorMirrors uform_gpu_vendor_t.
Values
Nvidia | 1 | NVIDIA, through the CUDA driver. |
|---|---|---|
Amd | 2 | AMD, through the HIP runtime. |
Apple | 3 | Apple, through Metal. |
Implements
Clone, Copy, StructuralPartialEq, PartialEq, Eq, Debug, HashFunctions
capabilities_compiledComma-separated hardware capabilities the library was compiled with.
capabilities_availableComma-separated hardware capabilities this machine offers at runtime.
gpu_devicesLists every GPU this process can drive into devices, NVIDIA devices first, then AMD, and answers how many there are, which may exceed what devices holds.
pub fn gpu_devices(devices: &mut [GPUDevice]) -> Result<usize>A vendor this build does not compile, or whose runtime is absent, lists no devices, so a machine without GPUs answers zero.
Calls uform_gpu_devices.