# Executors > UForm for C An executor owns what a call runs on but a model does not: host threads or device streams, activation scratch, and the pinned buffer staging passes through. - `uform_executor_for_ram` creates an executor of host threads across the compute domains local to the memory domains of its `uform_ram_options_t`, 7/8 of their cores when `threads` is zero. - `uform_executor_on_pool` lends UForm a ForkUnion `fu_pool_t` another library spawned, so UForm, USearch and UStore in one process share one set of threads; the pool must outlive the executor. - `uform_executor_for_vram` opens one stream on every device of its `uform_vram_options_t`, named by their position in `uform_gpu_devices`, beside host threads for the decoding the devices cannot do. - `uform_executor_drain` waits until no call holds the executor, then applies the `uform_mood_t` its threads keep until the next call: parked sleeps in the kernel, spinning keeps polling. - `uform_executor_free` waits the same way, releases what the executor owns and returns what it borrowed. One executor serves any number of models of its residency, and a call pairing a model with an executor of another residency reports `uform_status_device_k`. A `NULL` executor runs the call on the calling thread, final when it returns. `cpu_class` and `gpu_class` in the options pin the class a call dispatches as, and `uform_status_unsupported_k` answers a class the machine cannot run. ### Classes and Devices Dispatch picks one tuned variant per CPU or GPU family, named by `uform_cpu_class_t` and `uform_gpu_class_t`, whose enumerators and ordinals are USearch's. `uform_ram_options_t::cpu_class` and `uform_vram_options_t::gpu_class` pin one, and the `unknown` zero derives the hardware's own: - A CPU pin dispatches as if the CPU were of that class, taking every rung whose features it holds, and a class the CPU cannot run reports `uform_status_unsupported_k`. - A GPU pin dispatches every named device as that class, and a device that cannot run it reports `uform_status_unsupported_k`. - Unpinned devices of one VRAM executor must share a class, or the call reports `uform_status_config_mismatch_k`. `uform_gpu_devices` lists every GPU the process can drive, NVIDIA devices first, then AMD, each as a `uform_gpu_device_t` of its vendor, vendor ordinal, class, compute units, memory and name. A device's position in that listing is the ordinal `uform_vram_options_t::devices` names it by, and a machine without GPUs lists none and succeeds. `uform_cpu_class_name` and `uform_gpu_class_name` spell a class, like `intel-haswell` or `nvidia-hopper`, and the `_from_name` verbs read one back. `#include ` `#include ` ## uform_cpu_class_t The CPU families UForm ships tuned variants for - dispatch reports its pick in this vocabulary, and callers may pin one. ```c enum uform_cpu_class_t ``` The enumerators and their ordinals are USearch's. - `uform_cpu_class_unknown_k` = `0` - `uform_cpu_class_serial_k`: The portable floor: no SIMD assumptions. - `uform_cpu_class_intel_haswell_k`: Haswell-class x86: AVX2 & FMA, the 256-bit floor. - `uform_cpu_class_intel_sierra_k`: Sierra Forest-class x86 - the E-core Xeons: AVX-VNNI and its successors without AVX-512. - `uform_cpu_class_intel_skylake_k`: Skylake-class x86: the AVX-512 foundation tier. - `uform_cpu_class_intel_icelake_k`: Icelake-class x86: AVX-512 VNNI & VPOPCNT. - `uform_cpu_class_amd_genoa_k`: Genoa-class x86 - Zen 4 and newer: native bf16 dot products over AVX-512. - `uform_cpu_class_intel_sapphire_k`: Sapphire Rapids-class x86: the Genoa bundle plus AVX-512 FP16 and the AMX-BF16 & AMX-INT8 tiles. - `uform_cpu_class_intel_granite_k`: Granite Rapids-class x86: the Sapphire Rapids bundle plus the AMX-FP16 tiles. - `uform_cpu_class_intel_diamond_k`: Diamond Rapids-class x86: the Granite Rapids bundle plus AVX10.2's FP8 conversions. - `uform_cpu_class_aws_g2_k`: Graviton2-class Arm - Neoverse N1, the bare NEON floor; phones predating Armv8.6 land here too. - `uform_cpu_class_apple_m2_k`: Armv8.6+ NEON: dot products, bf16 & f16 fused multiplies - Apple M2, Cortex-A710 & X2 class silicon. - `uform_cpu_class_apple_m4_k`: Apple M4-class Arm: the Armv8.6 NEON bundle plus the SME outer-product tiles. - `uform_cpu_class_aws_g5_k`: Graviton5-class Arm - Neoverse V3, Armv9.2; reserved, as its kits equal `apple_m2's`. - `uform_cpu_class_nvidia_vera_k`: Vera-class Arm - NVIDIA's Armv9.2-generation cores with FP8 dot products. - `uform_cpu_class_alibaba_c930_k`: Alibaba C930-class RISC-V - T-Head's XuanTie core, RVA23 vectors. - `uform_cpu_class_wasm32_v128_k`: WebAssembly every engine loads: 32-bit, single-threaded, the fixed-width simd128 tier. - `uform_cpu_class_wasm64_extended_k`: WebAssembly with relaxed SIMD, shared memory for real thread pools, and 64-bit addressing. - `uform_cpu_class_monoversioned_k`: The one class of a monoversioned build: every capability its floor implies or its preset vouches for, as the kernel libraries compiled them. Spellings: ``` char const * uform_cpu_class_name(uform_cpu_class_t cpu_class); ``` ``` uform_cpu_class_t uform_cpu_class_from_name(uform_bytes_view_t name); ``` ## uform_gpu_class_t The GPU families UForm ships tuned variants for, the device twin of `[uform_cpu_class_t](#uform_cpu_class_t):` one class per generation of kernel kits. ```c enum uform_gpu_class_t ``` The enumerators and their ordinals are USearch's. - `uform_gpu_class_unknown_k` = `0` - `uform_gpu_class_nvidia_ampere_k`: Ampere-class NVIDIA and every GPU without a class of its own - A100, A10 & RTX 30. - `uform_gpu_class_nvidia_ada_k`: Ada-class NVIDIA - L4, L40S & RTX 40: the fp8 conversions. - `uform_gpu_class_nvidia_hopper_k`: Hopper-class NVIDIA - H100 & H200. - `uform_gpu_class_nvidia_blackwell_k`: Blackwell-class NVIDIA - B200, B300 & GB300. - `uform_gpu_class_nvidia_blackwellrtx_k`: Consumer Blackwell NVIDIA - RTX 50, RTX PRO 6000 & DGX Spark. - `uform_gpu_class_nvidia_rubin_k`: Rubin-class NVIDIA - the Vera-Rubin platform's device half. - `uform_gpu_class_amd_cdna3_k`: CDNA3 AMD - MI300X & MI325X: gfx942, FNUZ fp8. - `uform_gpu_class_amd_cdna4_k`: CDNA4 AMD - MI350X & MI355X: gfx950, OCP fp8. - `uform_gpu_class_amd_cdna5_k`: CDNA5 AMD - MI400: gfx1250. - `uform_gpu_class_apple_metal7_k`: Apple7-family GPUs and every Apple GPU without a class of its own - M1 & M2. - `uform_gpu_class_apple_metal9_k`: Apple9-family GPUs - M3 & M4. - `uform_gpu_class_apple_metal10_k`: Apple10-family GPUs - M5. Spellings: ``` char const * uform_gpu_class_name(uform_gpu_class_t gpu_class); ``` ``` uform_gpu_class_t uform_gpu_class_from_name(uform_bytes_view_t name); ``` ## uform_gpu_vendor_t The GPU vendors one binary drives at once. ```c enum uform_gpu_vendor_t ``` - `uform_gpu_vendor_unknown_k` = `0` - `uform_gpu_vendor_nvidia_k`: NVIDIA, through the CUDA driver. - `uform_gpu_vendor_amd_k`: AMD, through the HIP runtime. - `uform_gpu_vendor_apple_k`: Apple, through Metal. ## uform_gpu_device_t One GPU as UForm numbers it: its position in [`uform_gpu_devices`](#uform_gpu_devices) is the UForm device ordinal every VRAM call names. ```c struct uform_gpu_device_t ``` - `vendor` `uform_gpu_vendor_t`: The vendor whose runtime drives the device. - `vendor_ordinal` `uint32_t`: The device's ordinal in its vendor's runtime, as `nvidia-smi` or `rocm-smi` list it. - `gpu_class` `uform_gpu_class_t`: The class whose kernels run on it. - `compute_units` `uint32_t`: Streaming multiprocessors or compute units. - `memory_bytes` `uint64_t`: Device memory in bytes. - `name` `char`: The vendor's own name for the device, NUL-terminated and truncated to fit. ## uform_ram_options_t The host memory domains a RAM-resident model and executor place their bytes in - one NUMA node each. ```c struct uform_ram_options_t ``` - `memory_domains` `int32_t const *`: NUMA node IDs, borrowed for the duration of the call; `NULL` derives every node the process may allocate on. - `memory_domains_count` `size_t`: Number of entries in `memory_domains`; zero derives. - `cpu_class` `uform_cpu_class_t`: The class to dispatch as if the CPU were of it, admitting every rung whose features it holds; [`uform_cpu_class_unknown_k`](#uform_cpu_class_unknown_k) derives the CPU's own. ## uform_vram_options_t The devices a VRAM-resident model, trainer and executor place their bytes in. ```c struct uform_vram_options_t ``` - `devices` `int32_t const *`: UForm device ordinals, positions in the listing [`uform_gpu_devices`](#uform_gpu_devices) reports, borrowed for the duration of the call; `NULL` derives one free device. - `devices_count` `size_t`: Number of entries in `devices`; zero derives. - `gpu_class` `uform_gpu_class_t`: The class every named device dispatches as; [`uform_gpu_class_unknown_k`](#uform_gpu_class_unknown_k) derives the devices' own, which must agree, or the call reports [`uform_status_config_mismatch_k`](https://www.unum.cloud/docs/uform/c/errors.md#uform_status_config_mismatch_k). ## uform_executor_t Everything a call needs to run but a model does not own: the host threads or device streams that advance the work, the activation scratch, and the pinned bounce buffer staging passes through. ```c typedef struct uform_executor_handle_t * uform_executor_t; ``` One executor serves any number of models of its residency, so several models share one machine's cores or devices instead of each claiming a private set; a mismatch reports [`uform_status_device_k`](https://www.unum.cloud/docs/uform/c/errors.md#uform_status_device_k). Scratch grows to the largest batch asked of it, is rewound per call, and is kept between calls. Calls on one executor run in submission order. Every executor tracks its deferred calls in a fixed table sized at creation, so submitting never allocates. The library stages only to change layout on the side that computes, or to make pageable memory DMA-able: | Case | Action | |:-|:-| | Device inputs on the executor's device | read in place | | Pinned or managed host inputs | a direct asynchronous upload | | Pageable host inputs | the pinned bounce buffer, then an upload | | Encoded media needing a host decoder | decoded on the executor's host threads, then uploaded | | Outputs in an arena on the executor's device | the kernel writes them directly | ## uform_mesh_options_t Where this process sits among the processes training one model, and how they meet. ```c struct uform_mesh_options_t ``` - `host_domain_index` `size_t`: This process's index among `host_domain_count`. - `host_domain_count` `size_t`: Processes in the run, one per host; zero derives one, a run with no peers. - `gpus_per_multimem_domain` `size_t`: GPUs one NVLink multicast domain spans, a fabric property no local probe sees; zero derives the host. - `directory` `uform_bytes_view_t`: The path of the directory every host sees, where processes exchange their facts; required with peers. - `job` `uform_bytes_view_t`: The run's unique name, naming one directory inside `directory`; required with peers. - `timeout_seconds` `double`: Seconds any wait for a peer lasts before the call fails; zero derives 120. ## uform_mesh_t One process's place among the processes of a run: the topology every process agreed on at bootstrap, and the InfiniBand domain joining its NVLink domains. ```c typedef struct uform_mesh_handle_t * uform_mesh_t; ``` The process-wide half of a run's transport, made once and borrowed by every trainer, so no consumer wires its own adapters. The NVLink multicast domain is not here: its size is the sum of what its consumers reserve, so each trainer makes its own over the mesh's devices. Bootstrap is a one-time rendezvous through files in `directory`; after it, processes talk only over NVLink and InfiniBand. ## uform_multimem_tier_t How a mesh's NVLink domains reduce. ```c enum uform_multimem_tier_t ``` - `uform_multimem_tier_switch_reduced_k` = `0`: The NVLink switch reduces through multicast memory. - `uform_multimem_tier_peer_read_k` = `1`: Every member reads its peers in a fixed order, where multicast is absent; bit-identical results. Spellings: ``` char const * uform_multimem_tier_name(uform_multimem_tier_t tier); ``` ``` uform_multimem_tier_t uform_multimem_tier_from_name( uform_bytes_view_t name); ``` ## uform_mesh_description_t What a mesh reports about itself. ```c struct uform_mesh_description_t ``` - `gpus_per_host_domain` `size_t`: GPUs one process drives. - `gpus_per_multimem_domain` `size_t`: GPUs one NVLink domain spans. - `gpus_per_colocated_domain` `size_t`: GPUs in the whole run. - `host_domain_index` `size_t`: This process's index among the run's processes. - `first_gpu_index` `size_t`: The run-wide index of this process's first GPU, where its share of a sharded dataset begins. - `multimem_tier` `uform_multimem_tier_t`: How its NVLink domains reduce. - `gpu_class` `uform_gpu_class_t`: The class every GPU of the run dispatches as: the executor's pin, or the devices' own. ## uform_gpu_devices Lists every GPU this process can drive, NVIDIA devices first, then AMD, so the position of a device is the UForm ordinal `[uform_vram_options_t](#uform_vram_options_t)` names it by. ```c void uform_gpu_devices(uform_gpu_devices_t *arguments); ``` A vendor this build does not compile, or whose runtime is absent, lists no devices, so the call succeeds with a zero count on a machine without GPUs. - `outcome`: Output for the outcome, final when the call returns. - `devices`: Output for up to `devices_capacity` devices, in UForm ordinal order; may be `NULL`. - `devices_capacity`: Entries `devices` holds. - `devices_count`: Output for every device this process can drive, which may exceed `devices_capacity`. ## uform_executor_for_ram Spawns host threads for RAM-resident models, interleaved across the compute domains local to the named memory domains. ```c void uform_executor_for_ram(uform_executor_for_ram_t *arguments); ``` - `executor`: Output for the new executor. - `outcome`: Output for the outcome, final once its status leaves [`uform_status_deferred_k`](https://www.unum.cloud/docs/uform/c/errors.md#uform_status_deferred_k). - `ram`: The memory domains of the models it will drive; copied, not borrowed. - `threads`: Threads to spawn; zero takes 7/8 of the cores those domains carry. - `deferred_limit`: Deferred calls it tracks at once, sizing its deferred table; zero takes a default. ## uform_executor_on_pool Creates a RAM executor over a borrowed ForkUnion pool, so UForm, USearch and UStore in one process share one set of threads while each keeps its own scratch. ```c void uform_executor_on_pool(uform_executor_on_pool_t *arguments); ``` - `executor`: Output for the new executor. - `outcome`: Output for the outcome, final once its status leaves [`uform_status_deferred_k`](https://www.unum.cloud/docs/uform/c/errors.md#uform_status_deferred_k). - `ram`: The memory domains of the models it will drive; copied, not borrowed. - `pool`: The ForkUnion `fu_pool_t` to borrow; it must outlive the executor, and keeps its lender's mood. - `deferred_limit`: Deferred calls it tracks at once, sizing its deferred table; zero takes a default. ## uform_executor_for_vram Opens one submission stream on every named device, all of them driven by whichever host thread submits; borrowing the caller's streams reports [`uform_status_unsupported_k`](https://www.unum.cloud/docs/uform/c/errors.md#uform_status_unsupported_k) for now. ```c void uform_executor_for_vram(uform_executor_for_vram_t *arguments); ``` Text and JPEG images never leave the device; the rest of decoding - container demux, PNG and WebP, audio, and every modality on AMD - runs on host threads the executor spawns beside its streams, 7/8 of the cores local to the memory domains `ram` names, dispatching as its `cpu_class` . A GPU executor never changes process-wide device flags. - `executor`: Output for the new executor. - `outcome`: Output for the outcome, final once its status leaves [`uform_status_deferred_k`](https://www.unum.cloud/docs/uform/c/errors.md#uform_status_deferred_k). - `vram`: The devices of the models it will drive; copied, not borrowed. - `ram`: The memory domains and the CPU class of its host decoders; copied, not borrowed. - `streams`: The __optional__ borrowed CUDA streams, one per device in `vram` order; `NULL` creates its own. - `scratch_limit`: The __optional__ ceiling on scratch bytes per device; zero lets scratch grow as batches ask. - `deferred_limit`: Deferred calls it tracks at once, sizing its deferred table; zero takes a default. ## uform_executor_drain Waits for every call submitted before the drain began, so later submissions can't hold it up, then sets the mood the executor adopts whenever it next goes idle. ```c void uform_executor_drain(uform_executor_drain_t *arguments); ``` Reports only executor-level failures. - `executor`: The executor whose submitted calls are awaited. - `outcome`: Output for an executor-level failure, such as a device fault; never one of a drained call. - `mood`: What the executor does whenever it next goes idle; the last drain wins. ## uform_executor_free Drains every call, releases the scratch and the streams it owns, returns borrowed streams and pools, and frees the executor; `NULL` is safe, and it cannot fail, so takes no outcome. ```c void uform_executor_free(uform_executor_t executor); ``` ## uform_capabilities_compiled The hardware capabilities this binary was compiled with, as a comma-separated static string. ```c char const * uform_capabilities_compiled(void); ``` ## uform_capabilities_available The hardware capabilities the current machine offers at runtime, as a comma-separated static string. ```c char const * uform_capabilities_available(void); ``` ## uform_mesh_bootstrap Joins this process to the run's mesh; every process of the run calls it with the same count, multicast extent, directory, job and GPU class, or the call reports [`uform_status_config_mismatch_k`](https://www.unum.cloud/docs/uform/c/errors.md#uform_status_config_mismatch_k). ```c void uform_mesh_bootstrap(uform_mesh_bootstrap_t *arguments); ``` - `mesh`: Output for the new mesh, written once every process has joined. - `executor`: A VRAM executor whose devices are this process's members of the mesh. - `outcome`: Output for the outcome, final once its status leaves [`uform_status_deferred_k`](https://www.unum.cloud/docs/uform/c/errors.md#uform_status_deferred_k). - `completion`: Whether the call blocks or returns once queued; zero blocks. - `options`: Where this process sits and how the processes meet. ## uform_mesh_describe Reports a mesh's shape and this process's place in it. ```c void uform_mesh_describe(uform_mesh_describe_t *arguments); ``` - `mesh`: The mesh to be described. - `outcome`: Output for the outcome, final once its status leaves [`uform_status_deferred_k`](https://www.unum.cloud/docs/uform/c/errors.md#uform_status_deferred_k). - `description`: Output, filled whole. ## uform_mesh_free Leaves the mesh; `NULL` is safe, and it cannot fail, so takes no outcome. ```c void uform_mesh_free(uform_mesh_t mesh); ``` Every trainer borrowing it must be freed first.