Executors | UForm for Mojo

One executor serves any number of models of its residency. Executor(library) owns nothing and runs every call handed it on the calling thread. There is a constructor per C one, and each fixes a kind that must match the residency of every model it drives:

var threads = Executor.for_ram(library, RamOptions.on(domains), threads=8)  # zero takes 7/8 of the cores
var shared = Executor.on_pool(library, fork_union_pool)                       # borrows a fu_pool_t
var streams = Executor.for_vram(library, VramOptions.on(devices))             # one host thread, a stream per device
threads.drain(Mood.SPINNING)                                                # waits until no call holds it, then keeps spinning

An executor runs one call at a time, so threads that genuinely overlap take one executor apiece. A call finding another one holding its executor finishes at once with Status.CONTENDED.

API

StructMultimemTierHow a mesh's NVLink domains reduce, uform_multimem_tier_t.

struct MultimemTier

Fields

`code`	`c_int`	The raw `uform_multimem_tier_t` value the ABI exchanges.

Values

`SWITCH_REDUCED`	`MultimemTier(Int32(0))`	The NVLink switch reduces through multicast memory.
`PEER_READ`	`MultimemTier(Int32(1))`	Every member reads its peers in a fixed order, where multicast is absent.

Conforms to

Copyable, Deinitable, Equatable, ImplicitlyCopyable, Movable, RegisterPassable, TrivialRegisterPassable

StructMeshDescriptionWhat a mesh reports about itself, uform_mesh_description_t.1

struct MeshDescription

Fields

`gpus_per_host_domain`	`c_size_t`	GPUs one process drives.
`gpus_per_multimem_domain`	`c_size_t`	GPUs one NVLink domain spans.
`gpus_per_colocated_domain`	`c_size_t`	GPUs in the whole run.
`host_domain_index`	`c_size_t`	This process's index among the run's processes.
`first_gpu_index`	`c_size_t`	The run-wide index of this process's first GPU, where its share of a sharded dataset begins.
`multimem_tier`	`MultimemTier`	How its NVLink domains reduce.
`gpu_class`	`GPUClass`	The class every GPU of the run dispatches as.

Conforms to

Copyable, Defaultable, Deinitable, ImplicitlyCopyable, Movable

__init__Zeroes every field, which is what a failed report leaves.

def __init__(out self)

StructMeshOptionsWhere this process sits among the processes training one model, and how they meet, uform_mesh_options_t.1

struct MeshOptions

Fields

`host_domain_index`	`Int`	This process's index among `host_domain_count`.
`host_domain_count`	`Int`	Processes in the run, one per host; zero derives one, a run with no peers.
`gpus_per_multimem_domain`	`Int`	GPUs one NVLink multicast domain spans; zero derives the host.
`directory`	`String`	The directory every host sees, where processes exchange their facts; required with peers.
`job`	`String`	The run's unique name inside `directory`; required with peers.
`timeout_seconds`	`Float64`	Seconds any wait for a peer lasts before the call fails; zero derives 120.

Conforms to

Copyable, Defaultable, Deinitable, Movable

__init__A run with no peers.

def __init__(out self)

StructMeshOne process's place among the processes of a run, borrowed by every trainer over the same executor; released when it goes out of scope, after every trainer borrowing it.4

struct Mesh

Fields

`library`	`Library`	The loaded engine, held so the release can still reach `uform_mesh_free`.
`address`	`Address`	The opaque `uform_mesh_t`.

Conforms to

Deinitable, Movable

__init__Adopts a handle the engine just created.

def __init__(out self, *, library: Library, address: Int)

__deinit__Leaves the mesh. Calls uform_mesh_free.

def __deinit__(deinit self)

bootstrapJoins this process to the run's mesh, waiting for every peer. Calls uform_mesh_bootstrap.

def bootstrap(executor: Executor, options: MeshOptions = MeshOptions()) -> Self

Parameters

`executorExecutor`		A device executor whose devices are this process's members of the mesh.
`optionsMeshOptions`		Where this process sits and how the processes meet.

Raises

Error with CONFIG_MISMATCH when peers disagree, or UNSUPPORTED in a build without the trainer.

describeReports the mesh's shape and this process's place in it. Calls uform_mesh_describe.

def describe(self) -> MeshDescription

StructCPUClassThe CPU family dispatch picks, or a placement pins, uform_cpu_class_t.

struct CPUClass

Fields

`code`	`c_int`	The raw `uform_cpu_class_t` value the ABI exchanges.

Values

`UNKNOWN`	`CPUClass(Int32(0))`	Derives the CPU's own class.
`SERIAL`	`CPUClass(Int32(1))`	The portable floor: no SIMD assumptions.
`INTEL_HASWELL`	`CPUClass(Int32(2))`	Haswell-class x86: AVX2 and FMA.
`INTEL_SIERRA`	`CPUClass(Int32(3))`	Sierra Forest-class x86: AVX-VNNI without AVX-512.
`INTEL_SKYLAKE`	`CPUClass(Int32(4))`	Skylake-class x86: the AVX-512 foundation tier.
`INTEL_ICELAKE`	`CPUClass(Int32(5))`	Icelake-class x86: AVX-512 VNNI and VPOPCNT.
`AMD_GENOA`	`CPUClass(Int32(6))`	Genoa-class x86: native bf16 dot products over AVX-512.
`INTEL_SAPPHIRE`	`CPUClass(Int32(7))`	Sapphire Rapids-class x86: AVX-512 FP16 and the AMX-BF16 and AMX-INT8 tiles.
`INTEL_GRANITE`	`CPUClass(Int32(8))`	Granite Rapids-class x86: the Sapphire Rapids bundle plus the AMX-FP16 tiles.
`INTEL_DIAMOND`	`CPUClass(Int32(9))`	Diamond Rapids-class x86: the Granite Rapids bundle plus AVX10.2's FP8 conversions.
`AWS_G2`	`CPUClass(Int32(10))`	Graviton2-class Arm, the bare NEON floor.
`APPLE_M2`	`CPUClass(Int32(11))`	Armv8.6+ NEON: dot products, bf16 and f16 fused multiplies.
`APPLE_M4`	`CPUClass(Int32(12))`	Apple M4-class Arm: the Armv8.6 NEON bundle plus the SME outer-product tiles.
`AWS_G5`	`CPUClass(Int32(13))`	Graviton5-class Arm, Neoverse V3.
`NVIDIA_VERA`	`CPUClass(Int32(14))`	Vera-class Arm with FP8 dot products.
`ALIBABA_C930`	`CPUClass(Int32(15))`	Alibaba C930-class RISC-V with RVA23 vectors.
`WASM32_V128`	`CPUClass(Int32(16))`	WebAssembly every engine loads: 32-bit, single-threaded, simd128.
`WASM64_EXTENDED`	`CPUClass(Int32(17))`	WebAssembly with relaxed SIMD, shared memory, and 64-bit addressing.
`MONOVERSIONED`	`CPUClass(Int32(18))`	The one class of a monoversioned build.

Conforms to

Copyable, Deinitable, Equatable, ImplicitlyCopyable, Movable, RegisterPassable, TrivialRegisterPassable

StructGPUClassThe GPU family dispatch picks, or a placement pins, uform_gpu_class_t.

struct GPUClass

Fields

`code`	`c_int`	The raw `uform_gpu_class_t` value the ABI exchanges.

Values

`UNKNOWN`	`GPUClass(Int32(0))`	Derives the devices' own class.
`NVIDIA_AMPERE`	`GPUClass(Int32(1))`	Ampere-class NVIDIA and every GPU without a class of its own.
`NVIDIA_ADA`	`GPUClass(Int32(2))`	Ada-class NVIDIA: the fp8 conversions.
`NVIDIA_HOPPER`	`GPUClass(Int32(3))`	Hopper-class NVIDIA.
`NVIDIA_BLACKWELL`	`GPUClass(Int32(4))`	Blackwell-class NVIDIA datacenter parts.
`NVIDIA_BLACKWELLRTX`	`GPUClass(Int32(5))`	Consumer Blackwell NVIDIA.
`NVIDIA_RUBIN`	`GPUClass(Int32(6))`	Rubin-class NVIDIA.
`AMD_CDNA3`	`GPUClass(Int32(7))`	CDNA3 AMD: gfx942, FNUZ fp8.
`AMD_CDNA4`	`GPUClass(Int32(8))`	CDNA4 AMD: gfx950, OCP fp8.
`AMD_CDNA5`	`GPUClass(Int32(9))`	CDNA5 AMD: gfx1250.
`APPLE_METAL7`	`GPUClass(Int32(10))`	Apple7-family GPUs and every Apple GPU without a class of its own.
`APPLE_METAL9`	`GPUClass(Int32(11))`	Apple9-family GPUs.
`APPLE_METAL10`	`GPUClass(Int32(12))`	Apple10-family GPUs.

Conforms to

Copyable, Deinitable, Equatable, ImplicitlyCopyable, Movable, RegisterPassable, TrivialRegisterPassable

StructGPUVendorThe GPU vendors one binary drives at once, uform_gpu_vendor_t.

struct GPUVendor

Fields

`code`	`c_int`	The raw `uform_gpu_vendor_t` value the ABI exchanges.

Values

`UNKNOWN`	`GPUVendor(Int32(0))`	No vendor named.
`NVIDIA`	`GPUVendor(Int32(1))`	NVIDIA, through the CUDA driver.
`AMD`	`GPUVendor(Int32(2))`	AMD, through the HIP runtime.
`APPLE`	`GPUVendor(Int32(3))`	Apple, through Metal.

Conforms to

Copyable, Deinitable, Equatable, ImplicitlyCopyable, Movable, RegisterPassable, TrivialRegisterPassable

StructGPUDeviceOne GPU as UForm numbers it, uform_gpu_device_t: its position in Library.gpu_devices is the ordinal every VRAM placement names it by.2

struct GPUDevice

Fields

`vendor`	`GPUVendor`	The vendor whose runtime drives the device.
`vendor_ordinal`	`UInt32`	The device's ordinal in its vendor's runtime.
`gpu_class`	`GPUClass`	The class whose kernels run on it.
`compute_units`	`UInt32`	Streaming multiprocessors or compute units.
`memory_bytes`	`UInt64`	Device memory in bytes.

Conforms to

Copyable, Defaultable, Deinitable, Movable

__init__An empty entry for the listing to fill.

def __init__(out self)

nameThe vendor's own name for the device.

def name(self) -> String

StructRamOptionsThe host memory domains a RAM-resident model and executor place their bytes in, one NUMA node each.3

struct RamOptions[origin: ImmOrigin = ImmUntrackedOrigin]

Parameters

`originImmOrigin`	comptime	The borrowed node ID array, kept alive for as long as the options are used.

Fields

`memory_domains`	`Optional[Pointer[Int32, origin]]`	NUMA node IDs; absent derives every node the process may allocate on.
`memory_domains_count`	`Int`	Entries in `memory_domains`; zero derives.
`cpu_class`	`CPUClass`	The class to dispatch as; `UNKNOWN` derives the CPU's own.

Conforms to

Copyable, Deinitable, ImplicitlyCopyable, Movable

__init__Derives every memory domain the process may allocate on.

def __init__(out self, *, cpu_class: CPUClass = CPUClass.UNKNOWN)

onNames the NUMA nodes explicitly, borrowing the caller's array for the call.

def on[domains_origin: ImmOrigin, //](
    memory_domains: Span[Int32, domains_origin],
    cpu_class: CPUClass = CPUClass.UNKNOWN
) -> RamOptions[domains_origin]

*× 2The C layout, erased only inside a verb so the borrow lives through the call.

def arguments(self) -> RamOptionsArguments

StructVramOptionsThe devices a VRAM-resident model, trainer and executor place their bytes in.2

struct VramOptions[origin: ImmOrigin = ImmUntrackedOrigin]

Parameters

`originImmOrigin`	comptime	The borrowed ordinal array, kept alive for as long as the options are used.

Fields

`devices`	`Optional[Pointer[Int32, origin]]`	UForm device ordinals, positions in `Library.gpu_devices`; absent derives one free device.
`devices_count`	`Int`	Entries in `devices`; zero derives.
`gpu_class`	`GPUClass`	The class every named device dispatches as; `UNKNOWN` derives the devices' own.

Conforms to

Copyable, Deinitable, ImplicitlyCopyable, Movable

__init__Derives one free device.

def __init__(out self, *, gpu_class: GPUClass = GPUClass.UNKNOWN)

onNames the device ordinals explicitly, borrowing the caller's array for the call.

def on[devices_origin: ImmOrigin, //](
    devices: Span[Int32, devices_origin],
    gpu_class: GPUClass = GPUClass.UNKNOWN
) -> VramOptions[devices_origin]

StructExecutorThe threads or streams a verb runs on, freed once it and every device arena it made go out of scope.5

struct Executor

Executor(library) owns nothing: every verb handed it runs on the calling thread. The spawning constructors fix a kind that must match the residency of every model they drive. Calls on one executor start in submission order, so threads that genuinely overlap take one apiece.

Fields

`library`	`Library`	The loaded engine, held so its symbol table outlives every call made through the handle.
`address`	`Address`	The opaque `uform_executor_t` every verb hands back to the engine; `NULL_ADDRESS` runs inline.

Conforms to

Deinitable, Movable

__init__The caller-thread executor, which every verb accepts whatever the model's residency.

def __init__(out self, library: Library)

def __init__(out self, *, library: Library, address: Int)

Adopts a handle that is already spawned, which is how the factories hand one back.

Parameters

`libraryLibrary`		The loaded engine, kept so the shared object stays mapped.

for_ramSpawns host threads for RAM-resident models. Calls uform_executor_for_ram.

def for_ram[ram_origin: ImmOrigin, //](
    library: Library,
    ram: RamOptions[ram_origin] = RamOptions(cpu_class=CPUClass.UNKNOWN),
    threads: Int = Int(0),
    deferred_limit: Int = Int(0)
) -> Self

Parameters

`libraryLibrary`		The loaded engine, kept so the shared object stays mapped.
`ramRamOptions[ram_origin]`		The memory domains of the models it will drive; copied, not borrowed.
`threadsInt`		Threads to spawn; zero takes 7/8 of the cores those domains carry.
`deferred_limitInt`		Deferred calls it tracks at once; zero takes a default.

Raises

Error if the domains cannot be resolved or the OS refuses the threads.

on_poolSchedules onto a borrowed ForkUnion pool, so UForm, USearch and UStore in one process share one set of threads. Calls uform_executor_on_pool.

def on_pool[ram_origin: ImmOrigin, //](
    library: Library,
    pool: Int,
    ram: RamOptions[ram_origin] = RamOptions(cpu_class=CPUClass.UNKNOWN),
    deferred_limit: Int = Int(0)
) -> Self

Parameters

`libraryLibrary`		The loaded engine, kept so the shared object stays mapped.
`poolInt`		A live ForkUnion `fu_pool_t`, which must outlive the executor.
`ramRamOptions[ram_origin]`		The memory domains of the models it will drive; copied, not borrowed.
`deferred_limitInt`		Deferred calls it tracks at once; zero takes a default.

Raises

Error if the pool or the domains are refused.

for_vramOpens one submission stream on every named device, all driven by the calling thread, and host threads for the decoding the devices cannot do. Calls uform_executor_for_vram.

def for_vram[vram_origin: ImmOrigin, ram_origin: ImmOrigin, //](
    library: Library,
    vram: VramOptions[vram_origin] = VramOptions(gpu_class=GPUClass.UNKNOWN),
    ram: RamOptions[ram_origin] = RamOptions(cpu_class=CPUClass.UNKNOWN),
    streams: Int = Int(0),
    scratch_limit: Int = Int(0),
    deferred_limit: Int = Int(0)
) -> Self

Parameters

`libraryLibrary`		The loaded engine, kept so the shared object stays mapped.
`vramVramOptions[vram_origin]`		The devices of the models it will drive; copied, not borrowed.
`ramRamOptions[ram_origin]`		The memory domains and CPU class of its host decoders; copied, not borrowed.
`streamsInt`		Borrowed CUDA streams, one per device; any but `NULL_ADDRESS` reports `UNSUPPORTED` until borrowing is wired.
`scratch_limitInt`		Ceiling on scratch bytes per device; zero lets scratch grow as batches ask.
`deferred_limitInt`		Deferred calls it tracks at once; zero takes a default.

Raises

Error if a device cannot be selected, or with UNSUPPORTED in a build without one.

drainWaits for every call submitted before the drain, then adopts mood whenever it next idles. Calls uform_executor_drain.

def drain(self, mood: Mood = Mood.PARKED)

Raises

Error for a failure of the executor itself, such as a device fault, never one of a drained call, which reports through its own outcome.