# Executors

> UForm for Mojo

One executor serves any number of models of its residency.
`Executor(library)` owns nothing and runs every call handed it on the calling thread.
There is a constructor per C one, and each fixes a kind that must match the residency of every model it drives:

```mojo
var threads = Executor.for_ram(library, RamOptions.on(domains), threads=8)  # zero takes 7/8 of the cores
var shared = Executor.on_pool(library, fork_union_pool)                       # borrows a fu_pool_t
var streams = Executor.for_vram(library, VramOptions.on(devices))             # one host thread, a stream per device
threads.drain(Mood.SPINNING)                                                # waits until no call holds it, then keeps spinning
```

An executor runs one call at a time, so threads that genuinely overlap take one executor apiece.
A call finding another one holding its executor finishes at once with `Status.CONTENDED`.

`from uform import MultimemTier, MeshDescription, MeshOptions, …`

## MultimemTier

How a mesh's NVLink domains reduce, `uform_multimem_tier_t`.

```mojo
struct MultimemTier
```

- `SWITCH_REDUCED` = `MultimemTier(Int32(0))`: The NVLink switch reduces through multicast memory.
- `PEER_READ` = `MultimemTier(Int32(1))`: Every member reads its peers in a fixed order, where multicast is absent.
- `code` `c_int`: The raw `uform_multimem_tier_t` value the ABI exchanges.

Conforms to: `Copyable`, `Deinitable`, `Equatable`, `ImplicitlyCopyable`, `Movable`, `RegisterPassable`, `TrivialRegisterPassable`

## MeshDescription

What a mesh reports about itself, `uform_mesh_description_t`.

```mojo
struct MeshDescription
```

- `gpus_per_host_domain` `c_size_t`: GPUs one process drives.
- `gpus_per_multimem_domain` `c_size_t`: GPUs one NVLink domain spans.
- `gpus_per_colocated_domain` `c_size_t`: GPUs in the whole run.
- `host_domain_index` `c_size_t`: This process's index among the run's processes.
- `first_gpu_index` `c_size_t`: The run-wide index of this process's first GPU, where its share of a sharded dataset begins.
- `multimem_tier` `MultimemTier`: How its NVLink domains reduce.
- `gpu_class` `GPUClass`: The class every GPU of the run dispatches as.

Conforms to: `Copyable`, `Defaultable`, `Deinitable`, `ImplicitlyCopyable`, `Movable`

## __init__

Zeroes every field, which is what a failed report leaves.

```mojo
def __init__(out self)
```

## MeshOptions

Where this process sits among the processes training one model, and how they meet, `uform_mesh_options_t`.

```mojo
struct MeshOptions
```

- `host_domain_index` `Int`: This process's index among `host_domain_count`.
- `host_domain_count` `Int`: Processes in the run, one per host; zero derives one, a run with no peers.
- `gpus_per_multimem_domain` `Int`: GPUs one NVLink multicast domain spans; zero derives the host.
- `directory` `String`: The directory every host sees, where processes exchange their facts; required with peers.
- `job` `String`: The run's unique name inside `directory`; required with peers.
- `timeout_seconds` `Float64`: Seconds any wait for a peer lasts before the call fails; zero derives 120.

Conforms to: `Copyable`, `Defaultable`, `Deinitable`, `Movable`

## __init__

A run with no peers.

```mojo
def __init__(out self)
```

## Mesh

One process's place among the processes of a run, borrowed by every trainer over the same executor; released when it goes out of scope, after every trainer borrowing it.

```mojo
struct Mesh
```

- `library` `Library`: The loaded engine, held so the release can still reach `uform_mesh_free`.
- `address` `Address`: The opaque `uform_mesh_t`.

Conforms to: `Deinitable`, `Movable`

## __init__

Adopts a handle the engine just created.

```mojo
def __init__(out self, *, library: Library, address: Int)
```

## __deinit__

Leaves the mesh. Calls `uform_mesh_free`.

```mojo
def __deinit__(deinit self)
```

## bootstrap

Joins this process to the run's mesh, waiting for every peer. Calls `uform_mesh_bootstrap`.

```mojo
def bootstrap(executor: Executor, options: MeshOptions = MeshOptions()) -> Self
```

- `executor`: A device executor whose devices are this process's members of the mesh.
- `options`: Where this process sits and how the processes meet.

Raises: [`Error`](https://www.unum.cloud/docs/uform/mojo/errors.md#Error) with [`CONFIG_MISMATCH`](https://www.unum.cloud/docs/uform/mojo/errors.md#CONFIG_MISMATCH) when peers disagree, or [`UNSUPPORTED`](https://www.unum.cloud/docs/uform/mojo/errors.md#UNSUPPORTED) in a build without the trainer.

## describe

Reports the mesh's shape and this process's place in it. Calls `uform_mesh_describe`.

```mojo
def describe(self) -> MeshDescription
```

## CPUClass

The CPU family dispatch picks, or a placement pins, `uform_cpu_class_t`.

```mojo
struct CPUClass
```

- `UNKNOWN` = `CPUClass(Int32(0))`: Derives the CPU's own class.
- `SERIAL` = `CPUClass(Int32(1))`: The portable floor: no SIMD assumptions.
- `INTEL_HASWELL` = `CPUClass(Int32(2))`: Haswell-class x86: AVX2 and FMA.
- `INTEL_SIERRA` = `CPUClass(Int32(3))`: Sierra Forest-class x86: AVX-VNNI without AVX-512.
- `INTEL_SKYLAKE` = `CPUClass(Int32(4))`: Skylake-class x86: the AVX-512 foundation tier.
- `INTEL_ICELAKE` = `CPUClass(Int32(5))`: Icelake-class x86: AVX-512 VNNI and VPOPCNT.
- `AMD_GENOA` = `CPUClass(Int32(6))`: Genoa-class x86: native bf16 dot products over AVX-512.
- `INTEL_SAPPHIRE` = `CPUClass(Int32(7))`: Sapphire Rapids-class x86: AVX-512 FP16 and the AMX-BF16 and AMX-INT8 tiles.
- `INTEL_GRANITE` = `CPUClass(Int32(8))`: Granite Rapids-class x86: the Sapphire Rapids bundle plus the AMX-FP16 tiles.
- `INTEL_DIAMOND` = `CPUClass(Int32(9))`: Diamond Rapids-class x86: the Granite Rapids bundle plus AVX10.2's FP8 conversions.
- `AWS_G2` = `CPUClass(Int32(10))`: Graviton2-class Arm, the bare NEON floor.
- `APPLE_M2` = `CPUClass(Int32(11))`: Armv8.6+ NEON: dot products, bf16 and f16 fused multiplies.
- `APPLE_M4` = `CPUClass(Int32(12))`: Apple M4-class Arm: the Armv8.6 NEON bundle plus the SME outer-product tiles.
- `AWS_G5` = `CPUClass(Int32(13))`: Graviton5-class Arm, Neoverse V3.
- `NVIDIA_VERA` = `CPUClass(Int32(14))`: Vera-class Arm with FP8 dot products.
- `ALIBABA_C930` = `CPUClass(Int32(15))`: Alibaba C930-class RISC-V with RVA23 vectors.
- `WASM32_V128` = `CPUClass(Int32(16))`: WebAssembly every engine loads: 32-bit, single-threaded, simd128.
- `WASM64_EXTENDED` = `CPUClass(Int32(17))`: WebAssembly with relaxed SIMD, shared memory, and 64-bit addressing.
- `MONOVERSIONED` = `CPUClass(Int32(18))`: The one class of a monoversioned build.
- `code` `c_int`: The raw `uform_cpu_class_t` value the ABI exchanges.

Conforms to: `Copyable`, `Deinitable`, `Equatable`, `ImplicitlyCopyable`, `Movable`, `RegisterPassable`, `TrivialRegisterPassable`

## GPUClass

The GPU family dispatch picks, or a placement pins, `uform_gpu_class_t`.

```mojo
struct GPUClass
```

- `UNKNOWN` = `GPUClass(Int32(0))`: Derives the devices' own class.
- `NVIDIA_AMPERE` = `GPUClass(Int32(1))`: Ampere-class NVIDIA and every GPU without a class of its own.
- `NVIDIA_ADA` = `GPUClass(Int32(2))`: Ada-class NVIDIA: the fp8 conversions.
- `NVIDIA_HOPPER` = `GPUClass(Int32(3))`: Hopper-class NVIDIA.
- `NVIDIA_BLACKWELL` = `GPUClass(Int32(4))`: Blackwell-class NVIDIA datacenter parts.
- `NVIDIA_BLACKWELLRTX` = `GPUClass(Int32(5))`: Consumer Blackwell NVIDIA.
- `NVIDIA_RUBIN` = `GPUClass(Int32(6))`: Rubin-class NVIDIA.
- `AMD_CDNA3` = `GPUClass(Int32(7))`: CDNA3 AMD: gfx942, FNUZ fp8.
- `AMD_CDNA4` = `GPUClass(Int32(8))`: CDNA4 AMD: gfx950, OCP fp8.
- `AMD_CDNA5` = `GPUClass(Int32(9))`: CDNA5 AMD: gfx1250.
- `APPLE_METAL7` = `GPUClass(Int32(10))`: Apple7-family GPUs and every Apple GPU without a class of its own.
- `APPLE_METAL9` = `GPUClass(Int32(11))`: Apple9-family GPUs.
- `APPLE_METAL10` = `GPUClass(Int32(12))`: Apple10-family GPUs.
- `code` `c_int`: The raw `uform_gpu_class_t` value the ABI exchanges.

Conforms to: `Copyable`, `Deinitable`, `Equatable`, `ImplicitlyCopyable`, `Movable`, `RegisterPassable`, `TrivialRegisterPassable`

## GPUVendor

The GPU vendors one binary drives at once, `uform_gpu_vendor_t`.

```mojo
struct GPUVendor
```

- `UNKNOWN` = `GPUVendor(Int32(0))`: No vendor named.
- `NVIDIA` = `GPUVendor(Int32(1))`: NVIDIA, through the CUDA driver.
- `AMD` = `GPUVendor(Int32(2))`: AMD, through the HIP runtime.
- `APPLE` = `GPUVendor(Int32(3))`: Apple, through Metal.
- `code` `c_int`: The raw `uform_gpu_vendor_t` value the ABI exchanges.

Conforms to: `Copyable`, `Deinitable`, `Equatable`, `ImplicitlyCopyable`, `Movable`, `RegisterPassable`, `TrivialRegisterPassable`

## GPUDevice

One GPU as UForm numbers it, `uform_gpu_device_t`: its position in [`Library.gpu_devices`](https://www.unum.cloud/docs/uform/mojo/design.md#gpu_devices) is the ordinal every VRAM placement names it by.

```mojo
struct GPUDevice
```

- `vendor` `GPUVendor`: The vendor whose runtime drives the device.
- `vendor_ordinal` `UInt32`: The device's ordinal in its vendor's runtime.
- `gpu_class` `GPUClass`: The class whose kernels run on it.
- `compute_units` `UInt32`: Streaming multiprocessors or compute units.
- `memory_bytes` `UInt64`: Device memory in bytes.

Conforms to: `Copyable`, `Defaultable`, `Deinitable`, `Movable`

## __init__

An empty entry for the listing to fill.

```mojo
def __init__(out self)
```

## name

The vendor's own name for the device.

```mojo
def name(self) -> String
```

## RamOptions

The host memory domains a RAM-resident model and executor place their bytes in, one NUMA node each.

```mojo
struct RamOptions[origin: ImmOrigin = ImmUntrackedOrigin]
```

- `origin` (comptime): The borrowed node ID array, kept alive for as long as the options are used.

- `memory_domains` `Optional[Pointer[Int32, origin]]`: NUMA node IDs; absent derives every node the process may allocate on.
- `memory_domains_count` `Int`: Entries in `memory_domains`; zero derives.
- `cpu_class` `CPUClass`: The class to dispatch as; [`UNKNOWN`](https://www.unum.cloud/docs/uform/mojo/encoding.md#UNKNOWN) derives the CPU's own.

Conforms to: `Copyable`, `Deinitable`, `ImplicitlyCopyable`, `Movable`

## __init__

Derives every memory domain the process may allocate on.

```mojo
def __init__(out self, *, cpu_class: CPUClass = CPUClass.UNKNOWN)
```

## on

Names the NUMA nodes explicitly, borrowing the caller's array for the call.

```mojo
def on[domains_origin: ImmOrigin, //](
    memory_domains: Span[Int32, domains_origin],
    cpu_class: CPUClass = CPUClass.UNKNOWN
) -> RamOptions[domains_origin]
```

## *

The C layout, erased only inside a verb so the borrow lives through the call.

```mojo
def arguments(self) -> RamOptionsArguments
```

## VramOptions

The devices a VRAM-resident model, trainer and executor place their bytes in.

```mojo
struct VramOptions[origin: ImmOrigin = ImmUntrackedOrigin]
```

- `origin` (comptime): The borrowed ordinal array, kept alive for as long as the options are used.

- `devices` `Optional[Pointer[Int32, origin]]`: UForm device ordinals, positions in [`Library.gpu_devices`](https://www.unum.cloud/docs/uform/mojo/design.md#gpu_devices); absent derives one free device.
- `devices_count` `Int`: Entries in `devices`; zero derives.
- `gpu_class` `GPUClass`: The class every named device dispatches as; [`UNKNOWN`](https://www.unum.cloud/docs/uform/mojo/encoding.md#UNKNOWN) derives the devices' own.

Conforms to: `Copyable`, `Deinitable`, `ImplicitlyCopyable`, `Movable`

## __init__

Derives one free device.

```mojo
def __init__(out self, *, gpu_class: GPUClass = GPUClass.UNKNOWN)
```

## on

Names the device ordinals explicitly, borrowing the caller's array for the call.

```mojo
def on[devices_origin: ImmOrigin, //](
    devices: Span[Int32, devices_origin],
    gpu_class: GPUClass = GPUClass.UNKNOWN
) -> VramOptions[devices_origin]
```

## Executor

The threads or streams a verb runs on, freed once it and every device arena it made go out of scope.

```mojo
struct Executor
```

`Executor(library)` owns nothing: every verb handed it runs on the calling thread. The spawning
constructors fix a kind that must match the residency of every model they drive. Calls on one
executor start in submission order, so threads that genuinely overlap take one apiece.

- `library` `Library`: The loaded engine, held so its symbol table outlives every call made through the handle.
- `address` `Address`: The opaque `uform_executor_t` every verb hands back to the engine; `NULL_ADDRESS` runs inline.

Conforms to: `Deinitable`, `Movable`

## __init__

The caller-thread executor, which every verb accepts whatever the model's residency.

```mojo
def __init__(out self, library: Library)
```

```mojo
def __init__(out self, *, library: Library, address: Int)
```

Adopts a handle that is already spawned, which is how the factories hand one back.

- `library`: The loaded engine, kept so the shared object stays mapped.

## for_ram

Spawns host threads for RAM-resident models. Calls `uform_executor_for_ram`.

```mojo
def for_ram[ram_origin: ImmOrigin, //](
    library: Library,
    ram: RamOptions[ram_origin] = RamOptions(cpu_class=CPUClass.UNKNOWN),
    threads: Int = Int(0),
    deferred_limit: Int = Int(0)
) -> Self
```

- `library`: The loaded engine, kept so the shared object stays mapped.
- `ram`: The memory domains of the models it will drive; copied, not borrowed.
- `threads`: Threads to spawn; zero takes 7/8 of the cores those domains carry.
- `deferred_limit`: Deferred calls it tracks at once; zero takes a default.

Raises: [`Error`](https://www.unum.cloud/docs/uform/mojo/errors.md#Error) if the domains cannot be resolved or the OS refuses the threads.

## on_pool

Schedules onto a borrowed ForkUnion pool, so UForm, USearch and UStore in one process share one set of threads. Calls `uform_executor_on_pool`.

```mojo
def on_pool[ram_origin: ImmOrigin, //](
    library: Library,
    pool: Int,
    ram: RamOptions[ram_origin] = RamOptions(cpu_class=CPUClass.UNKNOWN),
    deferred_limit: Int = Int(0)
) -> Self
```

- `library`: The loaded engine, kept so the shared object stays mapped.
- `pool`: A live ForkUnion `fu_pool_t`, which must outlive the executor.
- `ram`: The memory domains of the models it will drive; copied, not borrowed.
- `deferred_limit`: Deferred calls it tracks at once; zero takes a default.

Raises: [`Error`](https://www.unum.cloud/docs/uform/mojo/errors.md#Error) if the pool or the domains are refused.

## for_vram

Opens one submission stream on every named device, all driven by the calling thread, and host threads for the decoding the devices cannot do. Calls `uform_executor_for_vram`.

```mojo
def for_vram[vram_origin: ImmOrigin, ram_origin: ImmOrigin, //](
    library: Library,
    vram: VramOptions[vram_origin] = VramOptions(gpu_class=GPUClass.UNKNOWN),
    ram: RamOptions[ram_origin] = RamOptions(cpu_class=CPUClass.UNKNOWN),
    streams: Int = Int(0),
    scratch_limit: Int = Int(0),
    deferred_limit: Int = Int(0)
) -> Self
```

- `library`: The loaded engine, kept so the shared object stays mapped.
- `vram`: The devices of the models it will drive; copied, not borrowed.
- `ram`: The memory domains and CPU class of its host decoders; copied, not borrowed.
- `streams`: Borrowed CUDA streams, one per device; any but `NULL_ADDRESS` reports [`UNSUPPORTED`](https://www.unum.cloud/docs/uform/mojo/errors.md#UNSUPPORTED) until borrowing is wired.
- `scratch_limit`: Ceiling on scratch bytes per device; zero lets scratch grow as batches ask.
- `deferred_limit`: Deferred calls it tracks at once; zero takes a default.

Raises: [`Error`](https://www.unum.cloud/docs/uform/mojo/errors.md#Error) if a device cannot be selected, or with [`UNSUPPORTED`](https://www.unum.cloud/docs/uform/mojo/errors.md#UNSUPPORTED) in a build without one.

## drain

Waits for every call submitted before the drain, then adopts `mood` whenever it next idles. Calls `uform_executor_drain`.

```mojo
def drain(self, mood: Mood = Mood.PARKED)
```

Raises: [`Error`](https://www.unum.cloud/docs/uform/mojo/errors.md#Error) for a failure of the executor itself, such as a device fault, never one of a drained call, which reports through its own outcome.