uform.Executor.for_ram(threads=0, *, memory_domains=None, cpu_class=None) creates an executor of host threads, and uform.Executor.for_vram(devices=None, *, scratch_limit=0, gpu_class=None) opens one stream per device.
A device is named by its position in uform.gpu_devices(), which lists every GPU the process can drive as a uform.GPUDevice, NVIDIA devices first, then AMD.
A uform.CPUClass or uform.GPUClass pin dispatches as that class, and None derives the hardware’s own. uform.Executor.on_pool(pool) mirrors the C verb that borrows a ForkUnion pool, and raises UnsupportedError until borrowing is designed.
An executor runs one call at a time, and a call finding another one holding it raises ContendedError once waited for. A deferred call on the executor’s own threads finishes while Python only awaits it, and its progress callbacks run on those threads, holding the GIL while they do. executor.drain(uform.Mood.PARKED) waits until no call holds the executor and its devices are idle, then applies the mood.
In a with block, it waits for its calls and closes on exit.
CPUClassThe CPU families dispatch picks from, valued as uform_cpu_class_t; a RAM placement may pin one.class CPUClass(IntEnum)SERIAL | 1 | |
|---|---|---|
INTEL_HASWELL | 2 | |
INTEL_SIERRA | 3 | |
INTEL_SKYLAKE | 4 | |
INTEL_ICELAKE | 5 | |
AMD_GENOA | 6 | |
INTEL_SAPPHIRE | 7 | |
INTEL_GRANITE | 8 | |
INTEL_DIAMOND | 9 | |
AWS_G2 | 10 | |
APPLE_M2 | 11 | |
APPLE_M4 | 12 | |
AWS_G5 | 13 | |
NVIDIA_VERA | 14 | |
ALIBABA_C930 | 15 | |
WASM32_V128 | 16 | |
WASM64_EXTENDED | 17 | |
MONOVERSIONED | 18 |
GPUClassThe GPU families dispatch picks from, valued as uform_gpu_class_t; a VRAM placement may pin one.class GPUClass(IntEnum)NVIDIA_AMPERE | 1 | |
|---|---|---|
NVIDIA_ADA | 2 | |
NVIDIA_HOPPER | 3 | |
NVIDIA_BLACKWELL | 4 | |
NVIDIA_BLACKWELLRTX | 5 | |
NVIDIA_RUBIN | 6 | |
AMD_CDNA3 | 7 | |
AMD_CDNA4 | 8 | |
AMD_CDNA5 | 9 | |
APPLE_METAL7 | 10 | |
APPLE_METAL9 | 11 | |
APPLE_METAL10 | 12 |
GPUVendorThe vendor whose runtime drives a GPU, valued as uform_gpu_vendor_t.MultimemTierHow a mesh's NVLink domains reduce, valued as uform_multimem_tier_t.GPUDeviceOne GPU this process can drive, its position in gpu_devices() being its ordinal.6class GPUDevice(structseq[Any], tuple[GPUVendor, int, GPUClass | None, int, int, str])vendorThe vendor whose runtime drives the device.vendor: GPUVendorvendor_ordinalThe device's ordinal in its vendor's runtime, as nvidia-smi or rocm-smi list it.vendor_ordinal: intgpu_classThe class whose kernels run on it, or None.gpu_class: GPUClass | Nonecompute_unitsStreaming multiprocessors or compute units.compute_units: intmemory_bytesDevice memory in bytes.memory_bytes: intnameThe vendor's own name for the device.name: strExecutorThe threads or streams a verb runs on, shared across models and named by on and deferred.5class Executorfor_ramSpawns host threads for RAM-resident models.@classmethod
def for_ram(
threads: int = 0,
*,
memory_domains: Sequence[int] | None = None,
deferred_limit: int = 0,
cpu_class: CPUClass | None = None,
) -> ExecutorCalls uform_executor_for_ram.
on_poolBorrows a ForkUnion pool; raises UnsupportedError until borrowing is designed.@classmethod
def on_pool(
pool: int,
/,
*,
memory_domains: Sequence[int] | None = None,
deferred_limit: int = 0,
cpu_class: CPUClass | None = None,
) -> ExecutorCalls uform_executor_on_pool.
for_vramOpens one submission stream on every named device.@classmethod
def for_vram(
devices: Sequence[int] | None = None,
*,
scratch_limit: int = 0,
deferred_limit: int = 0,
gpu_class: GPUClass | None = None,
memory_domains: Sequence[int] | None = None,
cpu_class: CPUClass | None = None,
) -> ExecutorCalls uform_executor_for_vram.
drainWaits for every call submitted before it, then sets the idle mood.__exit__Closes it once its calls finish, or leaves that to its last live GPU arena.def __exit__(
kind: type[BaseException] | None,
value: BaseException | None,
traceback: TracebackType | None,
/,
) -> NoneCalls uform_executor_free.
MeshOptionsWhere this process sits among the processes training one model, and how they meet.7class MeshOptionsMeshOptionsMeshOptions(
*,
host_domain_index: int = 0,
host_domain_count: int = 0,
gpus_per_multimem_domain: int = 0,
directory: _PathLike | None = None,
job: str | None = None,
timeout_seconds: float = 0.0,
)host_domain_indexhost_domain_index: Final[int]host_domain_counthost_domain_count: Final[int]gpus_per_multimem_domaingpus_per_multimem_domain: Final[int]directorydirectory: Final[_PathLike | None]jobjob: Final[str | None]timeout_secondstimeout_seconds: Final[float]MeshDescriptionWhat a mesh reports about itself, as uform_mesh_description_t holds it.7class MeshDescription(structseq[Any], tuple[int, int, int, int, int, MultimemTier, GPUClass | None])gpus_per_host_domainGPUs one process drives.gpus_per_host_domain: intgpus_per_multimem_domainGPUs one NVLink domain spans.gpus_per_multimem_domain: intgpus_per_colocated_domainGPUs in the whole run.gpus_per_colocated_domain: inthost_domain_indexThis process's index among the run's processes.host_domain_index: intfirst_gpu_indexThe run-wide index of this process's first GPU.first_gpu_index: intmultimem_tierHow its NVLink domains reduce.multimem_tier: MultimemTiergpu_classThe class every GPU of the run dispatches as, or None.gpu_class: GPUClass | NoneMeshOne process's place among the processes of a run, borrowed by every trainer.2class MeshbootstrapJoins this process to the run's mesh.@classmethod
def bootstrap(
executor: Executor,
options: MeshOptions | None,
/,
) -> MeshCalls uform_mesh_bootstrap.
describeThe mesh's shape and this process's place in it.capabilities_compiledThe hardware capabilities this binary was compiled with.capabilities_availableThe hardware capabilities this machine offers at runtime.gpu_devicesEvery GPU this process can drive, a device's position being its ordinal.