struct CPUClassConforms to
Hashable, Sendable, CustomStringConvertible, EquatableA model says where its weights live; an Executor says who runs the calls on them.
One executor serves any number of models of its residency, so several models share one machine’s cores or devices.
let hostThreads = try Executor.forRam(memoryDomains: [0, 1], threads: 0)
let deviceStreams = try Executor.forVram(devices: [0, 1], streams: nil, scratchLimit: 1 << 30)
try hostThreads.drain(mood: .parked)Each factory takes every argument its C constructor has, and only those.
A thread count of zero takes 7/8 of the cores the memory domains carry, and forVram takes no thread count at all, since the submitting thread drives one stream per device. Executor.onPool(memoryDomains:cpuClass:pool:) mirrors the C verb that borrows a ForkUnion pool, and reports unsupported until the library wires borrowed pools. drain(mood:) waits until no call holds the executor and its devices are idle, then picks whether its threads park or spin.
API
StructCPUClassA CPU family UForm ships tuned kernels for, carrying the C enumerator; dispatch reports its pick in this vocabulary, and a RAM placement may pin one.20
serialThe portable floor, with no SIMD assumptions.
static let serial: CPUClassintelHaswellHaswell-class x86: AVX2 and FMA.
static let intelHaswell: CPUClassintelSierraSierra Forest-class x86: AVX-VNNI without AVX-512.
static let intelSierra: CPUClassintelSkylakeSkylake-class x86: the AVX-512 foundation.
static let intelSkylake: CPUClassintelIcelakeIcelake-class x86: AVX-512 VNNI and VPOPCNT.
static let intelIcelake: CPUClassamdGenoaGenoa-class x86: native bf16 dot products over AVX-512.
static let amdGenoa: CPUClassintelSapphireSapphire Rapids-class x86: AVX-512 FP16 and the AMX tiles.
static let intelSapphire: CPUClassintelGraniteGranite Rapids-class x86: the AMX-FP16 tiles.
static let intelGranite: CPUClassintelDiamondDiamond Rapids-class x86: AVX10.2's FP8 conversions.
static let intelDiamond: CPUClassawsG2Graviton2-class Arm: the bare NEON floor.
static let awsG2: CPUClassappleM2Armv8.6 NEON with dot products and bf16.
static let appleM2: CPUClassappleM4Apple M4-class Arm: the SME outer-product tiles.
static let appleM4: CPUClassawsG5Graviton5-class Arm: Armv9.2.
static let awsG5: CPUClassnvidiaVeraVera-class Arm: FP8 dot products.
static let nvidiaVera: CPUClassalibabaC930Alibaba C930-class RISC-V: RVA23 vectors.
static let alibabaC930: CPUClasswasm32V128WebAssembly with fixed-width simd128.
static let wasm32V128: CPUClasswasm64ExtendedWebAssembly with relaxed SIMD, threads and 64-bit addressing.
static let wasm64Extended: CPUClassmonoversionedThe one class of a monoversioned build, with every capability it compiled.
static let monoversioned: CPUClassdescriptionThe class as the library spells it, like intel-haswell.
initReads a spelling back, and nil for one no class carries.
StructGPUClassA GPU family UForm ships tuned kernels for, carrying the C enumerator; dispatch reports its pick in this vocabulary, and a VRAM placement may pin one.14
struct GPUClassConforms to
Hashable, Sendable, CustomStringConvertible, EquatablenvidiaAmpereAmpere-class NVIDIA, and every NVIDIA GPU without a class of its own.
static let nvidiaAmpere: GPUClassnvidiaAdaAda-class NVIDIA: the fp8 conversions.
static let nvidiaAda: GPUClassnvidiaHopperHopper-class NVIDIA.
static let nvidiaHopper: GPUClassnvidiaBlackwellBlackwell-class NVIDIA.
static let nvidiaBlackwell: GPUClassnvidiaBlackwellRTXConsumer Blackwell NVIDIA.
static let nvidiaBlackwellRTX: GPUClassnvidiaRubinRubin-class NVIDIA.
static let nvidiaRubin: GPUClassamdCDNA3CDNA3 AMD: gfx942.
static let amdCDNA3: GPUClassamdCDNA4CDNA4 AMD: gfx950.
static let amdCDNA4: GPUClassamdCDNA5CDNA5 AMD: gfx1250.
static let amdCDNA5: GPUClassappleMetal7Apple7-family GPUs, and every Apple GPU without a class of its own.
static let appleMetal7: GPUClassappleMetal9Apple9-family GPUs.
static let appleMetal9: GPUClassappleMetal10Apple10-family GPUs.
static let appleMetal10: GPUClassdescriptionThe class as the library spells it, like nvidia-hopper.
initReads a spelling back, and nil for one no class carries.
StructGPUVendorThe vendor whose runtime drives a GPU, carrying the C enumerator.4
unknownA vendor this binding has no name for.
static let unknown: GPUVendornvidiaNVIDIA, through the CUDA driver.
static let nvidia: GPUVendoramdAMD, through the HIP runtime.
static let amd: GPUVendorappleApple, through Metal.
static let apple: GPUVendorStructGPUDeviceOne GPU this process can drive; its position in gpuDevices() is the ordinal every VRAM placement names it by.6
vendorThe vendor whose runtime drives the device.
var vendor: GPUVendorvendorOrdinalThe device's ordinal in its vendor's runtime.
var vendorOrdinal: UInt32gpuClassThe class whose kernels run on it, or nil for none.
var gpuClass: GPUClass?computeUnitsStreaming multiprocessors or compute units.
var computeUnits: UInt32memoryBytesDevice memory in bytes.
var memoryBytes: UInt64nameThe vendor's own name for the device.
var name: StringClassExecutorThe host threads or device streams a call runs on, with the activation scratch and the staging they use.4
final class ExecutorOne executor serves any number of models of its residency, so several models share one machine's
cores or devices; a mismatch reports device. Calls on one executor run in submission order.
Conforms to
SendableforRamSpawns host threads for RAM-resident models, interleaved across the compute domains local to memoryDomains; an empty list derives every node the process may allocate on.
static func forRam(
memoryDomains: [Int32] = [],
cpuClass: CPUClass? = nil,
threads: Int = 0,
deferredLimit: Int = 0
) throws(UFormError) -> ExecutorZero
threads takes 7/8 of the cores those domains carry, and zero deferredLimit sizes the
deferred table by default. A cpuClass dispatches as if the CPU were of that class; nil
derives its own. Calls uform_executor_for_ram.
onPoolCreates a RAM executor over a borrowed ForkUnion fu_pool_t, which must outlive it.
static func onPool(
memoryDomains: [Int32] = [],
cpuClass: CPUClass? = nil,
pool: UnsafeMutableRawPointer,
deferredLimit: Int = 0
) throws(UFormError) -> ExecutorReports unsupported until the library wires borrowed pools. A cpuClass pins as forRam
does. Calls uform_executor_on_pool.
forVramOpens one submission stream on every named device, or borrows the caller's streams, one per device in devices order; an empty list derives one free device.
static func forVram(
devices: [Int32] = [],
gpuClass: GPUClass? = nil,
memoryDomains: [Int32] = [],
cpuClass: CPUClass? = nil,
streams: [UnsafeMutableRawPointer?]? = nil,
scratchLimit: Int = 0,
deferredLimit: Int = 0
) throws(UFormError) -> ExecutordrainWaits for every call submitted before it, then adopts mood whenever the executor next goes idle; a failure is the executor's own, never one of a drained call.
StructMultimemTierHow a mesh's NVLink domains reduce, carrying the C enumerator.4
struct MultimemTierConforms to
Hashable, Sendable, CustomStringConvertible, EquatableswitchReducedThe NVLink switch reduces through multicast memory.
static let switchReduced: MultimemTierpeerReadEvery member reads its peers in a fixed order, where multicast is absent.
static let peerRead: MultimemTierdescriptionThe tier as the library spells it.
initReads a spelling back, and nil for one no tier carries.
StructMeshOptionsWhere this process sits among the processes training one model, and how they meet.7
struct MeshOptionsMirrors
uform_mesh_options_t, every zero deriving as the C field documents.
Conforms to
SendablehostDomainIndexThis process's index among hostDomainCount.
var hostDomainIndex: InthostDomainCountProcesses in the run, one per host; zero derives one, a run with no peers.
var hostDomainCount: IntgpusPerMultimemDomainGPUs one NVLink multicast domain spans; zero derives the host.
var gpusPerMultimemDomain: IntdirectoryThe directory every host sees, where processes exchange their facts; required with peers.
var directory: StringjobThe run's unique name inside directory; required with peers.
var job: StringtimeoutSecondsSeconds any wait for a peer lasts before the call fails; zero derives 120.
var timeoutSeconds: DoubleinitTakes every field by name; zeros and empty strings derive as their fields document.
init(hostDomainIndex: Int = 0, hostDomainCount: Int = 0, gpusPerMultimemDomain: Int = 0, directory: String = "", job: String = "", timeoutSeconds: Double = 0)StructMeshDescriptionWhat a mesh reports about itself.7
gpusPerHostDomainGPUs one process drives.
var gpusPerHostDomain: IntgpusPerMultimemDomainGPUs one NVLink domain spans.
var gpusPerMultimemDomain: IntgpusPerColocatedDomainGPUs in the whole run.
var gpusPerColocatedDomain: InthostDomainIndexThis process's index among the run's processes.
var hostDomainIndex: IntfirstGpuIndexThe run-wide index of this process's first GPU, where its share of a sharded dataset begins.
var firstGpuIndex: IntmultimemTierHow its NVLink domains reduce.
var multimemTier: MultimemTiergpuClassThe class every GPU of the run dispatches as: the executor's pin, or the devices' own.
var gpuClass: GPUClass?ClassMeshOne process's place among the processes of a run, made once and borrowed by every trainer.2
bootstrapJoins this process to the run's mesh over the devices of executor; every process of the run calls it with the same count, multicast extent, directory and job.
static func bootstrap(
executor: Executor,
options: MeshOptions = MeshOptions()
) throws(UFormError) -> MeshCalls uform_mesh_bootstrap.