Runtime Dispatch

Capabilities answers for the CPU and each GPU group through its associated functions, and Capabilities::configure_thread enables CPU-specific acceleration features such as Intel AMX. It must be called once per thread before using AMX operations.

use numkong::{Capabilities, Capability};

let enabled = Capabilities::cpu_enabled();
enabled.configure_thread().unwrap();

println!("dispatching to {enabled}"); // like "serial,neon,neonhalf"
if enabled.contains(Capability::SapphireAmx) {
    println!("AMX enabled");
}

for ordinal in 0..Capabilities::cuda_count_devices().unwrap() {
    println!("CUDA device {ordinal} runs {}", Capabilities::cuda_enabled(ordinal).unwrap()); // like "cuda,ampere,hopper"
}

Each group reports a Capabilities set of Capability bits along two independent axes, plus the set dispatch uses:

• cpu_detected(), cuda_detected(ordinal) and their ROCm and Metal kin: what the device can execute, from CPUID or HWCAP on the CPU and from the runtime on a GPU
• cpu_compiled(), cuda_compiled() and their kin: what this binary contains, from the probes at build time
• cpu_enabled(), cuda_enabled(ordinal) and their kin: what dispatch runs within, i.e. both axes at once

Reach for the _enabled ones unless you specifically mean one of the raw axes. cpu_detected() describes the machine and says nothing about whether a kernel was compiled in, so a build whose ISA probes failed still reports your CPU’s full feature set while containing no SIMD kernels at all. The crate keeps no process state: every CPU kernel call passes Capabilities::CPUS, and the library clamps it to the CPU capabilities it settles as it loads. No verb allocates: a GPU verb takes any memory its device reaches, which the runtime checks per call. Stream::new(capabilities, ordinal) makes a stream on a device, Stream::default(capabilities) names the vendor’s default, and stream.synchronize() waits for it. UnifiedAllocator::new(&stream) allocates memory the host and that stream’s device both address, the only unified memory the crate hands out. stream.scope(|scope| ...) runs the safe _on verbs, each queueing on that stream with its capabilities, and joins it before anything they borrow is released. Capabilities::stream_init_cuda(ordinal) and Capabilities::stream_free_cuda(stream) stay as the unsafe raw handles, with rocm_ and metal_ twins.

Call configure_thread at the start of every thread that will use AMX operations. In a thread-pool setting, each worker thread needs its own call. The function is idempotent and cheap to call more than once on the same thread.