# Runtime Dispatch > NumKong for C++ ### Capabilities and Devices Linking the library is the default, and the way to ship one binary across many CPU generations. Every query writes its answer through a pointer and returns an `nk_status_t`. CPU capabilities are reported along two independent axes, plus the mask to dispatch with: - `nk_cpu_capabilities_detected` is what this CPU can execute, from CPUID or the OS, or where the OS cannot be asked, from what the compiler flags guarantee. - `nk_cpu_capabilities_compiled` is what this binary contains, from the ISA probes at build time. - `nk_cpu_capabilities_enabled` is both axes at once, the mask to pass every CPU call, and always holds `nk_cap_serial_k`. Ask for `enabled` unless you specifically mean one of the raw axes. The two axes are independent, and conflating them fails quietly rather than loudly: a binary whose ISA probes failed still reports this machine's full `detected` mask while containing no SIMD kernels at all. To dispatch over fewer capabilities, pass a narrower mask to the call itself; there is no process-wide setting to change. `nk_cpu_configure_thread` prepares the calling thread for the capabilities it is given, and only those. Most capabilities need nothing: AMX on Linux costs one `arch_prctl` syscall, which grants tile state to the whole process, and fused BF16 dot products on Arm cost one `FPCR` write per thread. ```c nk_capability_t enabled = nk_cap_serial_k; nk_cpu_capabilities_enabled(&enabled); nk_cpu_configure_thread(enabled); if (enabled & nk_cap_sapphireamx_k) { /* AMX both detected and compiled in */ } nk_dot_bf16_best(a, b, n, &dot, enabled & ~nk_cap_sapphireamx_k, NULL); // dispatch without AMX ``` GPUs are more ISAs with a mask of their own, one device at a time. Each vendor has its own queries, and a device index is that runtime's own ordinal: the one `cudaSetDevice` or `hipSetDevice` takes, or the position in Metal's device list. `nk_cuda_count_devices` counts the devices, and `nk_cuda_capabilities_detected`, `nk_cuda_capabilities_compiled`, and `nk_cuda_capabilities_enabled` report one device's capabilities the way their CPU twins do; `nk_rocm_*` and `nk_metal_*` are the same for the other vendors. Each vendor's baseline, `nk_cap_cuda_k`, `nk_cap_rocm_k`, or `nk_cap_metal_k`, plays the role of `nk_cap_serial_k`. A GPU capability, like `nk_dots_packed_bf16_ampere`, takes its CPU twin's arguments, queues on the trailing stream — a `cudaStream_t`, a `hipStream_t`, or an `id`, with null for the default one — and returns without waiting. Only the capability queries take an ordinal; the stream names the device for everything else. `nk_memory_allocate_unified_best` and `nk_memory_free_unified_best` hand out and return memory the host and that device both address, and `nk_stream_synchronize_best` waits for the stream. Batched kernels that may need scratch, like the packed and symmetric families and attention, take a `workspace, workspace_bytes` pair before the capabilities and never allocate. Their `*_workspace_size_*` twin reports the bytes from the shapes alone, and a zero-byte workspace runs the scratch-free path where the capability has one. ```c nk_size_t devices = 0, workspace_bytes = 0; nk_capability_t gpu = 0; if (nk_cuda_count_devices(&devices) == nk_success_k && nk_cuda_capabilities_enabled(0, &gpu) == nk_success_k) { nk_dots_packed_workspace_size_f64_best(rows, columns, depth, gpu, &workspace_bytes); void *workspace = NULL; nk_memory_allocate_unified_best(workspace_bytes, &workspace, gpu, cuda_stream); nk_dots_packed_f64_best(a, b_packed, c, rows, columns, depth, a_stride, c_stride, workspace, workspace_bytes, gpu, cuda_stream); nk_stream_synchronize_best(gpu, cuda_stream); } ``` `nk_capabilities_name` spells any such mask as the names bindings accept, like "serial,neon,neonhalf", into a buffer of `NUMKONG_CAPABILITIES_NAME_CAPACITY` bytes. For exact register-level details, see `capabilities.h`. Capability kernels, like `nk_dot_f32_haswell`, stay callable directly if you want to pin a path for testing or benchmarking. `#include ` `#include ` ## stream_t Owns a device stream; buffers using it must be released before the stream. ```cpp class stream_t ``` ## device_kind_t Which runtime a device belongs to, as the `nk__*` C functions name it. ```cpp enum device_kind_t ``` - `cpu_k` - `cuda_k` - `rocm_k` - `metal_k` ## device_t One device NumKong can run kernels on: the host CPU, or a GPU by its runtime's own ordinal. ```cpp class device_t ``` ## cpu The host CPU, which every build has. ```cpp device_t cpu() noexcept; ``` ## count How many devices of `kind` the process sees: one CPU, or [`missing_gpu_k`](https://www.unum.cloud/docs/numkong/cpp/quickstart.md#missing_gpu_k) and zero GPUs. ```cpp expected count(device_kind_t kind) noexcept; ``` ## make Device `ordinal` of `kind`, or [`missing_gpu_k`](https://www.unum.cloud/docs/numkong/cpp/quickstart.md#missing_gpu_k) past the last one. ```cpp expected make( device_kind_t kind, std::size_t ordinal) noexcept; ``` ## capabilities_detected What this device runs, whether or not this binary holds kernels for it; zero on failure. ```cpp expected capabilities_detected() const noexcept; ``` ## capabilities_compiled What this binary holds kernels for on devices of this kind, whether or not this one runs. ```cpp nk_capability_t capabilities_compiled() const noexcept; ``` ## capabilities_enabled Both at once: the mask for this device's dispatch points; zero on failure. ```cpp expected capabilities_enabled() const noexcept; ``` ## configure_thread Prepares the calling thread for the CPU kernels of `capabilities`; GPUs have no such kernels, so they report [`missing_kernel_k`](https://www.unum.cloud/docs/numkong/cpp/quickstart.md#missing_kernel_k). ```cpp status_t configure_thread(nk_capability_t capabilities) const noexcept; ``` ## default_capabilities The mask every wrapper dispatches over by default: the CPU's enabled capabilities, as `[device_t::cpu()](#device_t)`.capabilities_enabled() reports them, or none in header-only builds, whose dispatch points are stubs. ```cpp nk_capability_t default_capabilities() noexcept; ``` A zero mask names no capability, so the wrappers run their C++ templates instead, the references every capability is tested against. Header-only and linked C++ units must not share a binary.