# Runtime Dispatch

> NumKong for Go

A `Device` is the host CPU, from `CPU()`, or a GPU by its runtime's own ordinal, from `NewDevice(DeviceCUDA, 0)` and its `DeviceROCm` and `DeviceMetal` twins, counted by `CountDevices`.
`Device.ConfigureThread` pins the current goroutine to an OS thread via `runtime.LockOSThread`, enables the CPU-specific state the given capabilities need, such as Intel AMX tiles, then returns an unlock function.
Goroutines can migrate between OS threads, so thread-local state (AMX tiles) would be lost without pinning.

```go
cpu := nk.CPU()
enabled, _ := cpu.CapabilitiesEnabled() // what dispatch runs within: detected on this CPU and compiled in
unlock, _ := cpu.ConfigureThread(enabled) // lock thread, configure what dispatch uses
defer unlock()                            // release the OS thread on return

fmt.Println(enabled)                 // like "serial,neon,neonhalf,neonfhm,neonsdot"
fmt.Println(enabled.Has(nk.CapNEON)) // test one capability

if gpu, err := nk.NewDevice(nk.DeviceMetal, 0); err == nil {
	fmt.Println(gpu.CapabilitiesEnabled()) // like "metal,apple9"
}
```

`CapabilitiesDetected` and `CapabilitiesCompiled` report the two raw axes, what the device executes and what this binary contains for its kind.
Every kernel call passes `CapCPUs`, which the library clamps to the CPU's `CapabilitiesEnabled`, settled as it loads, so the package keeps no mask of its own.
Every capability is a typed `Capability` constant, like `CapSerial`, `CapNEON`, `CapHaswell`, `CapSapphireAMX`, `CapSME`, `CapAmpere`, and `CapApple9`, and `CapCPUs`, `CapGPUs`, and `CapAny` group them.
These are useful for logging the active platform or gating optional benchmark paths.

`import "github.com/ashvardanian/NumKong"`

## DeviceKind

DeviceKind is the runtime a device belongs to, as the `nk_<kind>_\*` C functions name it.

```go
type DeviceKind int
```

- `DeviceCPU` = `iota`
- `DeviceCUDA`
- `DeviceROCm`
- `DeviceMetal`

## Capability

Capability is one capability of a CPU or a GPU, or a set of them.

```go
type Capability uint64
```

- `CapSerial`: Always: Fallback
- `CapNEON`: 2013: ARM NEON
- `CapNEONHalf`: 2017: ARM NEON FP16
- `CapNEONBFDot`: 2020: ARM NEON BF16
- `CapNEONFHM`: 2018: ARM NEON FP16 FML
- `CapNEONSDot`: 2017: ARM NEON i8 dot
- `CapNEONFP8`: ARM NEON FP8
- `CapSVE`: 2020: ARM SVE
- `CapSVEHalf`: 2020: ARM SVE FP16
- `CapSVESDot`: 2020: ARM SVE i8 dot
- `CapSVEBFDot`: 2021: ARM SVE BF16
- `CapSVE2`: 2022: ARM SVE2
- `CapSME`: 2024: ARM SME
- `CapSMEF64`: 2024: ARM SME F64
- `CapSMEBi32`: 2025+: ARM SME BI32I32
- `CapHaswell`: 2013: Intel AVX2
- `CapAlder`: 2021: Intel AVX2+VNNI
- `CapSierra`: 2024: Intel AVXVNNIINT8
- `CapSkylake`: 2017: Intel AVX-512
- `CapIceLake`: 2019: Intel AVX-512 VNNI
- `CapGenoa`: 2020: AMD AVX-512 BF16
- `CapTurin`: 2024: AMD Turin AVX-512 CD
- `CapSapphire`: 2023: Intel AVX-512 FP16
- `CapDiamond`: 2025+: Intel AVX10.2
- `CapSapphireAMX`: 2023: Intel Sapphire AMX
- `CapGraniteAMX`: 2024: Intel Granite AMX FP16
- `CapDiamondAMX`: Intel Diamond Rapids AMX
- `CapRVV`: 2023: RISC-V Vector
- `CapRVVBF16`: 2023: RISC-V Zvfbfwma
- `CapRVVHalf`: 2023: RISC-V Zvfh
- `CapRVVBB`: RISC-V: Byte-Byte extensions
- `CapV128`: 2021: WASM SIMD128
- `CapV128Relaxed`: 2022: WASM Relaxed SIMD
- `CapPowerVSX`: Power VSX 128-bit SIMD
- `CapLoongsonASX`: LoongArch LASX 256-bit SIMD
- `CapCUDA`: Any CUDA device
- `CapAmpere`: 2020: NVIDIA SM 8.0
- `CapAda`: 2022: NVIDIA SM 8.9
- `CapHopper`: 2022: NVIDIA SM 9.x
- `CapBlackwell`: 2024: NVIDIA SM 10.x
- `CapBlackwellRTX`: 2025: NVIDIA SM 12.x
- `CapBlackwellUltra`: 2025: NVIDIA SM 10.3
- `CapROCm`: Any ROCm device
- `CapCDNA3`: 2023: AMD gfx942
- `CapCDNA4`: 2025: AMD gfx950
- `CapCDNA5`: AMD gfx1250
- `CapMetal`: Any Metal device
- `CapApple9`: 2023: Apple GPU family 9
- `CapApple10`: Apple GPU family 10
- `CapCPUs`: Every CPU capability, which every kernel call passes
- `CapGPUs`: Every GPU capability
- `CapAny` = `^Capability(0)`: Every capability

## Has

Has reports whether any bit of capability is in c.

```go
func (c Capability) Has(capability Capability) bool
```

## String

String names the capabilities in c, comma-separated, like "serial,haswell".

```go
func (c Capability) String() string
```

## Device

Device is one device NumKong can run kernels on: the host CPU, or a GPU by its runtime's own ordinal, the one `cudaSetDevice` or `hipSetDevice` takes, or the position in Metal's device list.

```go
type Device struct {
    Kind    DeviceKind
    Ordinal int
}
```

## CPU

CPU returns the host CPU, which every build has.

```go
func CPU() Device
```

## NewDevice

NewDevice returns device ordinal of kind, failing past the last one.

```go
func NewDevice(kind DeviceKind, ordinal int) (Device, error)
```

## CapabilitiesCompiled

CapabilitiesCompiled returns the capabilities whose kernels were compiled in for devices of d's kind, whether or not d runs them.

```go
func (d Device) CapabilitiesCompiled() Capability
```

## CapabilitiesDetected

CapabilitiesDetected returns the capabilities d runs, whether or not they were compiled in.

```go
func (d Device) CapabilitiesDetected() (Capability, error)
```

## CapabilitiesEnabled

CapabilitiesEnabled returns the mask d's kernel calls run within: [Device.CapabilitiesDetected](#CapabilitiesDetected) and [Device.CapabilitiesCompiled](#CapabilitiesCompiled) at once.

```go
func (d Device) CapabilitiesEnabled() (Capability, error)
```

On the CPU the library settles it as it loads, it always has [CapSerial](#CapSerial), and every kernel call of this package passes [CapCPUs](#CapCPUs) clamped to it.

## ConfigureThread

ConfigureThread pins the goroutine to an OS thread, configures its SIMD state for capabilities, usually the CPU's [Device.CapabilitiesEnabled](#CapabilitiesEnabled), and returns the unlock function.

```go
func (d Device) ConfigureThread(
    capabilities Capability,
) (func(), error)
```

Call it, typically via defer, once the SIMD work is done. GPUs have no thread state to configure.

## CountDevices

CountDevices returns how many devices of kind the process sees: one CPU, or the GPUs its runtime counts, failing without one.

```go
func CountDevices(kind DeviceKind) (int, error)
```