# Runtime Dispatch

> NumKong for Python

`import numkong`

## Capability

CPU SIMD capabilities, one bit each, as the `capabilities_*` functions report and set them.

```python
class Capability(IntFlag)
```

- `SERIAL`: Serial code without SIMD, available on every CPU.
- `NEON`: Arm NEON, the AArch64 Advanced SIMD baseline.
- `NEONHALF`: NEON with half-precision arithmetic, FEAT_FP16.
- `NEONBFDOT`: NEON with BFloat16 dot products, FEAT_BF16.
- `NEONFHM`: NEON with half-precision multiply-accumulate into single precision, FEAT_FHM.
- `NEONSDOT`: NEON with 8-bit integer dot products, FEAT_DotProd.
- `NEONFP8`: NEON with FP8 dot products, FEAT_FP8DOT4.
- `SVE`: Arm Scalable Vector Extension.
- `SVEHALF`: SVE with half-precision arithmetic.
- `SVESDOT`: SVE with 8-bit integer matrix multiplies, FEAT_I8MM.
- `SVEBFDOT`: SVE with BFloat16 dot products.
- `SVE2`: Arm SVE2.
- `SME`: Arm Scalable Matrix Extension.
- `SMEF64`: SME with double-precision outer products, FEAT_SME_F64F64.
- `SMEBI32`: SME with binary outer products into 32-bit integers, SME_BI32I32.
- `HASWELL`: x86 AVX2, FMA, F16C, BMI1 and BMI2, as on Intel Haswell.
- `ALDER`: [`HASWELL`](#HASWELL) plus AVX-VNNI, as on Intel Alder Lake.
- `SIERRA`: [`ALDER`](#ALDER) plus AVX-VNNI-INT8, as on Intel Sierra Forest.
- `SKYLAKE`: [`HASWELL`](#HASWELL) plus AVX-512 F, DQ, BW and VL, as on Intel Skylake-X.
- `ICELAKE`: [`SKYLAKE`](#SKYLAKE) plus AVX-512 VNNI, IFMA, BITALG, VBMI, VBMI2 and VPOPCNTDQ, as on Ice Lake.
- `GENOA`: [`SKYLAKE`](#SKYLAKE) plus AVX-512 BF16 and VNNI, as on AMD Genoa.
- `TURIN`: [`GENOA`](#GENOA) plus AVX-512 VBMI2 and VP2INTERSECT, as on AMD Turin.
- `SAPPHIRE`: [`SKYLAKE`](#SKYLAKE) plus AVX-512 FP16, as on Intel Sapphire Rapids.
- `DIAMOND`: [`SAPPHIRE`](#SAPPHIRE) plus AVX10.2, as on Intel Diamond Rapids.
- `SAPPHIREAMX`: [`SAPPHIRE`](#SAPPHIRE) and [`GENOA`](#GENOA) plus AMX tiles with BF16 and INT8, as on Intel Sapphire Rapids.
- `GRANITEAMX`: [`SAPPHIREAMX`](#SAPPHIREAMX) plus AMX-FP16, as on Intel Granite Rapids.
- `DIAMONDAMX`: [`SAPPHIREAMX`](#SAPPHIREAMX) plus AMX-FP8 and AMX-AVX512, as on Intel Diamond Rapids.
- `RVV`: RISC-V Vector extension.
- `RVVBF16`: RVV with BFloat16 widening multiply-accumulate, Zvfbfwma.
- `RVVHALF`: RVV with half-precision arithmetic, Zvfh.
- `RVVBB`: RVV with basic bit manipulation, Zvbb.
- `V128`: WebAssembly 128-bit SIMD.
- `V128RELAXED`: WebAssembly Relaxed SIMD.
- `POWERVSX`: IBM Power VSX at ISA 3.0, as on POWER9.
- `LOONGSONASX`: LoongArch LASX 256-bit SIMD.

## cpu_capabilities_detected

What this CPU can execute.

```python
def cpu_capabilities_detected() -> Capability
```

## cpu_capabilities_compiled

What this binary holds CPU kernels for.

```python
def cpu_capabilities_compiled() -> Capability
```

## cpu_capabilities_enabled

What CPU kernels run within: both of the above, settled as the library loads.

```python
def cpu_capabilities_enabled() -> Capability
```

## cpu_configure_thread

Prepare the calling thread for the CPU kernels of `capabilities`.

```python
def cpu_configure_thread(capabilities: Capability, /) -> None
```

## cuda_count_devices

How many CUDA devices this process sees, zero without the runtime or its kernels.

```python
def cuda_count_devices() -> int
```

## cuda_capabilities_detected

What CUDA device `ordinal` can execute, raising ValueError past the last one.

```python
def cuda_capabilities_detected(ordinal: int, /) -> Capability
```

## cuda_capabilities_compiled

What this binary holds CUDA kernels for.

```python
def cuda_capabilities_compiled() -> Capability
```

## cuda_capabilities_enabled

The mask CUDA device `ordinal` dispatches with, raising ValueError past the last one.

```python
def cuda_capabilities_enabled(ordinal: int, /) -> Capability
```

## stream_init_cuda

A new `cudaStream_t` on CUDA device `ordinal` as an integer, raising ValueError past the last one.

```python
def stream_init_cuda(ordinal: int, /) -> int
```

## stream_free_cuda

Free a stream [`stream_init_cuda`](#stream_init_cuda) made, once [`synchronize`](#synchronize) joined its work.

```python
def stream_free_cuda(stream: int, /) -> None
```

## rocm_count_devices

How many ROCm devices this process sees, zero without the runtime or its kernels.

```python
def rocm_count_devices() -> int
```

## rocm_capabilities_detected

What ROCm device `ordinal` can execute, raising ValueError past the last one.

```python
def rocm_capabilities_detected(ordinal: int, /) -> Capability
```

## rocm_capabilities_compiled

What this binary holds ROCm kernels for.

```python
def rocm_capabilities_compiled() -> Capability
```

## rocm_capabilities_enabled

The mask ROCm device `ordinal` dispatches with, raising ValueError past the last one.

```python
def rocm_capabilities_enabled(ordinal: int, /) -> Capability
```

## stream_init_rocm

A new `hipStream_t` on ROCm device `ordinal` as an integer, raising ValueError past the last one.

```python
def stream_init_rocm(ordinal: int, /) -> int
```

## stream_free_rocm

Free a stream [`stream_init_rocm`](#stream_init_rocm) made, once [`synchronize`](#synchronize) joined its work.

```python
def stream_free_rocm(stream: int, /) -> None
```

## metal_count_devices

How many Metal devices this process sees, zero without the runtime or its kernels.

```python
def metal_count_devices() -> int
```

## metal_capabilities_detected

What Metal device `ordinal` can execute, raising ValueError past the last one.

```python
def metal_capabilities_detected(ordinal: int, /) -> Capability
```

## metal_capabilities_compiled

What this binary holds Metal kernels for.

```python
def metal_capabilities_compiled() -> Capability
```

## metal_capabilities_enabled

The mask Metal device `ordinal` dispatches with, raising ValueError past the last one.

```python
def metal_capabilities_enabled(ordinal: int, /) -> Capability
```

## stream_init_metal

A new `id<MTLCommandQueue>` on Metal device `ordinal` as an integer, raising ValueError past the last one.

```python
def stream_init_metal(ordinal: int, /) -> int
```

## stream_free_metal

Free a stream [`stream_init_metal`](#stream_init_metal) made, once [`synchronize`](#synchronize) joined its work.

```python
def stream_free_metal(stream: int, /) -> None
```

## synchronize

Wait for everything queued on `stream` of the device `capabilities` describes, None for its default.

```python
def synchronize(
    capabilities: Capability | int,
    /,
    stream: int | None = None,
) -> None
```