# Runtime Dispatch

> StringZilla for C++

Every kernel exists in a serial form plus one or more per-ISA forms, named with a backend suffix (`_serial`, `_westmere`, `_goldmont`, `_haswell`, `_skylake`, `_icelake`, `_neon`, `_neonaes`, `_neonsha`, `_sve`, `_sve2`, `_sve2aes`, `_v128`, `_v128relaxed`, `_rvv`, `_rvvcrypto`, `_loongsonasx`, `_powervsx`), and the engines add one per GPU vendor, `_cuda`, `_rocm` and `_metal`.
Each verb also has a dispatch point, like `sz_find_best` or `sz_hash_best`, taking the kernels' arguments plus a capability mask before the trailing stream, and running the best capability the mask shares with the verb's list, `serial` first on the CPU.
It returns `sz_missing_kernel_k` when no capability in the mask has the kernel, as a GPU-only mask does for a CPU-only verb.

- __Library builds, `STRINGZILLA_HEADER_ONLY=0`, the default.__
  Every capability the toolchain builds is compiled into one binary, and the dispatch points pick among them by the mask each call passes.
  This is how the prebuilt libraries ship, so a single binary runs optimally on any CPU.
- __Header-only builds, `STRINGZILLA_HEADER_ONLY=1`.__
  The kernels your compiler flags enable inline into your translation unit, so calling one of them by name, like `sz_find_haswell`, costs no indirection at all.
  The dispatch points and finders compile to stubs returning `sz_missing_library_k`, while the C++ layer runs the serial kernels over its default mask, which is empty here.

Capabilities are introspectable at both compile and run time.
`sz_capability_t` is a 64-bit mask from `capabilities.h`, one bit per capability.
Within each architecture the bits ascend by preference, so the highest bit a mask shares with a verb's kernels names the kernel to run.

- __Serial__: `sz_cap_serial_k`, bit 0, which every CPU runs.
- __x86__: `sz_cap_westmere_k`, `sz_cap_goldmont_k`, `sz_cap_haswell_k`, `sz_cap_skylake_k`, `sz_cap_icelake_k`.
- __Arm__: `sz_cap_neon_k`, `sz_cap_neonaes_k`, `sz_cap_neonsha_k`, `sz_cap_sve_k`, `sz_cap_sve2_k`, `sz_cap_sve2aes_k`.
- __RISC-V__: `sz_cap_rvv_k`, `sz_cap_rvvcrypto_k`.
- __WebAssembly__: `sz_cap_v128_k`, `sz_cap_v128relaxed_k`.
- __LoongArch and Power__: `sz_cap_loongsonasx_k`, `sz_cap_powervsx_k`.
- __GPUs__: one baseline per vendor above every CPU bit, `sz_cap_cuda_k` at bit 48, `sz_cap_rocm_k` at 56, and `sz_cap_metal_k` at 60.
  CUDA generations follow its baseline: `sz_cap_hopper_k` at 51 for clusters from compute capability 9.0, and `sz_cap_blackwell_k` at 52 for cluster launch control from 10.0.

`sz_cap_cpus_k` and `sz_cap_gpus_k` group the CPU and the GPU bits, and `sz_cap_any_k` sets every bit.

```c
sz_status_t sz_capabilities_detected_cpu(sz_capability_t *capabilities); // what this CPU runs
sz_status_t sz_capabilities_compiled_cpu(sz_capability_t *capabilities); // what this build holds kernels for
sz_capability_t sz_capabilities_compiled_cpu_mask(void);                 // the same, constant in the calling unit
sz_status_t sz_capabilities_enabled_cpu(sz_capability_t *capabilities);  // both at once, always with serial
sz_status_t sz_thread_configure_cpu(sz_capability_t capabilities);       // once per dispatching thread
sz_status_t sz_device_count_cuda(sz_size_t *count);
sz_status_t sz_capabilities_detected_cuda(sz_size_t ordinal, sz_capability_t *capabilities);
sz_status_t sz_capabilities_compiled_cuda(sz_capability_t *capabilities);
sz_status_t sz_capabilities_enabled_cuda(sz_size_t ordinal, sz_capability_t *capabilities);
sz_size_t sz_capabilities_name(sz_capability_t capabilities, char *buffer, sz_size_t capacity); // "serial,haswell"
char const *sz_status_name(sz_status_t status);
```

ROCm and Metal have the same four queries, spelled `sz_rocm_*` and `sz_metal_*`.
The library probes the CPU once per process and caches the answer, while header-only builds probe on every call.
The GPU queries ask the vendor's runtime every time, by that runtime's own device ordinal, and report no devices where the vendor isn't built.
The CPU probe inspects CPUID on x86, the AArch64 ID registers on Arm once the kernel's `HWCAP_CPUID` says it emulates `mrs`, falling back to NEON-only, `getauxval`/`riscv_hwprobe` on RISC-V, and the auxiliary-vector HWCAPs on LoongArch and Power.
Detection always reports the full hardware truth, independent of which tiers a build compiled in; `sz_capabilities_enabled_cpu()` intersects it with the compile-time mask.
Where the OS cannot be asked, detection reports only the capabilities the compiler's own flags guarantee, never the compiled ones, which the CPU may lack.
WebAssembly is the exception with no runtime probe at all — a module carrying unsupported SIMD opcodes fails validation at instantiation, so its capabilities are fixed at compile time.
Nothing is process-wide: narrowing the mask a call passes narrows the choice, so `capabilities & ~sz_cap_sve_k` skips SVE and `sz_cap_serial_k` alone runs the reference kernel.
A host-only process never starts a GPU driver, as only the GPU queries and the engines built for a GPU reach one.
`sz_thread_configure_cpu` prepares the calling thread for the capabilities it is given, and only those; every current capability needs nothing, so it is a no-op today, kept so bindings call it where NumKong's call theirs.

On Arm, having SVE in the capability mask doesn't mean SVE kernels always win: at the common 128-bit vector length the scalable kernels for length-sensitive operations — comparisons, memory transforms, substring search, UTF-8 token scanning — are often slower than their NEON twins, while crypto-heavy operations like hashing prefer SVE2 at any width.
So each such SVE kernel measures the register width on the running CPU (`svcntb`) and hands the 128-bit case to its NEON twin, and the dispatch point needs no special case.
To force a specific backend regardless, narrow the mask or call its kernel directly.

CMake, which every binding builds through, probes which capabilities the toolchain can __compile__: each capability's probe, `probes/<capability>.c`, calls one of its kernels, compiled header-only at the baseline flags as the library compiles it.
Each capability's headers scope its kernels with a target pragma, on every platform; LASX and POWER9 need GCC 15 or newer for that, as Clang's `lasxintrin.h` and `altivec.h` stay closed without `-mlasx` and `-mcpu=power9`.
Libraries compile every capability the toolchain builds and leave the rest to the mask each call passes; header-only builds enable the capabilities their own flags name.
`-D STRINGZILLA_TARGET_<CAPABILITY>=0` drops a capability from the libraries, and `=1` keeps one only where its probe compiles.

```c
#include <stdio.h>
#include <stringzilla/stringzilla.h>

int main(void) {
    sz_capability_t caps;
    sz_capabilities_enabled_cpu(&caps);
    char names[STRINGZILLA_CAPABILITIES_NAME_CAPACITY];
    sz_capabilities_name(caps, names, sizeof(names));
    printf("StringZilla %d.%d.%d, backends: %s\n", sz_version_major(), sz_version_minor(), sz_version_patch(), names);
    return 0;
}
```

You can also call any capability's kernel directly, with the same arguments short of the mask, when you have already established the target supports it — for example `sz_find_haswell(...)` or `sz_hash_icelake(...)` — which is handy for benchmarking and for pinning a code path in a controlled environment.

Picking a capability costs about a nanosecond per call: noise for long inputs, but not for hot short-string paths, like a sort's comparator or a per-token byte search.
There, call a capability's kernel directly, like `sz_order_neon` or `sz_find_byte_haswell`, or resolve one once through `sz_find_kernel_punned`, and call it many times:

```c
sz_kernel_order_t order = NULL;
sz_capability_t capability = 0;
sz_find_kernel_punned(sz_kernel_order_k, caps, (sz_kernel_punned_t *)&order, &capability);

sz_ordering_t ordering;
order("apple", 5, "banana", 6, &ordering, NULL);
```

It picks exactly what the dispatch point picks for the same mask, and returns `sz_missing_kernel_k` when no capability in the mask has the kernel.
Each family exports its own finder over its kinds, like `sz_compare_find_kernel` in `compare.h`, and `sz_find_kernel_punned` covers every family through one entry point.
Every verb has a kind, `sz_kernel_<verb>_k`, and a pointer type to cast the result to, `sz_kernel_<verb>_t`, taking the dispatch point's arguments short of the mask.
`sz_kernel_name` spells a kind without its `sz_kernel_` prefix and `_k` suffix, like `"find_byte"`, and `sz_kernel_named` maps such a name back, or to `sz_kernel_unknown_k`.
Verbs of one shape share a type, like `sz_kernel_find_t` for both `sz_kernel_find_k` and `sz_kernel_rfind_k`, `sz_kernel_utf8_segmenter_t` for the four tiling UTF-8 segmenters, and `sz_kernel_utf8_tokenizer_t` for the newline, whitespace and delimiter finders.
In C++, every wrapper that takes a mask, like `sz::lookup` or `sz::argsort`, defaults to `sz::default_capabilities()`, the enabled CPU mask, while `sz::device_t::make(kind, ordinal)` opens one GPU to ask its `capabilities_enabled()`, and `sz::device_t::cpu().configure_thread(mask)` wraps `sz_thread_configure_cpu`.

`#include <stringzilla/types.hpp>`

## device_kind_t

Which runtime a device belongs to, as the `sz_<kind>_*` C functions name it.

```cpp
enum device_kind_t
```

- `cpu_k`
- `cuda_k`
- `rocm_k`
- `metal_k`

## gpu_specs_t

Specifications of a typical NVIDIA GPU, such as A100 or H100.

```cpp
struct gpu_specs_t
```

- `vram_bytes` `std::size_t`
- `l2_bytes` `std::size_t`
- `constant_memory_bytes` `std::size_t`
- `shared_memory_bytes` `std::size_t`
- `streaming_multiprocessors` `std::size_t`
- `cuda_cores` `std::size_t`
- `reserved_memory_per_block` `std::size_t`
- `warp_size` `std::size_t`
- `max_blocks_per_multiprocessor` `std::size_t`
- `sm_code` `std::size_t`

Notes: We recommend compiling the code for the 90a compute capability, the newest with specialized optimizations.

See also: pack_sm_code, cores_per_multiprocessor

## device_t

One device StringZilla runs kernels on: the host CPU, or a GPU by its runtime's own ordinal.

```cpp
class device_t
```

## cpu

The host CPU, which every build has.

```cpp
device_t cpu() noexcept;
```

## count

How many devices of `kind` the process sees: one CPU, or [`missing_gpu_k`](https://www.unum.cloud/docs/stringzilla/cpp/types.md#missing_gpu_k) and zero GPUs.

```cpp
expected<std::size_t> count(device_kind_t kind) noexcept;
```

## make

Device `ordinal` of `kind`, or [`missing_gpu_k`](https://www.unum.cloud/docs/stringzilla/cpp/types.md#missing_gpu_k) past the last one.

```cpp
expected<device_t> make(
    device_kind_t kind,
    std::size_t ordinal) noexcept;
```

## capabilities_detected

What this device runs, whether or not this binary holds kernels for it; zero on failure.

```cpp
expected<sz_capability_t> capabilities_detected() const noexcept;
```

## capabilities_compiled

What this binary holds kernels for on devices of this kind, whether or not this one runs.

```cpp
sz_capability_t capabilities_compiled() const noexcept;
```

## capabilities_enabled

Both at once: the mask for this device's dispatch points; zero on failure.

```cpp
expected<sz_capability_t> capabilities_enabled() const noexcept;
```

## configure_thread

Prepares the calling thread for the CPU kernels of `capabilities`; GPUs have no such kernels, so they report [`missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/cpp/types.md#missing_kernel_k).

```cpp
status_t configure_thread(sz_capability_t capabilities) const noexcept;
```

## default_capabilities

The mask every wrapper dispatches over by default: the CPU's enabled capabilities, as `[device_t::cpu()](#device_t)`.capabilities_enabled() reports them, or none in header-only builds, whose dispatch points are stubs, so every wrapper runs the serial reference instead.

```cpp
sz_capability_t default_capabilities() noexcept;
```