# Runtime Dispatch > StringZilla for Rust The producers report `Capabilities`, a set of `Capability` bits, for the CPU or for one GPU by its runtime's own ordinal. ```rust use stringzilla::{self as sz, Capabilities, Capability}; let v = sz::version(); println!("StringZilla {}.{}.{}", v.major, v.minor, v.patch); let enabled = Capabilities::cpu_enabled(); enabled.configure_thread()?; println!("dispatching to {enabled}"); // like "serial,neon,neonaes,neonsha" for ordinal in 0..Capabilities::cuda_count_devices().unwrap_or(0) { println!("CUDA device {ordinal} runs {}", Capabilities::cuda_enabled(ordinal)?); // like "cuda" } ``` Each group, `cpu`, `cuda`, `rocm` and `metal`, reports along two independent axes, plus the set dispatch uses: - `cpu_detected()`, `cuda_detected(ordinal)`: what the device can execute, from CPUID or HWCAP on the CPU and from the runtime on a GPU - `cpu_compiled()`, `cuda_compiled()`: what this binary contains, from the probes at build time - `cpu_enabled()`, `cuda_enabled(ordinal)`: what dispatch uses, i.e. both axes at once Reach for the enabled set unless you specifically mean one of the raw axes. The detected set describes the machine and says nothing about whether a kernel was compiled in, so a build whose ISA probes failed still reports your CPU's full feature set while containing no SIMD kernels at all. The crate keeps no process state: every call passes `Capabilities::CPUS`, which the library clamps to the enabled set, while an engine keeps the capability it was built with. `cuda_count_devices()` fails with `Status::MissingGpu` for a vendor this build or machine has no device of, and `cuda_enabled(ordinal)` fails the same way past the last device. Call `configure_thread` at the start of every thread that runs kernels, to prepare it for the capabilities it passes. In a thread-pool setting, each worker thread needs its own call. The function is idempotent and cheap to call more than once on the same thread. ### Engines on a GPU Every engine constructor takes a `Stream`, whose capabilities it is built with, and a GPU stream builds it on that GPU, fixing the launch geometry once and sizing a round's device memory from `candidates_budget` for `OverlapEngine` and `haystacks_budget` for `SubstringsEngine`. The capabilities name the vendor and the stream names the device: `Stream::default(capabilities)` is the default stream of the default device, while `Stream::new(capabilities, ordinal)` makes one on any device by its ordinal and frees it on drop. `Stream::from_raw` borrows a `cudaStream_t`, a `hipStream_t` or an `id` the caller made, and is `unsafe` because no type system checks that the handle is live and belongs to the vendor. Every verb only queues its round on the stream of the `Scope` it takes, which `Stream::scope` hands its body and joins once the body returns, even when it fails. The verb borrows the engine, its inputs and its outputs for the whole scope, so none of them can be read or dropped before the device is done with them, and an engine runs one verb per scope. A stream of another group than the engine's is refused with `Status::DeviceMemoryMismatch`. A device engine reads its rounds where they lie, so its candidates and its outputs have to sit in memory the device reaches. `UnifiedAllocator::new(&stream)` is that memory on the stream's device, behind the standard `Allocator` trait, so `Vec::new_in`, `Strings` and any other allocator-aware collection lays its contents out there. `Strings::copy_in` on a CPU stream gathers a batch into one tape of bytes and offsets in that memory, and every engine reads the tape in place, while slices, host tapes and blocks of another vendor are refused with `Status::DeviceMemoryMismatch`. The conversions of a column on a GPU stream, like `escape_in` or `normalize_in`, run on that GPU and write into the allocator they are given, which has to be one the GPU reaches. ```rust use stringzilla::{Capabilities, LevenshteinEngine, LevenshteinSymbol, Stream, Strings, StringsView, UnifiedAllocator}; let cpu = Stream::default(Capabilities::cpu_enabled()); let stream = Stream::new(Capabilities::cuda_enabled(0)?, 0)?; let unified = UnifiedAllocator::new(&stream); let candidates = Strings::copy_in(StringsView::from_slices(&["sitting", "kitten"]), unified, &cpu)?; let mut distances = Vec::new_in(unified); distances.resize(2, 0usize); let mut engine = LevenshteinEngine::new(&["kitten"], LevenshteinSymbol::Bytes, &stream)?; stream.scope(|scope| engine.distances(scope, candidates.view(), None, &mut distances, 2))?; assert_eq!(distances[..], [3, 0]); ``` `stream.synchronize()` joins a stream outside any scope. A host engine imposes no such requirement: plain `Vec` and stack buffers are exactly what its verbs expect, and a column in any memory reads there too. `use stringzilla::Capability;` `use stringzilla::Capabilities;` `use stringzilla::Stream;` `use stringzilla::Scope;` ## Capability One capability, numbered like the C `sz_cap__k` bits: each architecture's in a run, ascending by dispatch preference, with the GPU vendors' groups above bit 47. ```rust pub enum Capability ``` - `Serial` = `{ _ }`: Always: fallback. - `Westmere` = `{ _ }`: Intel SSE4.2 + AES-NI. - `Goldmont` = `{ _ }`: Intel SHA-NI. - `Haswell` = `{ _ }`: Intel AVX2. - `Skylake` = `{ _ }`: Intel AVX-512. - `IceLake` = `{ _ }`: Intel AVX-512 VBMI2 + VAES. - `Neon` = `{ _ }`: Arm NEON. - `NeonAes` = `{ _ }`: Arm NEON + AES. - `NeonSha` = `{ _ }`: Arm NEON + SHA-256. - `Sve` = `{ _ }`: Arm SVE. - `Sve2` = `{ _ }`: Arm SVE2. - `Sve2Aes` = `{ _ }`: Arm SVE2 + AES. - `Rvv` = `{ _ }`: RISC-V Vector. - `RvvCrypto` = `{ _ }`: RISC-V Vector Crypto. - `V128` = `{ _ }`: WASM SIMD128. - `V128Relaxed` = `{ _ }`: WASM Relaxed SIMD. - `LoongsonAsx` = `{ _ }`: LoongArch LASX 256-bit SIMD. - `PowerVsx` = `{ _ }`: Power VSX 128-bit SIMD. - `Cuda` = `{ _ }`: NVIDIA: every CUDA device. - `Hopper` = `{ _ }`: NVIDIA: clusters, from compute capability 9.0. - `Blackwell` = `{ _ }`: NVIDIA: cluster launch control, from compute capability 10.0. - `Rocm` = `{ _ }`: AMD: every ROCm device. - `Metal` = `{ _ }`: Apple: every Metal device. Implements: `Clone`, `Copy`, `Debug`, `StructuralPartialEq`, `PartialEq`, `Eq`, `Hash`, `From<[Capability](#Capability)>`, `BitOr`, `BitOr<[Capability](#Capability)>`, `Display` ## Capabilities A set of capabilities, printed as comma-separated names like `serial,haswell`. ```rust pub struct Capabilities(_); ``` Examples: ``` use stringzilla::{Capabilities, Capability}; let enabled = Capabilities::cpu_enabled(); enabled.configure_thread()?; println!("dispatching to {enabled}"); assert!(enabled.contains(Capability::Serial)); let narrowed = enabled.without(Capability::Haswell); assert!(!narrowed.contains(Capability::Haswell)); // Counting a vendor with no devices fails with `Status::MissingGpu`. for ordinal in 0..Capabilities::cuda_count_devices().unwrap_or(0) { println!("CUDA device {ordinal} runs {}", Capabilities::cuda_enabled(ordinal)?); } ``` Implements: `Clone`, `Copy`, `Debug`, `Default`, `StructuralPartialEq`, `PartialEq`, `Eq`, `Hash`, `From<[Capability](#Capability)>`, `BitOr<[Capability](#Capability)>`, `BitOr`, `Display` ## ANY Every capability, C's `sz_cap_any_k`. ```rust pub const ANY: Self = _; ``` ## CPUS Every CPU capability, the bits below the first GPU vendor's, C's `sz_cap_cpus_k`. ```rust pub const CPUS: Self = _; ``` ## GPUS Every GPU capability, which the CPU never detects or enables, C's `sz_cap_gpus_k`. ```rust pub const GPUS: Self = _; ``` ## bits The raw `sz_capability_t` mask. ```rust pub const fn bits(self) -> u64 ``` ## contains Whether `capability` is in this set. ```rust pub const fn contains(self, capability: Capability) -> bool ``` ## without This set less `capability`, like `enabled.without(Capability::Skylake)`. ```rust pub const fn without(self, capability: Capability) -> Self ``` ## iter The capabilities in this set, in bit order. ```rust pub fn iter(self) -> impl Iterator ``` ## cpu_detected Capabilities this CPU supports, whether or not their kernels were compiled in. ```rust pub fn cpu_detected() -> Self ``` ## cpu_compiled CPU capabilities whose kernels were compiled in, whether or not this CPU supports them. ```rust pub fn cpu_compiled() -> Self ``` ## cpu_enabled CPU capabilities dispatch runs: [`Capabilities::cpu_detected`](#cpu_detected) & [`Capabilities::cpu_compiled`](#cpu_compiled), always with [`Capability::Serial`](#Serial). ```rust pub fn cpu_enabled() -> Self ``` The library clamps every CPU mask to this set itself, so calls without a mask pass [`Capabilities::CPUS`](#CPUS). ## configure_thread Sets up the calling thread for the CPU kernels in this set, usually [`Capabilities::cpu_enabled`](#cpu_enabled). ```rust pub fn configure_thread(self) -> Result<(), Status> ``` Call it once per thread before using those kernels; it is idempotent. ## cuda_count_devices How many CUDA devices this process sees, failing with [`Status::MissingGpu`](https://www.unum.cloud/docs/stringzilla/rust/types.md#MissingGpu) where the runtime finds none or this build lacks its kernels. ```rust pub fn cuda_count_devices() -> Result ``` ## cuda_detected Capabilities CUDA device `ordinal` supports, as its runtime numbers them, whether or not their kernels were compiled in; fails with [`Status::MissingGpu`](https://www.unum.cloud/docs/stringzilla/rust/types.md#MissingGpu) past the last device. ```rust pub fn cuda_detected(ordinal: usize) -> Result ``` ## cuda_compiled CUDA capabilities whose kernels were compiled in, whether or not any device supports them. ```rust pub fn cuda_compiled() -> Self ``` ## cuda_enabled Capabilities kernels run with on CUDA device `ordinal`: [`Capabilities::cuda_detected`](#cuda_detected) & [`Capabilities::cuda_compiled`](#cuda_compiled). ```rust pub fn cuda_enabled(ordinal: usize) -> Result ``` The mask names the vendor, and a stream the device. ## rocm_count_devices [`Capabilities::cuda_count_devices`](#cuda_count_devices), for ROCm. ```rust pub fn rocm_count_devices() -> Result ``` ## rocm_detected [`Capabilities::cuda_detected`](#cuda_detected), for ROCm. ```rust pub fn rocm_detected(ordinal: usize) -> Result ``` ## rocm_compiled [`Capabilities::cuda_compiled`](#cuda_compiled), for ROCm. ```rust pub fn rocm_compiled() -> Self ``` ## rocm_enabled [`Capabilities::cuda_enabled`](#cuda_enabled), for ROCm. ```rust pub fn rocm_enabled(ordinal: usize) -> Result ``` ## metal_count_devices [`Capabilities::cuda_count_devices`](#cuda_count_devices), for Metal. ```rust pub fn metal_count_devices() -> Result ``` ## metal_detected [`Capabilities::cuda_detected`](#cuda_detected), for Metal. ```rust pub fn metal_detected(ordinal: usize) -> Result ``` ## metal_compiled [`Capabilities::cuda_compiled`](#cuda_compiled), for Metal. ```rust pub fn metal_compiled() -> Self ``` ## metal_enabled [`Capabilities::cuda_enabled`](#cuda_enabled), for Metal. ```rust pub fn metal_enabled(ordinal: usize) -> Result ``` ## Stream A stream of one capability group's device, where engines, [`StringsView`](https://www.unum.cloud/docs/stringzilla/rust/types.md#StringsView) conversions and [`UnifiedAllocator`](https://www.unum.cloud/docs/stringzilla/rust/memory-operations.md#UnifiedAllocator) blocks queue work: a vendor's default when null, the CPU's always null. ```rust pub struct Stream ``` Examples: ``` use stringzilla::{Capabilities, Stream}; let cpu = Stream::default(Capabilities::cpu_enabled()); assert!(cpu.as_raw().is_null()); // A vendor with no devices fails with `Status::MissingGpu`. for ordinal in 0..Capabilities::cuda_count_devices().unwrap_or(0) { let stream = Stream::new(Capabilities::cuda_enabled(ordinal)?, ordinal)?; stream.synchronize()?; } ``` Implements: `Debug`, `Drop` ## default The default stream of the default device of the group [`capabilities`](#capabilities) name. ```rust pub fn default(capabilities: Capabilities) -> Self ``` ## new A new stream on device `ordinal` of the vendor [`capabilities`](#capabilities) name, a `cudaStream_t`, a `hipStream_t` or an `id`, freed on drop. ```rust pub fn new(capabilities: Capabilities, ordinal: usize) -> Result ``` The CPU gets [`Stream::default`](#default). Fails with [`Status::MissingGpu`](https://www.unum.cloud/docs/stringzilla/rust/types.md#MissingGpu) past the last device or where this build lacks the vendor. ## from_raw Borrows a stream the caller made and keeps owning, which is never freed here. ```rust pub unsafe fn from_raw(capabilities: Capabilities, handle: *mut c_void) -> Self ``` Safety: `handle` must be null or a live stream of the vendor [`capabilities`](#capabilities) name, a `cudaStream_t`, a `hipStream_t` or an `id`, null on the CPU, and outlive the result. ## capabilities The capabilities every consumer of this stream dispatches with. ```rust pub fn capabilities(&self) -> Capabilities ``` ## as_raw The vendor's handle, null for a default stream. ```rust pub fn as_raw(&self) -> *mut c_void ``` ## synchronize Waits for everything queued on this stream, after which whatever its rounds wrote is readable from the host. ```rust pub fn synchronize(&self) -> Result<(), Status> ``` The CPU has nothing to wait for. ## scope Runs `body`, whose engine verbs queue on this stream, then joins the stream, even when `body` fails or unwinds, and reports the first failure, the body's or the join's. ```rust pub fn scope<'env, Output>( &'env self, body: impl for<'scope> FnOnce(&'scope Scope<'scope, 'env>) -> Result, ) -> Result ``` A verb borrows its engine, inputs and outputs for the whole scope, so none of them can be touched before the join, and an engine runs one verb per scope. Examples: ``` use stringzilla::{Capabilities, LevenshteinEngine, LevenshteinSymbol, Stream, StringsView}; let stream = Stream::default(Capabilities::cpu_enabled()); let mut engine = LevenshteinEngine::new(&["kitten"], LevenshteinSymbol::Bytes, &stream)?; let mut distances = [0usize; 1]; let candidates = StringsView::from_slices(&["sitting"]); stream.scope(|scope| engine.distances(scope, candidates, None, &mut distances, 1))?; assert_eq!(distances[0], 3); ``` ## Scope The span of a [`Stream::scope`](#scope) body, which every engine verb takes to queue on that stream. ```rust pub struct Scope<'scope, 'env: 'scope> ``` A verb borrows what it reads and writes for `'scope`, which ends only once the stream is joined.