Runtime Dispatch

The producers report Capabilities, a set of Capability bits, for the CPU or for one GPU by its runtime’s own ordinal.

use stringzilla::{self as sz, Capabilities, Capability};

let v = sz::version();
println!("StringZilla {}.{}.{}", v.major, v.minor, v.patch);

let enabled = Capabilities::cpu_enabled();
enabled.configure_thread()?;
println!("dispatching to {enabled}"); // like "serial,neon,neonaes,neonsha"

for ordinal in 0..Capabilities::cuda_count_devices().unwrap_or(0) {
    println!("CUDA device {ordinal} runs {}", Capabilities::cuda_enabled(ordinal)?); // like "cuda"
}

Each group, cpu, cuda, rocm and metal, reports along two independent axes, plus the set dispatch uses:

• cpu_detected(), cuda_detected(ordinal): what the device can execute, from CPUID or HWCAP on the CPU and from the runtime on a GPU
• cpu_compiled(), cuda_compiled(): what this binary contains, from the probes at build time
• cpu_enabled(), cuda_enabled(ordinal): what dispatch uses, i.e. both axes at once

Reach for the enabled set unless you specifically mean one of the raw axes. The detected set describes the machine and says nothing about whether a kernel was compiled in, so a build whose ISA probes failed still reports your CPU’s full feature set while containing no SIMD kernels at all. The crate keeps no process state: every call passes Capabilities::CPUS, which the library clamps to the enabled set, while an engine keeps the capability it was built with. cuda_count_devices() fails with Status::MissingGpu for a vendor this build or machine has no device of, and cuda_enabled(ordinal) fails the same way past the last device.

Call configure_thread at the start of every thread that runs kernels, to prepare it for the capabilities it passes. In a thread-pool setting, each worker thread needs its own call. The function is idempotent and cheap to call more than once on the same thread.

§ Engines on a GPU

Every engine constructor takes a Stream, whose capabilities it is built with, and a GPU stream builds it on that GPU, fixing the launch geometry once and sizing a round’s device memory from candidates_budget for OverlapEngine and haystacks_budget for SubstringsEngine. The capabilities name the vendor and the stream names the device: Stream::default(capabilities) is the default stream of the default device, while Stream::new(capabilities, ordinal) makes one on any device by its ordinal and frees it on drop. Stream::from_raw borrows a cudaStream_t, a hipStream_t or an id<MTLCommandQueue> the caller made, and is unsafe because no type system checks that the handle is live and belongs to the vendor.

Every verb only queues its round on the stream of the Scope it takes, which Stream::scope hands its body and joins once the body returns, even when it fails. The verb borrows the engine, its inputs and its outputs for the whole scope, so none of them can be read or dropped before the device is done with them, and an engine runs one verb per scope. A stream of another group than the engine’s is refused with Status::DeviceMemoryMismatch.

A device engine reads its rounds where they lie, so its candidates and its outputs have to sit in memory the device reaches. UnifiedAllocator::new(&stream) is that memory on the stream’s device, behind the standard Allocator trait, so Vec::new_in, Strings and any other allocator-aware collection lays its contents out there. Strings::copy_in on a CPU stream gathers a batch into one tape of bytes and offsets in that memory, and every engine reads the tape in place, while slices, host tapes and blocks of another vendor are refused with Status::DeviceMemoryMismatch. The conversions of a column on a GPU stream, like escape_in or normalize_in, run on that GPU and write into the allocator they are given, which has to be one the GPU reaches.

use stringzilla::{Capabilities, LevenshteinEngine, LevenshteinSymbol, Stream, Strings, StringsView, UnifiedAllocator};

let cpu = Stream::default(Capabilities::cpu_enabled());
let stream = Stream::new(Capabilities::cuda_enabled(0)?, 0)?;
let unified = UnifiedAllocator::new(&stream);
let candidates = Strings::copy_in(StringsView::from_slices(&["sitting", "kitten"]), unified, &cpu)?;
let mut distances = Vec::new_in(unified);
distances.resize(2, 0usize);

let mut engine = LevenshteinEngine::new(&["kitten"], LevenshteinSymbol::Bytes, &stream)?;
stream.scope(|scope| engine.distances(scope, candidates.view(), None, &mut distances, 2))?;
assert_eq!(distances[..], [3, 0]);

stream.synchronize() joins a stream outside any scope. A host engine imposes no such requirement: plain Vec and stack buffers are exactly what its verbs expect, and a column in any memory reads there too.