# StringZilla for C++ > SIMD and GPU-accelerated strings, from C++ It bundles: - a precompiled __static__ or __shared__ library that compiles every capability once and picks one per call, for maximum portability, - the C headers ``, which declare that library, or compile the kernels inline with `STRINGZILLA_HEADER_ONLY=1`, - the C++ headers ``, built on the library. The plain C ABI exposes each kernel family as a stable C 99 surface: substring and byte-set search, non-cryptographic hashing and checksums, lexicographic comparison, sorting and intersection of string collections, collection conversions from escaping and normalization to byte translation, and the stateful cross-product engines for edit distances, window overlap, and multi-pattern search. The thin C++ binding rebuilds the STL `` and `` surface on top of those kernels, adding an owning Small-String-Optimized container, allocation-free splitting and partitioning views, and free functions for hashing, sorting, and translation. Every verb has a dispatch point, like `sz_find_best`, which runs the best capability a mask shares with the verb's list, and the mask to pass on the CPU is the one `sz_capabilities_enabled_cpu` reports. There is no hidden global allocation and no thread pool: every function that may allocate takes an explicit `sz_allocator_t *`, and an engine holds the blocks its allocator handed it until you free it. An engine is prepared by `sz_levenshtein_engine_init`, and its twins for overlap and substrings, for one device's capabilities and a stream, which names the device, so choosing a device is choosing the mask and the stream an engine is built with rather than setting a global. Its rounds, such as `sz_levenshtein_distances`, take a trailing stream: null on the CPU, while on a GPU they enqueue there and return, leaving the join to the caller. A GPU engine's init uses its stream for its own work alone and sizes a round's device memory from the budgets it takes, so no compute verb allocates or joins, and a round carrying more haystacks than its budget, or on Metal more candidates, is refused with `sz_unexpected_dimensions_k`. Overlap and Levenshtein engines keep no per-round state on a CUDA device, so several streams may score one engine at once, while the rounds of one substrings engine share its arena and its report, so the caller orders them. The headers compile as freestanding C 99 — set `STRINGZILLA_WITH_LIBC=0` to drop the libc dependency — and as C++20 or newer for the `sz::` layer. Per-family hubs `find.h`, `hash.h`, `sort.h`, `compare.h`, `intersect.h`, `strings.h`, `small_string.h`, `levenshtein.h`, `overlap.h`, and `substrings.h` declare the family's dispatch points and its kernels, each capability's under its `STRINGZILLA_TARGET_*` macro, from the `c/` header root. The kernels are defined under the matching subdirectories of the `include/` root — `find/haswell.h`, `hash/icelake.h`, `strings/neon.h`, and so on — and the C++ headers live in the `cpp/` root; a build passes all three as include paths. The library compiles the kernels once, one capability per unit in `src/capabilities/`, and header-only builds include them into every translation unit. ## Installation The headers declare the StringZilla library, so the simplest integration links it and includes the umbrella header. Its dispatch points pick the best capability at runtime, so one binary runs optimally on any CPU. ```c #include // C ``` ```cpp #include // C++ namespace sz = ashvardanian::stringzilla; ``` Each family can also be included on its own when you only need a slice of the API and want to keep compile times down: ```c #include // `sz_find_best`, `sz_rfind_best`, byte-set scans #include // `sz_hash_best`, `sz_bytesum_best`, incremental state #include // `sz_sort_order_best` #include // `sz_equal_best`, `sz_order_best` #include // `sz_intersect_best` #include // `sz_strings_to_tape_best`, `sz_strings_lookup_best` #include // `sz_string_t` SSO container #include // `sz_levenshtein_engine_init`, `sz_levenshtein_distances` #include // `sz_overlap_engine_init`, `sz_overlap_scores` #include // `sz_substrings_engine_init`, `sz_substrings_counts` ``` The three engine families prepare a batch of queries once and score it against many batches of candidates, writing into a strided block the caller supplies rather than allocating one per round. Each init takes the mask, the allocator and a stream last, and each round is a single verb with no `_best` twin, as the engine already holds the capability its init picked: ```c sz_status_t sz_levenshtein_engine_init(sz_levenshtein_engine_t *engine, sz_strings_view_t const *queries, sz_levenshtein_symbol_t symbol, sz_capability_t capabilities, sz_allocator_t *allocator, sz_stream_t stream); sz_status_t sz_levenshtein_distances(sz_levenshtein_engine_t *engine, sz_strings_view_t const *candidates, sz_size_t const *symbol_counts, sz_size_t const *order, sz_size_t bound, sz_size_t *distances, sz_size_t distances_stride, sz_stream_t stream); void sz_levenshtein_engine_free(sz_levenshtein_engine_t *engine, sz_stream_t stream); ``` Their per-tier building blocks and their design notes live beside the kernels, in [`levenshtein/README.md`](https://github.com/ashvardanian/StringZilla/blob/main/include/stringzilla/levenshtein/README.md), [`overlap/README.md`](https://github.com/ashvardanian/StringZilla/blob/main/include/stringzilla/overlap/README.md), and [`substrings/README.md`](https://github.com/ashvardanian/StringZilla/blob/main/include/stringzilla/substrings/README.md). ### CMake, Precompiled Library With CMake 3.14 or newer, pull the project in with `FetchContent` and link the static target `stringzilla::static`, which compiles every capability the toolchain can build. ```cmake include(FetchContent) FetchContent_Declare( stringzilla GIT_REPOSITORY https://github.com/ashvardanian/StringZilla.git GIT_TAG main) FetchContent_MakeAvailable(stringzilla) target_link_libraries(your_app PRIVATE stringzilla::static) ``` `add_subdirectory(stringzilla)` works the same way. Pulled in as a subproject, the static library is compiled only once something links it, and `STRINGZILLA_BUILD_SHARED` defaults to off. Turn that on for the shared target `stringzilla::shared`: ```cmake set(STRINGZILLA_BUILD_SHARED ON) FetchContent_MakeAvailable(stringzilla) target_link_libraries(your_app PRIVATE stringzilla::shared) ``` Either library's dispatch points pick a capability by the mask each call passes, so one artifact runs optimally on any CPU. ### CMake, Header Only C code that wants no compiled dependency links the interface target `stringzilla::header`, which adds `include/` to your search path and defines `STRINGZILLA_HEADER_ONLY=1`: ```cmake target_link_libraries(your_app PRIVATE stringzilla::header) ``` The kernels your compiler flags enable then compile into your translation units, and you call them by name, like `sz_find_haswell`. Every dispatch point and finder reports `sz_missing_library_k` there, so the C++ layer runs the serial kernels over its default mask, which is empty in these builds. ### CMake, Installed Package Configure with `-DSTRINGZILLA_INSTALL=ON` to install the headers and libraries system-wide, then find them from an unrelated project: ```cmake find_package(stringzilla REQUIRED) target_link_libraries(your_app PRIVATE stringzilla::shared) ``` `stringzilla::shared` joins the package whenever `STRINGZILLA_BUILD_SHARED` was on for the install. `stringzilla::header` always does, as it needs no submodules and compiles nothing, so it works straight out of a release tarball. A shared-library install also writes `lib/pkgconfig/stringzilla.pc`, for build systems that read pkg-config rather than CMake: ```sh cc main.c $(pkg-config --cflags --libs stringzilla) -o main ``` ### Build Knobs The behavior of the core is controlled entirely by preprocessor macros, so you can tune it without touching the sources. The most important ones, mirrored from `stringzilla.h`: | Macro | Default | Effect | | :----------------------------------- | -------: | :--------------------------------------- | | `STRINGZILLA_HEADER_ONLY` | `0` | Inline kernels instead of the library | | `STRINGZILLA_DEBUG` | `0` | Debug assertions and logging | | `STRINGZILLA_WITH_LIBC` | `1` | `0` builds freestanding, without libc | | `STRINGZILLA_ALLOW_MISALIGNED_LOADS` | platform | Unaligned word loads in SWAR fallbacks | | `STRINGZILLA_SWAR_THRESHOLD` | `24` | Length below which scalar loops are used | | `STRINGZILLA_DEFAULT_ALIGNMENT` | platform | Default alignment and padding policy | `STRINGZILLA_HEADER_ONLY` chooses between the two builds: with `0`, the default, the dispatch points live in the library, which compiles every capability and picks one by the mask each call passes; with `1` the kernels your compiler flags enable inline into your translation unit, and the dispatch points report `sz_missing_library_k`. `STRINGZILLA_WITH_LIBC=0` builds freestanding without the C standard library, which disables the default `malloc`-based allocator and the `offsetof` static checks. `STRINGZILLA_ALLOW_MISALIGNED_LOADS` allows unaligned word loads in the SWAR fallbacks where the platform permits. Per-ISA backends are toggled with their own macros; if left undefined they are auto-detected from the compiler's target flags: `STRINGZILLA_TARGET_WESTMERE` for SSE4.2 + AES-NI, `STRINGZILLA_TARGET_GOLDMONT` for SHA-NI, `STRINGZILLA_TARGET_HASWELL` for AVX2, `STRINGZILLA_TARGET_SKYLAKE` for AVX-512 F/BW/VL, `STRINGZILLA_TARGET_ICELAKE` for AVX-512 VBMI + VAES, then `STRINGZILLA_TARGET_NEON`, `STRINGZILLA_TARGET_NEONAES`, `STRINGZILLA_TARGET_NEONSHA`, `STRINGZILLA_TARGET_SVE`, `STRINGZILLA_TARGET_SVE2`, and `STRINGZILLA_TARGET_SVE2AES` on ARM, `STRINGZILLA_TARGET_V128` and `STRINGZILLA_TARGET_V128RELAXED` for WebAssembly SIMD128, `STRINGZILLA_TARGET_RVV` and `STRINGZILLA_TARGET_RVVCRYPTO` for the RISC-V Vector extension, `STRINGZILLA_TARGET_LOONGSONASX` for LoongArch, and `STRINGZILLA_TARGET_POWERVSX` for IBM Power. The umbrella header also exposes the version triple as `STRINGZILLA_VERSION_MAJOR`/`_MINOR`/`_PATCH`, with matching `sz_version_major()`, `sz_version_minor()`, and `sz_version_patch()` accessors. ### Compilers and Platforms The headers require only C 99 and C++ 20, so any reasonably modern compiler builds the serial and SWAR baseline. The per-ISA SIMD kernels and the project's own tests are CI-validated with these toolchains: | Toolchain | Recommended Versions | | :-------- | :------------------- | | GCC | 13 or newer | | Clang | 18 or newer | | MSVC | Visual Studio 2022 | | NVCC | CUDA 13+ | The RVV kernels need Clang 21 or newer, as older Clang hides the 64-bit vector types from a unit compiled without `v`, and an older compiler builds the libraries without them. On macOS, prefer Homebrew Clang over Apple Clang; on Windows, MinGW with GCC works alongside MSVC. NVCC with CUDA 13 builds the device backends of the engine families, reached by building an engine with the mask `sz_capabilities_enabled_cuda` reports for a device. HIP-Clang builds the same sources for AMD GPUs, reached through `sz_capabilities_enabled_rocm`. Built with `STRINGZILLA_METAL_ARCH`, the three engines also run on Apple GPUs of the Apple7 family and newer, through the same constructors and verbs. A device round reads memory the device reaches, and strings only as a tape, never through an accessor, which is host code. `strings.h` declares allocation and gathering into tapes; `capabilities.h` declares stream creation, synchronization and release. Their `_best` dispatch points take a mask, with `_serial`, `_cuda`, `_rocm` and `_metal` twins, the GPU ones exported only where built: - `sz_allocator_init_unified_best` fills a stateless `sz_allocator_t` whose blocks the host and the device share: the heap on the CPU, managed memory on CUDA and ROCm, shared buffers on Metal, each made on the device of the stream it is called with. - `sz_strings_to_tape_best` gathers strings of any layout into a `sz_strings_span_t` tape of payload and offsets, two blocks from the allocator it is given, and `sz_strings_span_free` returns both. - `sz_stream_synchronize_best` waits for one stream, after which what a round wrote is readable from the host. ```c sz_capabilities_enabled_cuda(0, &caps); sz_capabilities_enabled_cpu(&cpu_caps); sz_allocator_init_unified_best(&unified, caps); sz_strings_span_t tape = {0}; sz_strings_to_tape_best(&host_candidates, NULL, host_candidates.count, &unified, &tape, cpu_caps, NULL); sz_strings_view_t candidates; sz_strings_from_span(&tape, &candidates); sz_levenshtein_engine_init(&engine, &queries, sz_levenshtein_bytes_k, caps, NULL, stream); sz_levenshtein_distances(&engine, &candidates, NULL, NULL, STRINGZILLA_SIZE_MAX, distances, candidates.count, stream); sz_stream_synchronize_best(caps, stream); sz_levenshtein_engine_free(&engine, stream); sz_strings_span_free(&tape, &unified, stream); ``` A null stream is the default stream of the default device, and a vendor's devices are only numbered by its producers, like `sz_capabilities_enabled_cuda(ordinal, &caps)`. StringZilla also __compiles to WebAssembly__: the `wasm32` toolchain targets `wasm32-wasip1`, and `STRINGZILLA_MARCH` names the module's one SIMD capability, `serial`, `v128` or the default `v128relaxed`, which enables the `STRINGZILLA_TARGET_V128` and `STRINGZILLA_TARGET_V128RELAXED` kernels listed above. ## Types The C++ layer lives in `namespace ashvardanian::stringzilla`, conventionally aliased as `sz`, and is built on the types below plus a 256-bit `byteset`. - `sz::string_slice` — a non-owning, length-bounded view over `char` or `char const`. Two aliases exist: the read-only `sz::string_view_t = string_slice` and the mutable `sz::string_span_t = string_slice`. - `sz::string>` — an owning, Small-String-Optimized `char` container; the alias `sz::string_t` uses the default `std::allocator`. A stateful allocator travels with the string under the usual `std::allocator_traits` propagation rules, and an empty one costs no bytes. - `sz::strings_view_t` — a borrowed column of strings, wrapping a `sz_strings_view_t` in any of its layouts. - `sz::strings>` — an owned, move-only column, a `sz_strings_span_t` tape and the allocator of its blocks; the alias `sz::strings_t` uses the default. - `sz::byteset_t` — the C++ wrapper around `sz_byteset_t`, constructible from an initializer list, a `(pointer, count)` pair, or a `std::array`, and composable with `operator|`. - `sz::status_t` — a scoped enum mirroring `sz_status_t` with `success_k`, `bad_alloc_k`, `invalid_utf8_k`, `contains_duplicates_k`, and the rest, returned by the fallible free functions and the string's allocating verbs. `sz::string_view_t` is implicitly constructible from a `char const *`, a `(pointer, length)` pair, `std::string`, and `std::string_view`, and converts back to both; `sz::string_t` adds construction from repeats, sub-ranges, initializer lists, and lazy `sz::concatenate(...)` expressions. Both satisfy the `std::string`/`std::string_view` member surface — `data`, `size`, `length`, `empty`, `at`, `front`, `back`, iterators, `substr`, `compare`, the relational operators, and on C++20 `operator<=>` — so they are drop-in for most code. Both also stream through `operator<<`, and `std::hash` / `std::hash` specializations are provided so they work as unordered-container keys out of the box. User-defined literals live in `sz::literals`: ```cpp using namespace sz::literals; sz::string_view_t v = "needle"_sv; // a view, no allocation sz::byteset_t s = " \t\n\r"_bs; // a byte-set from a literal ``` Convenience byte-sets are available as free functions: `sz::whitespaces_set()`, `sz::newlines_set()`, `sz::digits_set()`, `sz::hexdigits_set()`, `sz::octdigits_set()`, and `sz::ascii_letters_set()`. ## Searching and Counting The view and the owning string expose `find`/`rfind` for sub-strings, single characters, and byte-sets, plus `contains`, `starts_with`, `ends_with`, and the full `find_first_of` / `find_last_of` / `find_first_not_of` / `find_last_not_of` family taking either a `byteset` or a view. For repeated matching the view offers lazy, allocation-free ranges: `find_all(needle)` and `rfind_all(needle)`, each with `sz::include_overlaps` / `sz::exclude_overlaps` tag overloads, alongside `find_all(byteset)` and `rfind_all(byteset)`. An empty needle matches at every offset from 0 to `size()` inclusive, so it yields `size() + 1` zero-width matches. ```cpp #include namespace sz = ashvardanian::stringzilla; int main() { sz::string_view_t haystack = "the quick brown fox"; auto pos = haystack.find("quick"); // index, or sz::string_view_t::npos assert(pos == 4); assert(haystack.contains("brown")); assert(haystack.starts_with("the")); assert(haystack.ends_with("fox")); std::size_t words = 0; for (auto word : haystack.find_all(sz::whitespaces_set())) (void)word, ++words; auto vowels = sz::byteset_t {"aeiou", 5}; auto first_vowel = haystack.find_first_of(vowels); assert(first_vowel == 2); // index of 'e' return 0; } ``` The view also offers character-class predicates — `is_alpha`, `is_alnum`, `is_ascii`, `is_digit`, `is_lower`, `is_upper`, `is_space`, `is_printable`, and the general `contains_only(byteset)`. ## Splitting and Partitioning This is a C++-only convenience layer; the C ABI provides the underlying scans `sz_find_best` and `sz_find_byteset_best` that you compose by hand. `partition` and `rpartition` return a three-way `sz::string_partition_result` struct with `before`, `match`, and `after` members around the first or last occurrence of a pattern. The pattern can be a view, a single character, or a byte-set. `split`, `rsplit`, and `splitlines` return lazy ranges of `string_view`s. `split(view)` yields a `sz::find_splits_view`, `rsplit(view)` yields a `sz::rfind_splits_view`, and the byte-set overloads `split(byteset)` and `rsplit(byteset)` yield the matching character-splitter views. `split(byteset)` and `rsplit(byteset)` default to whitespace, and `splitlines()` defaults to the newline set. A trailing separator yields a final empty segment, and an empty separator never splits, yielding the whole text as the only segment. The key property is that these ranges are _lazy_ — they borrow from the source buffer and walk it one `string_view` at a time, so the next boundary is computed only when you ask for it. Splitting a large document into millions of tokens therefore costs __no per-token allocation__: each iteration step advances a cursor and hands back a borrowed view. This is the decisive contrast with the idiomatic STL approach, where splitting builds a `std::vector` and heap-allocates a fresh string for every element, copying its bytes out of the source. ```cpp #include namespace sz = ashvardanian::stringzilla; int main() { sz::string_view_t csv = "alice,30,engineer"; sz::string_partition_result first = csv.partition(','); sz::string_view_t name = first.before; assert(name == "alice"); assert(first.match == ","); assert(first.after == "30,engineer"); std::size_t fields = 0; for (sz::string_view_t field : csv.split(",")) // "alice", "30", "engineer" ++fields; // no string is allocated per field assert(fields == 3); sz::string_view_t doc = "line one\nline two\nline three"; for (sz::string_view_t line : doc.splitlines()) (void)line; return 0; } ``` Each range exposes `begin`/`end`, dereferences to a `sz::string_view_t`, and can be consumed in a single pass. When you genuinely need the tokens to outlive the source you can still materialize them into a `std::vector`, but only then do you pay for storage — the default path stays allocation-free. ## Trimming and Translating Trimming is exposed in C++ on the view: `strip(byteset)`, `lstrip(byteset)`, and `rstrip(byteset)` return a narrowed view with the matching leading/trailing bytes removed — no copy is made. ```cpp #include namespace sz = ashvardanian::stringzilla; int main() { sz::string_view_t padded = " \t spaced out \n"; sz::string_view_t tight = padded.strip(sz::whitespaces_set()); assert(tight == "spaced out"); return 0; } ``` Translation — mapping every byte through a 256-entry lookup table — is offered both at the C ABI level through `sz_strings_lookup_best`, documented under [String Collections and Containers](#string-collections-and-containers), and in C++ through `sz::lookup`: ```cpp #include namespace sz = ashvardanian::stringzilla; int main() { sz::look_up_table_t to_upper = sz::look_up_table_t::identity(); for (int i = 'a'; i <= 'z'; ++i) to_upper[(char)i] = (char)(i - 32); sz::string_t text; if (sz::failed(text.assign("hello"))) return 1; text.lookup(to_upper); // in place assert(text == "HELLO"); return 0; } ``` The C side ships ready-made table initializers — `sz_strings_lookup_init_lower`, `sz_strings_lookup_init_upper`, and `sz_strings_lookup_init_ascii`. To select an ASCII-only fast path, scan for the complement of `sz_byteset_init_ascii` with `sz_find_byteset_best`, which reaches the SIMD byteset kernels instead of a scalar loop. ## Unicode Segmentation Splitting Unicode text into the units a human actually perceives is deceptively hard, and getting it wrong corrupts data. A single emoji can be dozens of bytes and several codepoints yet one perceived character, CJK runs carry no spaces, and a period is not always the end of a sentence. StringZilla ships the real algorithms — UAX-29 graphemes, words, and sentences, plus UAX-14 line-break opportunities — as lazy ranges that borrow from the source and yield one `sz::string_view_t` per segment with no per-segment allocation. A string's length is really three different numbers, and only the grapheme count matches what a user sees: ```cpp #include #include namespace sz = ashvardanian::stringzilla; int main() { sz::string_view_t family = "👨‍👩‍👧‍👦"; // one emoji, one perceived character std::size_t codepoints = 0; for (sz_rune_t r : family.utf8_runes()) (void)r, ++codepoints; std::size_t graphemes = 0; for (sz::string_view_t g : family.utf8_graphemes()) (void)g, ++graphemes; assert(family.size() == 25); // 25 bytes on the wire assert(codepoints == 7); // four people joined by three zero-width joiners assert(graphemes == 1); // one grapheme cluster return 0; } ``` This is exactly why truncating by bytes or codepoints tears emoji apart — it can slice a skin-tone modifier off its base, or split a flag in half. Iterating graphemes lets you cut on a real boundary: ```cpp sz::string_view_t msg = "🙏🏽👍🏿✨ thanks"; std::size_t cut = 0, seen = 0; for (sz::string_view_t g : msg.utf8_graphemes()) { if (seen++ == 3) break; cut += g.size(); } assert(msg.sub(0, cut) == "🙏🏽👍🏿✨"); // the three emoji keep their skin-tone modifiers ``` Word and sentence boundaries are just as treacherous: contractions, CJK, decimals, and URLs all defeat naive splitting. UAX-29 keeps `can't` whole while separating CJK ideographs that no space divides, and refuses to break a sentence inside `$9.99`: ```cpp sz::string_view_t phrase = "can't stop 北京!"; assert(*phrase.utf8_wordbreaks().begin() == "can't"); // apostrophe is interior; 北 and 京 split apart sz::string_view_t prose = "She paid $9.99. Cheap."; std::size_t sentences = 0; for (sz::string_view_t s : prose.utf8_sentences()) (void)s, ++sentences; assert(sentences == 2); // the dot inside $9.99 is not a sentence break ``` The full family of ranges, each borrowing from the source and yielding `sz::string_view_t` segments, or `sz_rune_t` for runes. Naming follows one rule: the bare name (`newlines`/`whitespaces`/`delimiters`) yields the __separators__ the kernel finds, while `split_*` yields the content __between__ them: - `utf8_runes()` — every codepoint as a decoded `sz_rune_t` UTF-32 scalar, with U+FFFD for ill-formed bytes and a truncated tail. - `utf8_graphemes()` — UAX-29 grapheme clusters, the user-perceived characters. - `utf8_wordbreaks()` — all UAX-29 word segments (words and the separators between them). - `utf8_sentences()` — UAX-29 sentence boundaries. - `utf8_linebreaks()` — UAX-14 line-break opportunities, both hard breaks and soft wrap points. - `utf8_split_newlines()` / `utf8_newlines()` — content between the seven Unicode newlines (CRLF as one), or the newline runs themselves. - `utf8_split_whitespaces()` / `utf8_whitespaces()` — content between Unicode "White_Space" runs (tokens), or the whitespace runs. - `utf8_split_delimiters()` / `utf8_delimiters()` — content between any Unicode punctuation/symbol/separator, or the delimiter runs. On any `split_*` range, `.skip_empty()` drops empty segments and `.with_separators()` interleaves segments and separators losslessly (concatenation reconstructs the input). The `utf8_wordbreaks`, `utf8_graphemes`, `utf8_sentences`, and `utf8_linebreaks` ranges _tile_ the input — every byte belongs to exactly one segment, with no gaps and no empty slices. Because every range borrows and walks the buffer once, segmenting a multi-megabyte document into graphemes or words allocates nothing. In C, `sz_utf8_graphemes_best` and its three tiling siblings write one length per segment, each segment starting where the previous one ended, so after a full batch the caller resumes at the sum of its lengths. The newline, whitespace and delimiter finders write offsets and lengths of their matches, and after a full batch `bytes_consumed` ends exactly at the last match; otherwise it is the whole input. `sz_utf8_decode_best` treats the end of the text as the end of the input, decoding a truncated final sequence to one U+FFFD, so a caller feeding it chunks splits them at a rune boundary. ## Case Folding and Normalization Case-insensitive matching across full Unicode is more than ASCII `tolower` — `ß` matches `ss`, accents fold, and the matched byte-length can differ from the needle's. `utf8_uncased_search` searches case-insensitively _without pre-folding the haystack_: it folds on the fly, so you never allocate a folded copy of a large document just to search it. ```cpp #include #include namespace sz = ashvardanian::stringzilla; int main() { sz::string_view_t hay = "Take the STRAßE downtown"; auto m = hay.utf8_uncased_search("strasse"); // no folded copy of `hay` is made assert(m.offset == 9); assert(hay.sub(m.offset, m.offset + m.length) == "STRAßE"); // matched the mixed-case ß form return 0; } ``` The result carries both the `offset` and the matched `length`, since folding can make the matched span longer or shorter than the needle. To search many haystacks for one needle, prepare it once as a `sz::utf8_uncased_needle_t`, which borrows the needle's bytes, and pass it to `utf8_uncased_search` or to `utf8_uncased_matches`, the lazy range of every non-overlapping match. Matches sit on codepoint boundaries, so an empty needle matches at every boundary, the end included. When you need the folded text itself, fold a `sz::string_t` in place: ```cpp sz::string_t greeting; if (sz::failed(greeting.assign("Grüße"))) return 1; if (sz::failed(greeting.utf8_uncased_fold())) return 1; // in place; `ß` expands to `ss` assert(greeting == "grüsse"); ``` In C, `sz_utf8_uncased_needle_init_best` analyses the needle once into a `sz_utf8_uncased_needle_t`, and `sz_utf8_uncased_search_best` only reads it, so any number of searches, under any capabilities, share one analysis. The prepared needle points at the needle's bytes rather than copying them, so they must outlive it: ```c #include sz_utf8_uncased_needle_t needle; sz_utf8_uncased_needle_init_best("café", 5, &needle, capabilities, NULL); sz_cptr_t hit = NULL; sz_size_t match_length = 0; sz_utf8_uncased_search_best(haystack, haystack_length, &needle, &hit, &match_length, capabilities, NULL); // `hit` points at the first case-insensitive match, or NULL; reuse `needle` for the next haystack. ``` An empty needle matches at the start of every haystack, with a zero `match_length`. ## Hashing and Checksums In C++, hashing is exposed as methods — `view.hash(seed = 0)`, `view.bytesum()`, and `view.hash_multiseed(seeds, hashes)` — and as the function object `sz::hash{}` together with `sz::equal_to{}`, which are convenient as template arguments for `std::unordered_map`/`std::unordered_set`. `sz::fill_random(slice, nonce)` fills a mutable span with reproducible noise. ```cpp #include #include #include #include namespace sz = ashvardanian::stringzilla; int main() { // Drop the SIMD hash and equality into the STL containers as template arguments. std::unordered_map counts; for (char const *word : {"apple", "banana", "apple"}) counts[word] += 1; assert(counts["apple"] == 2); assert(counts.size() == 2); // `sz::string_t` keys also work in the plain containers, via the bundled `std::hash` specialization. std::unordered_set uniq = {"apple", "banana"}; assert(uniq.count("apple") == 1); sz::string_t noise(16, '\0'); sz::fill_random(noise.span(), /*nonce*/ 7); // reproducible noise assert(noise.size() == 16); return 0; } ``` ## Sorting and Sets These operate on a read-only `sz_strings_view_t` in any layout and never reorder the underlying data — they fill an `order` permutation array instead. Sorting is __stable__, so equal elements keep their input order, supports descending order via a `reverse` flag, and supports partial / top-K sorting via `top_count` — pass `0` to sort everything. Every allocation these kernels need is routed through the `sz_allocator_t` you supply — pass `NULL` for the default `malloc`-based one — so, unlike `std::sort` or `thrust::sort`, no scratch memory is ever allocated behind your back. Matching the all-`noexcept` surface, the C++ sort takes any container plus a string-extractor callable and writes the permutation into a caller-owned span, returning a `status_t`: ```cpp status_t sz::argsort(container, extractor, span order, top_count = 0, reverse = false, capabilities = sz::default_capabilities(), stream = nullptr, allocator = {}); status_t sz::argsort_utf8_uncased(container, extractor, span order, top_count = 0, reverse = false, capabilities = sz::default_capabilities(), stream = nullptr, allocator = {}); expected sz::intersect(first, extractor, second, extractor, seed, span first_positions, span second_positions, capabilities = sz::default_capabilities(), stream = nullptr, allocator = {}); ``` The outputs are sized `std::span` views over caller-owned storage, and the scratch memory comes from `allocator`. Size `order` to `container.size()` — the whole span receives a permutation, and a non-zero `top_count` only requests that the first `top_count` entries be fully sorted. Size each position span to `min(first.size(), second.size())`; a shorter span returns `unexpected_dimensions_k`. `intersect` returns the number of matched pairs as an `expected`, so there is no separate count out-parameter. ```cpp #include #include #include namespace sz = ashvardanian::stringzilla; int main() { std::vector names = {"banana", "apple", "cherry"}; sz_size_t order[3]; sz::status_t status = sz::argsort( names, [](sz::string_t const &s) -> sz::string_view_t { return s; }, {order, 3}); assert(status == sz::status_t::success_k); assert(order[0] == 1 && order[1] == 0 && order[2] == 2); // apple, banana, cherry auto [sorted, sort_status] = sz::argsort(names); assert(sz::succeeded(sort_status) && sorted[0] == 1); return 0; } ``` Without output spans, `sz::argsort`, `sz::argsort_utf8_uncased` and `sz::intersect` allocate their results from the allocator passed last and return them in an `expected`, never throwing. ### String Collections and Containers In C++, `sz::string_t` is the owning, SSO container and `sz::string_span_t` is its mutable view; both rebuild the `std::string` surface, and nothing in them throws. Mutating members (`append`, `push_back`, `insert`, `erase`, `replace`, `resize`, `reserve`, `shrink_to_fit`, `clear`, `swap`, `assign`) are present, alongside `data`/`c_str`/`size`/`capacity` and the relational operators. Every allocating verb returns a `sz::status_t`, or an `sz::expected` beside an iterator, and `insert_at`, `erase_slice` and `replace_slice` take signed offsets counted from the end, like Python's slices. Copies and conversions come from `copy` and `make`, which return an `sz::expected`, and positions past the end clamp to it. ```cpp #include namespace sz = ashvardanian::stringzilla; int main() { sz::expected made = sz::string_t::make("hello"); if (!made) return 1; sz::string_t greeting = std::move(made.value); if (sz::failed(greeting.append(", world"))) return 1; assert(greeting == "hello, world"); if (sz::failed(greeting.push_back('!'))) return 1; // allocating verbs return a status if (sz::failed(greeting.reserve(1024))) return 1; if (sz::failed(greeting.replace_all("l", "L"))) return 1; assert(greeting.starts_with("heLLo")); return 0; } ``` Collections pair the same way: `sz::strings_view_t` borrows an Arrow `String` or `LargeString` array from `(data, offsets, count)`, a span of `sz::string_view_t`, or an owned column, and `sz::strings_t` owns one. Its conversions, `gather`, `escape`, `unescape`, `lookup` and `normalize`, return a fresh `sz::strings` inside an `sz::expected` with the status, and leave nothing allocated on failure. `find`, `hashes` and `to_rows` write into spans the caller owns. Every one takes a capability mask, `sz::default_capabilities()` unless given, and a stream, like the C dispatch points. ```cpp #include namespace sz = ashvardanian::stringzilla; int main() { char const payload[] = "alphabetagamma"; sz_u32_t const offsets[] = {0, 5, 9, 14}; sz::strings_view_t const column(payload, offsets, 3); // an Arrow `String` array, no copies auto const [escaped, escape_status] = column.escape(sz_escape_json_k); if (sz::failed(escape_status)) return 1; std::size_t malformed_index = 0; auto const [unescaped, unescape_status] = escaped.view().unescape(sz_escape_json_k, malformed_index); assert(sz::succeeded(unescape_status) && unescaped[1] == "beta"); std::size_t positions[3]; if (sz::failed(column.find("a", positions))) return 1; // the first match in every string return 0; } ``` ## Memory Operations ### Memory Ownership and Small String Optimization Most operations in StringZilla don't assume any memory ownership. But in addition to the read-only search-like operations StringZilla provides a minimalistic C and C++ implementations for a memory owning string "class". Like other efficient string implementations, it uses the [Small String Optimization][faq-sso] (SSO) to avoid heap allocations for short strings. The owning `sz_string_t` container, its union layout, and the full lifecycle API are documented under [String Collections and Containers](#string-collections-and-containers); this section covers the design rationale behind that layout. [faq-sso]: https://cpp-optimizations.netlify.app/small_strings/ A short string can be kept on the stack, if it fits within the inline character array. Before 2015 GCC string implementation was just 8 bytes, and could only fit 7 characters. Different STL implementations today have different thresholds for the Small String Optimization. Similar to GCC, StringZilla is 32 bytes in size, and similar to Clang it can fit 22 characters on stack. Our layout might be preferential, if you want to avoid branches. If you use a different compiler, you may want to check its SSO buffer size with a [simple Gist](https://gist.github.com/ashvardanian/c197f15732d9855c4e070797adf17b21). | | `libstdc++` in GCC 13 | `libc++` in Clang 17 | StringZilla | | :-------------- | --------------------: | -------------------: | ----------: | | String `sizeof` | 32 | 24 | 32 | | Inner Capacity | 15 | __22__ | __22__ | This design has been since ported to many high-level programming languages. Swift, for example, [can store 15 bytes](https://developer.apple.com/documentation/swift/substring/withutf8(_:)#discussion) in the `String` instance itself. StringZilla implements SSO at the C level, providing the `sz_string_t` union and a simple API for primary operations, covered with a worked example under [String Collections and Containers](#string-collections-and-containers). Unlike the conventional C strings, the `sz_string_t` is allowed to contain null characters. To safely print those, pass the unpacked length to `printf` as well. ```c sz_ptr_t string_start; sz_size_t string_length; sz_string_range(&string, &string_start, &string_length); printf("%.*s\n", (int)string_length, string_start); ``` ## Runtime Dispatch Every kernel exists in a serial form plus one or more per-ISA forms, named with a backend suffix (`_serial`, `_westmere`, `_goldmont`, `_haswell`, `_skylake`, `_icelake`, `_neon`, `_neonaes`, `_neonsha`, `_sve`, `_sve2`, `_sve2aes`, `_v128`, `_v128relaxed`, `_rvv`, `_rvvcrypto`, `_loongsonasx`, `_powervsx`), and the engines add one per GPU vendor, `_cuda`, `_rocm` and `_metal`. Each verb also has a dispatch point, like `sz_find_best` or `sz_hash_best`, taking the kernels' arguments plus a capability mask before the trailing stream, and running the best capability the mask shares with the verb's list, `serial` first on the CPU. It returns `sz_missing_kernel_k` when no capability in the mask has the kernel, as a GPU-only mask does for a CPU-only verb. - __Library builds, `STRINGZILLA_HEADER_ONLY=0`, the default.__ Every capability the toolchain builds is compiled into one binary, and the dispatch points pick among them by the mask each call passes. This is how the prebuilt libraries ship, so a single binary runs optimally on any CPU. - __Header-only builds, `STRINGZILLA_HEADER_ONLY=1`.__ The kernels your compiler flags enable inline into your translation unit, so calling one of them by name, like `sz_find_haswell`, costs no indirection at all. The dispatch points and finders compile to stubs returning `sz_missing_library_k`, while the C++ layer runs the serial kernels over its default mask, which is empty here. Capabilities are introspectable at both compile and run time. `sz_capability_t` is a 64-bit mask from `capabilities.h`, one bit per capability. Within each architecture the bits ascend by preference, so the highest bit a mask shares with a verb's kernels names the kernel to run. - __Serial__: `sz_cap_serial_k`, bit 0, which every CPU runs. - __x86__: `sz_cap_westmere_k`, `sz_cap_goldmont_k`, `sz_cap_haswell_k`, `sz_cap_skylake_k`, `sz_cap_icelake_k`. - __Arm__: `sz_cap_neon_k`, `sz_cap_neonaes_k`, `sz_cap_neonsha_k`, `sz_cap_sve_k`, `sz_cap_sve2_k`, `sz_cap_sve2aes_k`. - __RISC-V__: `sz_cap_rvv_k`, `sz_cap_rvvcrypto_k`. - __WebAssembly__: `sz_cap_v128_k`, `sz_cap_v128relaxed_k`. - __LoongArch and Power__: `sz_cap_loongsonasx_k`, `sz_cap_powervsx_k`. - __GPUs__: one baseline per vendor above every CPU bit, `sz_cap_cuda_k` at bit 48, `sz_cap_rocm_k` at 56, and `sz_cap_metal_k` at 60. CUDA generations follow its baseline: `sz_cap_hopper_k` at 51 for clusters from compute capability 9.0, and `sz_cap_blackwell_k` at 52 for cluster launch control from 10.0. `sz_cap_cpus_k` and `sz_cap_gpus_k` group the CPU and the GPU bits, and `sz_cap_any_k` sets every bit. ```c sz_status_t sz_capabilities_detected_cpu(sz_capability_t *capabilities); // what this CPU runs sz_status_t sz_capabilities_compiled_cpu(sz_capability_t *capabilities); // what this build holds kernels for sz_capability_t sz_capabilities_compiled_cpu_mask(void); // the same, constant in the calling unit sz_status_t sz_capabilities_enabled_cpu(sz_capability_t *capabilities); // both at once, always with serial sz_status_t sz_thread_configure_cpu(sz_capability_t capabilities); // once per dispatching thread sz_status_t sz_device_count_cuda(sz_size_t *count); sz_status_t sz_capabilities_detected_cuda(sz_size_t ordinal, sz_capability_t *capabilities); sz_status_t sz_capabilities_compiled_cuda(sz_capability_t *capabilities); sz_status_t sz_capabilities_enabled_cuda(sz_size_t ordinal, sz_capability_t *capabilities); sz_size_t sz_capabilities_name(sz_capability_t capabilities, char *buffer, sz_size_t capacity); // "serial,haswell" char const *sz_status_name(sz_status_t status); ``` ROCm and Metal have the same four queries, spelled `sz_rocm_*` and `sz_metal_*`. The library probes the CPU once per process and caches the answer, while header-only builds probe on every call. The GPU queries ask the vendor's runtime every time, by that runtime's own device ordinal, and report no devices where the vendor isn't built. The CPU probe inspects CPUID on x86, the AArch64 ID registers on Arm once the kernel's `HWCAP_CPUID` says it emulates `mrs`, falling back to NEON-only, `getauxval`/`riscv_hwprobe` on RISC-V, and the auxiliary-vector HWCAPs on LoongArch and Power. Detection always reports the full hardware truth, independent of which tiers a build compiled in; `sz_capabilities_enabled_cpu()` intersects it with the compile-time mask. Where the OS cannot be asked, detection reports only the capabilities the compiler's own flags guarantee, never the compiled ones, which the CPU may lack. WebAssembly is the exception with no runtime probe at all — a module carrying unsupported SIMD opcodes fails validation at instantiation, so its capabilities are fixed at compile time. Nothing is process-wide: narrowing the mask a call passes narrows the choice, so `capabilities & ~sz_cap_sve_k` skips SVE and `sz_cap_serial_k` alone runs the reference kernel. A host-only process never starts a GPU driver, as only the GPU queries and the engines built for a GPU reach one. `sz_thread_configure_cpu` prepares the calling thread for the capabilities it is given, and only those; every current capability needs nothing, so it is a no-op today, kept so bindings call it where NumKong's call theirs. On Arm, having SVE in the capability mask doesn't mean SVE kernels always win: at the common 128-bit vector length the scalable kernels for length-sensitive operations — comparisons, memory transforms, substring search, UTF-8 token scanning — are often slower than their NEON twins, while crypto-heavy operations like hashing prefer SVE2 at any width. So each such SVE kernel measures the register width on the running CPU (`svcntb`) and hands the 128-bit case to its NEON twin, and the dispatch point needs no special case. To force a specific backend regardless, narrow the mask or call its kernel directly. CMake, which every binding builds through, probes which capabilities the toolchain can __compile__: each capability's probe, `probes/.c`, calls one of its kernels, compiled header-only at the baseline flags as the library compiles it. Each capability's headers scope its kernels with a target pragma, on every platform; LASX and POWER9 need GCC 15 or newer for that, as Clang's `lasxintrin.h` and `altivec.h` stay closed without `-mlasx` and `-mcpu=power9`. Libraries compile every capability the toolchain builds and leave the rest to the mask each call passes; header-only builds enable the capabilities their own flags name. `-D STRINGZILLA_TARGET_=0` drops a capability from the libraries, and `=1` keeps one only where its probe compiles. ```c #include #include int main(void) { sz_capability_t caps; sz_capabilities_enabled_cpu(&caps); char names[STRINGZILLA_CAPABILITIES_NAME_CAPACITY]; sz_capabilities_name(caps, names, sizeof(names)); printf("StringZilla %d.%d.%d, backends: %s\n", sz_version_major(), sz_version_minor(), sz_version_patch(), names); return 0; } ``` You can also call any capability's kernel directly, with the same arguments short of the mask, when you have already established the target supports it — for example `sz_find_haswell(...)` or `sz_hash_icelake(...)` — which is handy for benchmarking and for pinning a code path in a controlled environment. Picking a capability costs about a nanosecond per call: noise for long inputs, but not for hot short-string paths, like a sort's comparator or a per-token byte search. There, call a capability's kernel directly, like `sz_order_neon` or `sz_find_byte_haswell`, or resolve one once through `sz_find_kernel_punned`, and call it many times: ```c sz_kernel_order_t order = NULL; sz_capability_t capability = 0; sz_find_kernel_punned(sz_kernel_order_k, caps, (sz_kernel_punned_t *)&order, &capability); sz_ordering_t ordering; order("apple", 5, "banana", 6, &ordering, NULL); ``` It picks exactly what the dispatch point picks for the same mask, and returns `sz_missing_kernel_k` when no capability in the mask has the kernel. Each family exports its own finder over its kinds, like `sz_compare_find_kernel` in `compare.h`, and `sz_find_kernel_punned` covers every family through one entry point. Every verb has a kind, `sz_kernel__k`, and a pointer type to cast the result to, `sz_kernel__t`, taking the dispatch point's arguments short of the mask. `sz_kernel_name` spells a kind without its `sz_kernel_` prefix and `_k` suffix, like `"find_byte"`, and `sz_kernel_named` maps such a name back, or to `sz_kernel_unknown_k`. Verbs of one shape share a type, like `sz_kernel_find_t` for both `sz_kernel_find_k` and `sz_kernel_rfind_k`, `sz_kernel_utf8_segmenter_t` for the four tiling UTF-8 segmenters, and `sz_kernel_utf8_tokenizer_t` for the newline, whitespace and delimiter finders. In C++, every wrapper that takes a mask, like `sz::lookup` or `sz::argsort`, defaults to `sz::default_capabilities()`, the enabled CPU mask, while `sz::device_t::make(kind, ordinal)` opens one GPU to ask its `capabilities_enabled()`, and `sz::device_t::cpu().configure_thread(mask)` wraps `sz_thread_configure_cpu`. ## Platform Notes ### What's Wrong with the C Standard Library? StringZilla is not a drop-in replacement for the C Standard Library. It's designed to be a safer and more modern alternative. Conceptually: 1. LibC strings are expected to be null-terminated, so to use the efficient LibC implementations on slices of larger strings, you'd have to copy them, which is more expensive than the original string operation. 2. LibC functionality is asymmetric — you can find the first and the last occurrence of a character within a string, but you can't find the last occurrence of a substring. 3. LibC function names are typically very short and cryptic. 4. LibC lacks crucial functionality like hashing and doesn't provide primitives for less critical but relevant operations like fuzzy matching. Something has to be said about its support for UTF-8. Aside from a single-byte `char` type, LibC provides `wchar_t`: - The size of `wchar_t` is not consistent across platforms. On Windows, it's typically 16 bits (suitable for UTF-16), while on Unix-like systems, it's usually 32 bits (suitable for UTF-32). This inconsistency can lead to portability issues when writing cross-platform code. - `wchar_t` is designed to represent wide characters in a fixed-width format (UTF-16 or UTF-32). In contrast, UTF-8 is a variable-length encoding, where each character can take from 1 to 4 bytes. This fundamental difference means that `wchar_t` and UTF-8 are incompatible. StringZilla [partially addresses those issues](#unicode-segmentation). ### What's Wrong with the C++ Standard Library? | C++ Code | Evaluation Result | Invoked Signature | | :----------------------------------- | :---------------- | :----------------------------- | | `"Loose"s.replace(2, 2, "vath"s, 1)` | `"Loathe"` 🤢 | `(pos1, count1, str2, pos2)` | | `"Loose"s.replace(2, 2, "vath", 1)` | `"Love"` 🥰 | `(pos1, count1, str2, count2)` | StringZilla is designed to be a drop-in replacement for the C++ Standard Templates Library. That said, some of the design decisions of STL strings are highly controversial, error-prone, and expensive. Most notably: 1. Argument order for `replace`, `insert`, `erase` and similar functions is impossible to guess. 2. Bounds-checking exceptions for `substr`-like functions are only thrown for one side of the range. 3. Returning string copies in `substr`-like functions results in absurd volume of allocations. 4. Incremental construction via `push_back`-like functions goes through too many branches. 5. Inconsistency between `string` and `string_view` methods, like the lack of `remove_prefix` and `remove_suffix`. Check the following set of asserts validating the `std::string` specification. It's not realistic to expect the average developer to remember the [14 overloads of `std::string::replace`][stl-replace]. [stl-replace]: https://en.cppreference.com/w/cpp/string/string/replace ```cpp using str = std::string; assert(str("hello world").substr(6) == "world"); assert(str("hello world").substr(6, 100) == "world"); // 106 is beyond the length of the string, but its OK assert_throws(str("hello world").substr(100), std::out_of_range); // 100 is beyond the length of the string assert_throws(str("hello world").substr(20, 5), std::out_of_range); // 20 is beyond the length of the string assert_throws(str("hello world").substr(-1, 5), std::out_of_range); // -1 casts to unsigned without any warnings... assert(str("hello world").substr(0, -1) == "hello world"); // -1 casts to unsigned without any warnings... assert(str("hello").replace(1, 2, "123") == "h123lo"); assert(str("hello").replace(1, 2, str("123"), 1) == "h23lo"); assert(str("hello").replace(1, 2, "123", 1) == "h1lo"); assert(str("hello").replace(1, 2, "123", 1, 1) == "h2lo"); assert(str("hello").replace(1, 2, str("123"), 1, 1) == "h2lo"); assert(str("hello").replace(1, 2, 3, 'a') == "haaalo"); assert(str("hello").replace(1, 2, {'a', 'b'}) == "hablo"); ``` To avoid those issues, StringZilla provides an alternative consistent interface. It supports signed arguments, and doesn't have more than 3 arguments per function or ```cpp using str = sz::string_view_t; str("a:b").front(1) == "a"; // no checks, unlike `substr` str("a:b").front(2) == "a:"; // take first 2 characters str("a:b").back(-1) == "b"; // accepting negative indices str("a:b").back(-2) == ":b"; // similar to Python's `"a:b"[-2:]` str("a:b").sub(1, -1) == ":"; // similar to Python's `"a:b"[1:-1]` str("a:b").sub(-2, -1) == ":"; // similar to Python's `"a:b"[-2:-1]` str("a:b").sub(-2, 1) == ""; // similar to Python's `"a:b"[-2:1]` "a:b"_sv[{-2, -1}] == ":"; // works on views and overloads `operator[]` ``` The C++ layer is a set of headers, so you can use the full API in some translation units and gradually transition to safer restricted API in others. Bonus - all the bound checking is branchless, so it has a constant cost and won't hurt your branch predictor. ### Beyond the C++ Standard Library, Learning from Python Python is arguably the most popular programming language for data science. In part, that's due to the simplicity of its standard interfaces. StringZilla brings some of that functionality to C++. - Content checks: `isalnum`, `isalpha`, `isascii`, `isdigit`, `islower`, `isspace`, `isupper`. - Trimming character sets: `lstrip`, `rstrip`, `strip`. - Trimming string matches: `remove_prefix`, `remove_suffix`. - Ranges of search results: `splitlines`, `split`, `rsplit`. - Number of non-overlapping substring matches: `count`. - Partitioning: `partition`, `rpartition`. For example, when parsing documents, it is often useful to split it into substrings. Most often, after that, you would compute the length of the skipped part, the offset and the length of the remaining part. This results in a lot of pointer arithmetic and is error-prone. StringZilla provides a convenient `partition` function, which returns a tuple of three string views, making the code cleaner. ```cpp auto parts = haystack.partition(':'); // Matching a character auto [before, match, after] = haystack.partition(':'); // Structure unpacking auto [before, match, after] = haystack.partition(sz::byteset_t(":;", 2)); // Character-set argument auto [before, match, after] = haystack.partition(" : "); // String argument auto [before, match, after] = haystack.rpartition(sz::whitespaces_set()); // Split around the last whitespace ``` Combining those with the `split` function, one can easily parse a CSV file or HTTP headers. ```cpp for (auto line : haystack.split("\r\n")) { auto [key, _, value] = line.partition(':'); headers[key.strip()] = value.strip(); } ``` Some other extensions are not present in the Python standard library either. Let's go through the C++ functionality category by category. - [Splits and Ranges](#splitting-and-partitioning). - Concatenating strings without allocations. - Random generation. - Edit distances and fuzzy search. Some of the StringZilla interfaces are not available even Python's native `str` class. Here is a sneak peek of the most useful ones. ```cpp text.hash(); // -> 64 bit unsigned integer text.ssize(); // -> 64 bit signed length to avoid `static_cast(text.size())` text.contains_only(sz::byteset_t(" \t", 2)); // == text.find_first_not_of(sz::byteset_t(" \t", 2)) == npos; text.contains(' '); // == text.find(' ') != npos; // Simpler slicing than `substr` text.front(10); // -> sz::string_view_t text.back(10); // -> sz::string_view_t // Safe variants, which clamp the range into the string bounds using sz::string_t::cap; text.front(10, cap) == text.front(std::min(10, text.size())); text.back(10, cap) == text.back(std::min(10, text.size())); // Character set filtering text.lstrip(sz::whitespaces_set()).rstrip(sz::newlines_set()); // like Python text.front(sz::whitespaces_set()); // all leading whitespaces text.back(sz::digits_set()); // all numerical symbols forming the suffix // Incremental construction using sz::string_t::unchecked; text.push_back('x'); // returns `bad_alloc_k` instead of throwing if growing fails text.push_back('x', unchecked); // no bounds checking, Rust style sz::concatenate(text, "@", domain, ".", tld); // No allocations ``` ### Compilation Settings and Debugging __`STRINGZILLA_DEBUG`__: > For maximal performance, the C library does not perform any bounds checking in Release builds. > In C++, bounds checking happens only in places where the STL `std::string` would do it. > If you want to enable more aggressive bounds-checking, define `STRINGZILLA_DEBUG` before including the header. > If not explicitly set, it will be inferred from the build type. __`STRINGZILLA_TARGET_GOLDMONT`, `STRINGZILLA_TARGET_WESTMERE`, `STRINGZILLA_TARGET_HASWELL`, `STRINGZILLA_TARGET_SKYLAKE`, `STRINGZILLA_TARGET_ICELAKE`, `STRINGZILLA_TARGET_NEON`, `STRINGZILLA_TARGET_NEONAES`, `STRINGZILLA_TARGET_NEONSHA`, `STRINGZILLA_TARGET_SVE`, `STRINGZILLA_TARGET_SVE2`, `STRINGZILLA_TARGET_SVE2AES`, `STRINGZILLA_TARGET_V128`, `STRINGZILLA_TARGET_V128RELAXED`, `STRINGZILLA_TARGET_RVV`, `STRINGZILLA_TARGET_LOONGSONASX`, `STRINGZILLA_TARGET_POWERVSX`__: > One can explicitly enable or disable individual SIMD families for compatibility or benchmarking purposes. > In header-only use the defaults are inferred from the compiler's predefined macros under your own `-march` flags; LASX and POWER9 need your `-mlasx` and `-mcpu=power9`, as their intrinsics headers hide without them. > The CMake build, which every binding goes through, resolves them from the `probes/` sources instead — compiling each capability's probe, which calls one of its kernels, to learn whether the toolchain builds them — so the libraries carry every compilable capability. > The same names work as CMake cache options (`-D STRINGZILLA_TARGET_SVE2=0`) and as Cargo environment variables (`STRINGZILLA_TARGET_SVE2=0 cargo build`); an explicit `1` keeps a capability only where its probe compiles. __`STRINGZILLA_TARGET_CUDA`, `STRINGZILLA_TARGET_ROCM`, `STRINGZILLA_TARGET_METAL`__: > Compile the GPU kernels of one vendor: CUDA wherever the unit is compiled as CUDA, by nvcc or by Clang, ROCm wherever it is compiled as HIP, and Metal wherever the build links Metal and Foundation, as `STRINGZILLA_METAL_ARCH` does. > The Metal kernels travel as embedded source and compile on the device at first use. > Each defaults to 1 wherever its vendor is present, and setting it to 0 keeps the host API while dropping that vendor's kernels. __`STRINGZILLA_HEADER_ONLY`__: > By default, the headers declare the StringZilla library, which compiles every capability once and dispatches among them at runtime, as the shared library and every binding do. > Set it to 1 to inline the kernels your compiler flags enable instead, calling them by name; the dispatch points and finders then report `sz_missing_library_k`. __`STRINGZILLA_ALLOW_MISALIGNED_LOADS`__: > Default is platform-dependent: enabled on x86 (where unaligned accesses are fast), disabled on others by default. > When enabled, many byte-level operations use word-sized loads, which can significantly accelerate the serial (SWAR) backend. > Consider enabling it explicitly if you are targeting platforms that support fast unaligned loads. __`STRINGZILLA_WITH_LIBC`__: > Both the library and the C headers can be built without LibC. > This may affect the type resolution system on obscure hardware platforms. __`STRINGZILLA_WITH_STL`__: > When using the C++ interface one can disable implicit conversions from `std::string` to `sz::string_t` and back. > If not needed, the `` and `` headers will be excluded, reducing compilation time. __`STRINGZILLA_BUILD_SHARED`, `STRINGZILLA_BUILD_TEST`, `STRINGZILLA_BUILD_BENCH`, `STRINGZILLA_MARCH`__ for CMake users: > When compiling the tests and benchmarks, you can explicitly set the target hardware architecture. > It's synonymous to GCC's `-march` flag and is used to enable/disable the appropriate instruction sets. > You can also disable the shared library build, if you don't need it.