Platform Notes

§ What’s Wrong with the C Standard Library?

StringZilla is not a drop-in replacement for the C Standard Library. It’s designed to be a safer and more modern alternative. Conceptually:

1. LibC strings are expected to be null-terminated, so to use the efficient LibC implementations on slices of larger strings, you’d have to copy them, which is more expensive than the original string operation.
2. LibC functionality is asymmetric — you can find the first and the last occurrence of a character within a string, but you can’t find the last occurrence of a substring.
3. LibC function names are typically very short and cryptic.
4. LibC lacks crucial functionality like hashing and doesn’t provide primitives for less critical but relevant operations like fuzzy matching.

Something has to be said about its support for UTF-8. Aside from a single-byte char type, LibC provides wchar_t:

• The size of wchar_t is not consistent across platforms. On Windows, it’s typically 16 bits (suitable for UTF-16), while on Unix-like systems, it’s usually 32 bits (suitable for UTF-32). This inconsistency can lead to portability issues when writing cross-platform code.
• wchar_t is designed to represent wide characters in a fixed-width format (UTF-16 or UTF-32). In contrast, UTF-8 is a variable-length encoding, where each character can take from 1 to 4 bytes. This fundamental difference means that wchar_t and UTF-8 are incompatible.

StringZilla partially addresses those issues.

§ What’s Wrong with the C++ Standard Library?

StringZilla is designed to be a drop-in replacement for the C++ Standard Templates Library. That said, some of the design decisions of STL strings are highly controversial, error-prone, and expensive. Most notably:

1. Argument order for replace, insert, erase and similar functions is impossible to guess.
2. Bounds-checking exceptions for substr-like functions are only thrown for one side of the range.
3. Returning string copies in substr-like functions results in absurd volume of allocations.
4. Incremental construction via push_back-like functions goes through too many branches.
5. Inconsistency between string and string_view methods, like the lack of remove_prefix and remove_suffix.

Check the following set of asserts validating the std::string specification. It’s not realistic to expect the average developer to remember the 14 overloads of std::string::replace.

using str = std::string;

assert(str("hello world").substr(6) == "world");
assert(str("hello world").substr(6, 100) == "world"); // 106 is beyond the length of the string, but its OK
assert_throws(str("hello world").substr(100), std::out_of_range);   // 100 is beyond the length of the string
assert_throws(str("hello world").substr(20, 5), std::out_of_range); // 20 is beyond the length of the string
assert_throws(str("hello world").substr(-1, 5), std::out_of_range); // -1 casts to unsigned without any warnings...
assert(str("hello world").substr(0, -1) == "hello world");          // -1 casts to unsigned without any warnings...

assert(str("hello").replace(1, 2, "123") == "h123lo");
assert(str("hello").replace(1, 2, str("123"), 1) == "h23lo");
assert(str("hello").replace(1, 2, "123", 1) == "h1lo");
assert(str("hello").replace(1, 2, "123", 1, 1) == "h2lo");
assert(str("hello").replace(1, 2, str("123"), 1, 1) == "h2lo");
assert(str("hello").replace(1, 2, 3, 'a') == "haaalo");
assert(str("hello").replace(1, 2, {'a', 'b'}) == "hablo");

To avoid those issues, StringZilla provides an alternative consistent interface. It supports signed arguments, and doesn’t have more than 3 arguments per function or

using str = sz::string_view_t;

str("a:b").front(1) == "a"; // no checks, unlike `substr`
str("a:b").front(2) == "a:"; // take first 2 characters
str("a:b").back(-1) == "b"; // accepting negative indices
str("a:b").back(-2) == ":b"; // similar to Python's `"a:b"[-2:]`
str("a:b").sub(1, -1) == ":"; // similar to Python's `"a:b"[1:-1]`
str("a:b").sub(-2, -1) == ":"; // similar to Python's `"a:b"[-2:-1]`
str("a:b").sub(-2, 1) == ""; // similar to Python's `"a:b"[-2:1]`
"a:b"_sv[{-2, -1}] == ":"; // works on views and overloads `operator[]`

The C++ layer is a set of headers, so you can use the full API in some translation units and gradually transition to safer restricted API in others. Bonus - all the bound checking is branchless, so it has a constant cost and won’t hurt your branch predictor.

§ Beyond the C++ Standard Library, Learning from Python

Python is arguably the most popular programming language for data science. In part, that’s due to the simplicity of its standard interfaces. StringZilla brings some of that functionality to C++.

• Content checks: isalnum, isalpha, isascii, isdigit, islower, isspace, isupper.
• Trimming character sets: lstrip, rstrip, strip.
• Trimming string matches: remove_prefix, remove_suffix.
• Ranges of search results: splitlines, split, rsplit.
• Number of non-overlapping substring matches: count.
• Partitioning: partition, rpartition.

For example, when parsing documents, it is often useful to split it into substrings. Most often, after that, you would compute the length of the skipped part, the offset and the length of the remaining part. This results in a lot of pointer arithmetic and is error-prone. StringZilla provides a convenient partition function, which returns a tuple of three string views, making the code cleaner.

auto parts = haystack.partition(':'); // Matching a character
auto [before, match, after] = haystack.partition(':'); // Structure unpacking
auto [before, match, after] = haystack.partition(sz::byteset_t(":;", 2)); // Character-set argument
auto [before, match, after] = haystack.partition(" : "); // String argument
auto [before, match, after] = haystack.rpartition(sz::whitespaces_set()); // Split around the last whitespace

Combining those with the split function, one can easily parse a CSV file or HTTP headers.

for (auto line : haystack.split("\r\n")) {
    auto [key, _, value] = line.partition(':');
    headers[key.strip()] = value.strip();
}

Some other extensions are not present in the Python standard library either. Let’s go through the C++ functionality category by category.

• Splits and Ranges.
• Concatenating strings without allocations.
• Random generation.
• Edit distances and fuzzy search.

Some of the StringZilla interfaces are not available even Python’s native str class. Here is a sneak peek of the most useful ones.

text.hash(); // -> 64 bit unsigned integer 
text.ssize(); // -> 64 bit signed length to avoid `static_cast<std::ssize_t>(text.size())`
text.contains_only(sz::byteset_t(" \t", 2)); // == text.find_first_not_of(sz::byteset_t(" \t", 2)) == npos;
text.contains(' '); // == text.find(' ') != npos;

// Simpler slicing than `substr`
text.front(10); // -> sz::string_view_t
text.back(10); // -> sz::string_view_t

// Safe variants, which clamp the range into the string bounds
using sz::string_t::cap;
text.front(10, cap) == text.front(std::min(10, text.size()));
text.back(10, cap) == text.back(std::min(10, text.size()));

// Character set filtering
text.lstrip(sz::whitespaces_set()).rstrip(sz::newlines_set()); // like Python
text.front(sz::whitespaces_set()); // all leading whitespaces
text.back(sz::digits_set()); // all numerical symbols forming the suffix

// Incremental construction
using sz::string_t::unchecked;
text.push_back('x'); // returns `bad_alloc_k` instead of throwing if growing fails
text.push_back('x', unchecked); // no bounds checking, Rust style

sz::concatenate(text, "@", domain, ".", tld); // No allocations

§ Compilation Settings and Debugging

STRINGZILLA_DEBUG:

For maximal performance, the C library does not perform any bounds checking in Release builds. In C++, bounds checking happens only in places where the STL std::string would do it. If you want to enable more aggressive bounds-checking, define STRINGZILLA_DEBUG before including the header. If not explicitly set, it will be inferred from the build type.

STRINGZILLA_TARGET_GOLDMONT, STRINGZILLA_TARGET_WESTMERE, STRINGZILLA_TARGET_HASWELL, STRINGZILLA_TARGET_SKYLAKE, STRINGZILLA_TARGET_ICELAKE, STRINGZILLA_TARGET_NEON, STRINGZILLA_TARGET_NEONAES, STRINGZILLA_TARGET_NEONSHA, STRINGZILLA_TARGET_SVE, STRINGZILLA_TARGET_SVE2, STRINGZILLA_TARGET_SVE2AES, STRINGZILLA_TARGET_V128, STRINGZILLA_TARGET_V128RELAXED, STRINGZILLA_TARGET_RVV, STRINGZILLA_TARGET_LOONGSONASX, STRINGZILLA_TARGET_POWERVSX:

One can explicitly enable or disable individual SIMD families for compatibility or benchmarking purposes. In header-only use the defaults are inferred from the compiler’s predefined macros under your own -march flags; LASX and POWER9 need your -mlasx and -mcpu=power9, as their intrinsics headers hide without them. The CMake build, which every binding goes through, resolves them from the probes/ sources instead — compiling each capability’s probe, which calls one of its kernels, to learn whether the toolchain builds them — so the libraries carry every compilable capability. The same names work as CMake cache options (-D STRINGZILLA_TARGET_SVE2=0) and as Cargo environment variables (STRINGZILLA_TARGET_SVE2=0 cargo build); an explicit 1 keeps a capability only where its probe compiles.

STRINGZILLA_TARGET_CUDA, STRINGZILLA_TARGET_ROCM, STRINGZILLA_TARGET_METAL:

Compile the GPU kernels of one vendor: CUDA wherever the unit is compiled as CUDA, by nvcc or by Clang, ROCm wherever it is compiled as HIP, and Metal wherever the build links Metal and Foundation, as STRINGZILLA_METAL_ARCH does. The Metal kernels travel as embedded source and compile on the device at first use. Each defaults to 1 wherever its vendor is present, and setting it to 0 keeps the host API while dropping that vendor’s kernels.

STRINGZILLA_HEADER_ONLY:

By default, the headers declare the StringZilla library, which compiles every capability once and dispatches among them at runtime, as the shared library and every binding do. Set it to 1 to inline the kernels your compiler flags enable instead, calling them by name; the dispatch points and finders then report sz_missing_library_k.

STRINGZILLA_ALLOW_MISALIGNED_LOADS:

Default is platform-dependent: enabled on x86 (where unaligned accesses are fast), disabled on others by default. When enabled, many byte-level operations use word-sized loads, which can significantly accelerate the serial (SWAR) backend. Consider enabling it explicitly if you are targeting platforms that support fast unaligned loads.

STRINGZILLA_WITH_LIBC:

Both the library and the C headers can be built without LibC. This may affect the type resolution system on obscure hardware platforms.

STRINGZILLA_WITH_STL:

When using the C++ interface one can disable implicit conversions from std::string to sz::string_t and back. If not needed, the <string> and <string_view> headers will be excluded, reducing compilation time.

STRINGZILLA_BUILD_SHARED, STRINGZILLA_BUILD_TEST, STRINGZILLA_BUILD_BENCH, STRINGZILLA_MARCH for CMake users:

When compiling the tests and benchmarks, you can explicitly set the target hardware architecture. It’s synonymous to GCC’s -march flag and is used to enable/disable the appropriate instruction sets. You can also disable the shared library build, if you don’t need it.