Case Folding and Normalization

Case-insensitive matching across full Unicode is more than ASCII tolower — ß matches ss, accents fold, and the matched byte-length can differ from the needle’s. utf8_uncased_search searches case-insensitively without pre-folding the haystack: it folds on the fly, so you never allocate a folded copy of a large document just to search it.

#include <stringzilla/stringzilla.hpp>
#include <cassert>
namespace sz = ashvardanian::stringzilla;

int main() {
    sz::string_view_t hay = "Take the STRAßE downtown";
    auto m = hay.utf8_uncased_search("strasse"); // no folded copy of `hay` is made
    assert(m.offset == 9);
    assert(hay.sub(m.offset, m.offset + m.length) == "STRAßE"); // matched the mixed-case ß form
    return 0;
}

The result carries both the offset and the matched length, since folding can make the matched span longer or shorter than the needle. To search many haystacks for one needle, prepare it once as a sz::utf8_uncased_needle_t, which borrows the needle’s bytes, and pass it to utf8_uncased_search or to utf8_uncased_matches, the lazy range of every non-overlapping match. Matches sit on codepoint boundaries, so an empty needle matches at every boundary, the end included. When you need the folded text itself, fold a sz::string_t in place:

sz::string_t greeting;
if (sz::failed(greeting.assign("Grüße"))) return 1;
if (sz::failed(greeting.utf8_uncased_fold())) return 1; // in place; `ß` expands to `ss`
assert(greeting == "grüsse");

In C, sz_utf8_uncased_needle_init_best analyses the needle once into a sz_utf8_uncased_needle_t, and sz_utf8_uncased_search_best only reads it, so any number of searches, under any capabilities, share one analysis. The prepared needle points at the needle’s bytes rather than copying them, so they must outlive it:

#include <stringzilla/stringzilla.h>

sz_utf8_uncased_needle_t needle;
sz_utf8_uncased_needle_init_best("café", 5, &needle, capabilities, NULL);
sz_cptr_t hit = NULL;
sz_size_t match_length = 0;
sz_utf8_uncased_search_best(haystack, haystack_length, &needle, &hit, &match_length, capabilities, NULL);
// `hit` points at the first case-insensitive match, or NULL; reuse `needle` for the next haystack.

An empty needle matches at the start of every haystack, with a zero match_length.