utf8_uncased_fold case-folds into a target buffer, utf8_norm applies a Unicode normal form, and utf8_find_denormalized checks conformance without rewriting.
pub fn utf8_uncased_fold<S: AsRef<[u8]>, T: AsMut<[u8]>>(source: S, target: &mut T) -> Result<usize, Status>;
pub fn utf8_norm<S: AsRef<[u8]>, T: AsMut<[u8]>>(source: S, form: NormalForm, target: &mut T) -> Result<usize, Status>;
pub fn utf8_find_denormalized<T: AsRef<[u8]>>(source: T, form: NormalForm) -> Option<usize>;utf8_uncased_fold applies Unicode case folding, for example ß → ss, returning the number of bytes written. utf8_norm normalizes to one of the five NormalForm variants: Nfd, Nfc, Nfkd, Nfkc, and NfkcCasefold, the Unicode toNFKC_Casefold that caseless search keys match on. utf8_find_denormalized is a fast check returning the byte offset of the first non-conforming byte, or None if already normalized:
use stringzilla::{self as sz, NormalForm};
let mut dest = [0u8; 33];
let len = sz::utf8_uncased_fold("HELLO WORLD", &mut dest).unwrap();
assert_eq!(&dest[..len], b"hello world");
// NFC check: a decomposed "café" (e + combining acute) violates NFC.
assert!(sz::utf8_find_denormalized("cafe\u{0301}", NormalForm::Nfc).is_some());
assert!(sz::utf8_find_denormalized("caf\u{00E9}", NormalForm::Nfc).is_none());
let mut out = vec![0u8; "cafe\u{0301}".len() * 18];
let n = sz::utf8_norm("cafe\u{0301}", NormalForm::Nfc, &mut out).unwrap();
assert_eq!(&out[..n], "caf\u{00E9}".as_bytes());The C kernels write without a capacity, so target buffers must hold the worst-case expansion whatever the input: source.len() * 3 for folding and source.len() * 18 for normalization.
A shorter one is refused with Status::UnexpectedDimensions before anything is written.
utf8_uncased_search locates a needle in a haystack under Unicode case folding.
pub fn utf8_uncased_search<H: AsRef<[u8]>, N: Utf8UncasedNeedleArg>(haystack: H, needle: N) -> Option<(usize, usize)>;Returns Some((offset, match_length)), where the matched length may differ from the needle length due to case folding — ß matching SS, for instance.
A Utf8UncasedNeedle is prepared once and only read by every search, so one needle serves many haystacks and threads, and Utf8UncasedMatches iterates all matches as IndexSpans.
Matches start on codepoint boundaries: overlapping iteration and any zero-length match advance one codepoint, so an empty needle matches at every codepoint boundary, the end included, three times in "aé":
use stringzilla::{self as sz, Utf8UncasedNeedle, Utf8UncasedMatches, IndexSpan};
assert_eq!(sz::utf8_uncased_search("Hello WORLD", "world"), Some((6, 5)));
let needle = Utf8UncasedNeedle::new(b"hello");
assert_eq!(sz::utf8_uncased_search(b"HELLO there", &needle), Some((0, 5)));
let spans: Vec<IndexSpan> = Utf8UncasedMatches::new(b"Hello hello", b"hello").collect();
assert_eq!(spans, vec![IndexSpan::new(0, 5), IndexSpan::new(6, 5)]);CaseSensitivityWhether a vocabulary matches needles byte-for-byte, or folds both sides to a shared case first.pub enum CaseSensitivityCorresponds to sz_case_sensitivity_t in the C API.
Cased | 0 | Byte-exact matching; needles may be arbitrary bytes, including malformed UTF-8. |
|---|---|---|
Uncased | 1 | Full Unicode case folding as CaseFolding.txt defines it; needles must be valid UTF-8. |
Debug, Clone, Copy, StructuralPartialEq, PartialEq, EqNormalFormUnicode normalization forms for UTF-8 normalization operations.pub enum NormalFormCorresponds to sz_normal_form_t in the C API.
Nfd | 0 | Canonical Decomposition. Decomposes precomposed characters into base + combining marks. |
|---|---|---|
Nfc | 1 | Canonical Decomposition followed by Canonical Composition. The most common Unicode form. |
Nfkd | 2 | Compatibility Decomposition. Decomposes ligatures and compatibility characters. |
Nfkc | 3 | Compatibility Decomposition followed by Canonical Composition. |
NfkcCasefold | 4 | NFKC with full case folding and without default-ignorable codepoints, the Unicode
toNFKC_Casefold that caseless identifiers and search keys are matched on. |
Debug, Clone, Copy, StructuralPartialEq, PartialEq, EqUtf8UncasedNeedleA needle prepared once for repeated uncased searches of UTF-8 text.5pub struct Utf8UncasedNeedle<'a>Mirrors the C sz_utf8_uncased_needle_t: the needle's bytes, borrowed for 'a, and the folding
analysis Self::new runs once. Searches only read it, so one needle serves any number of
threads at once.
use stringzilla::{utf8_uncased_search, Utf8UncasedNeedle};
let needle = Utf8UncasedNeedle::new(b"hello");
assert_eq!(utf8_uncased_search(b"Hello World", &needle), Some((0, 5)));
assert_eq!(utf8_uncased_search(b"say HELLO", &needle), Some((4, 5)));
newPrepares needle for uncased search, analysing how it folds.pub fn new(needle: &'a [u8]) -> Selfas_bytesReturns the needle bytes.pub const fn as_bytes(&self) -> &[u8]lenReturns the length of the needle in bytes.pub const fn len(&self) -> usizeis_emptyReturns true if the needle is empty.pub const fn is_empty(&self) -> boolfind_uncased_*_uncased_in× 2Performs the uncased search in the given haystack.Inherited from Utf8UncasedNeedleArg
fn find_uncased__uncased_in(self, haystack: &[u8]) -> Option<(usize, usize)>Utf8UncasedNeedleArgTrait for types that can be used as a uncased search needle.pub trait Utf8UncasedNeedleArgThis trait is implemented for:
AsRef<[u8]>, such as strings and byte slicesUtf8UncasedNeedle references for efficient repeated searchesimpl<Source: AsRef<[u8]>> Utf8UncasedNeedleArg for Sourceimpl Utf8UncasedNeedleArg for &Utf8UncasedNeedle<'_>Utf8UncasedMatchesAn iterator over uncased matches of a UTF-8 pattern in a string.2pub struct Utf8UncasedMatches<'a, O: Overlaps = NonOverlapping>This iterator yields IndexSpan values representing the byte offset and length
of each match. The match length may differ from the needle length due to Unicode
case folding: "ß" matches "SS", as German eszett expands to two characters.
The iterator prepares the needle once, at construction. Matches start on codepoint boundaries: overlapping iteration and any zero-length match advance one codepoint, so an empty needle matches at every codepoint boundary, the end included.
use stringzilla::{Utf8UncasedMatches, IndexSpan};
let haystack = b"Hello WORLD, hello world";
let matches: Vec<IndexSpan> = Utf8UncasedMatches::new(haystack, b"hello").collect();
assert_eq!(matches.len(), 2);
assert_eq!(matches[0], IndexSpan::new(0, 5));
assert_eq!(matches[1], IndexSpan::new(13, 5));
With overlapping matches:
use stringzilla::{Utf8UncasedMatches, IndexSpan};
let haystack = b"aAaAa";
let matches: Vec<IndexSpan> = Utf8UncasedMatches::new(haystack, b"aA").overlapping().collect();
assert_eq!(matches.len(), 4); // Overlapping matches
IteratornewCreates a new iterator for non-overlapping uncased matches.pub fn new(haystack: &'a [u8], needle: &'a [u8]) -> SelfoverlappingReport overlapping matches too, a compile-time policy returning the Overlapping variant.pub fn overlapping(self) -> Utf8UncasedMatches<'a, Overlapping>utf8_normNormalizes a UTF-8 string to the requested Unicode Normal Form, writing the result to a target buffer.pub fn utf8_norm<Source, Target>(
source: Source,
form: NormalForm,
target: &mut Target,
) -> Result<usize, Status>
where
Source: AsRef<[u8]>,
Target: AsMut<[u8]> + ?Sized,Covers all four standard forms: NFD, NFC, NFKD, and NFKC. NFC is the most common form on the web; NFD is useful for collation. Compatibility forms (NFKD/NFKC) additionally decompose ligatures and compatibility characters (e.g., U+FB03 ffi → "ffi"), and NFKC_Casefold folds case on top of NFKC, so "Straße" becomes "strasse".
Returns the number of bytes written to the target buffer.
source: The UTF-8 string to normalize.form: The Unicode normalization form to produce.target: The target buffer to write the normalized string.The C kernel writes without a capacity, so target must hold the worst-case expansion of
source.len() * 18 bytes whatever the input, or Status::UnexpectedDimensions is returned.
use stringzilla as sz;
use sz::NormalForm;
let source = "caf\u{00E9}"; // "café" NFC (precomposed é)
let mut dest = vec![0u8; source.len() * 18];
let len = sz::utf8_norm(source, NormalForm::Nfc, &mut dest).unwrap();
assert_eq!(&dest[..len], "caf\u{00E9}".as_bytes()); // unchanged — already NFC
utf8_find_denormalizedReturns the byte offset of the first byte in source that violates the given Unicode Normal Form, or None if source is already in the requested form.pub fn utf8_find_denormalized<Source>(source: Source, form: NormalForm) -> Option<usize>
where
Source: AsRef<[u8]>,This is a fast check — it does not produce the normalized output. Use it to avoid an
unnecessary utf8_norm call when the input is likely already normalized.
None if source already conforms to form.Some(offset) with the byte offset of the first offending byte otherwise.source: The UTF-8 string to inspect.form: The normalization form to check against.use stringzilla as sz;
use sz::NormalForm;
// NFD string (decomposed): base 'e' + combining acute U+0301
let nfd = "cafe\u{0301}";
assert!(sz::utf8_find_denormalized(nfd, NormalForm::Nfc).is_some());
assert!(sz::utf8_find_denormalized("café", NormalForm::Nfc).is_none());
utf8_uncased_searchPerforms uncased search for needle in UTF-8 haystack.pub fn utf8_uncased_search<Haystack, Needle>(
haystack: Haystack,
needle: Needle,
) -> Option<(usize, usize)>
where
Haystack: AsRef<[u8]>,
Needle: Utf8UncasedNeedleArg,Unlike ASCII uncased search, this handles Unicode case folding, like German ß matching "ss" and Turkish İ matching "i".
If found, returns Some((offset, match_length)) where:
offset is the byte position in haystack where the match startsmatch_length is the number of bytes matched in haystack, which may differ from
the needle lengthReturns None if no match is found.
haystack: The UTF-8 text to search in.needle: The UTF-8 pattern to search for.Basic usage with string slices:
use stringzilla as sz;
let haystack = "Hello WORLD";
if let Some((offset, len)) = sz::utf8_uncased_search(haystack, "world") {
assert_eq!(offset, 6);
assert_eq!(len, 5);
}
With a prepared needle for repeated searches:
use stringzilla::{utf8_uncased_search, Utf8UncasedNeedle};
let needle = Utf8UncasedNeedle::new(b"hello");
// Analysed once in `new`, then only read by each search
let result1 = utf8_uncased_search(b"Hello World", &needle);
let result2 = utf8_uncased_search(b"HELLO there", &needle);
assert_eq!(result1, Some((0, 5)));
assert_eq!(result2, Some((0, 5)));
utf8_uncased_orderCompares two UTF-8 strings in uncased manner.pub fn utf8_uncased_order<First, Second>(first: First, second: Second) -> Ordering
where
First: AsRef<[u8]>,
Second: AsRef<[u8]>,Uses Unicode case folding for comparison, handling characters like German ß, Turkish İ/ı, and other case variants.
Ordering::Less if first < secondOrdering::Equal if first == second uncasedlyOrdering::Greater if first > secondfirst: First UTF-8 string.second: Second UTF-8 string.use stringzilla as sz;
use std::cmp::Ordering;
assert_eq!(sz::utf8_uncased_order("Hello", "HELLO"), Ordering::Equal);
assert_eq!(sz::utf8_uncased_order("abc", "ABD"), Ordering::Less);
utf8_uncased_foldApplies Unicode case folding to a UTF-8 string, writing the result to a target buffer.pub fn utf8_uncased_fold<Source, Target>(
source: Source,
target: &mut Target,
) -> Result<usize, Status>
where
Source: AsRef<[u8]>,
Target: AsMut<[u8]> + ?Sized,Case folding normalizes text for uncased comparisons by mapping uppercase letters to their lowercase equivalents and handling special cases like German U+00DF → ss expansion.
Returns the number of bytes written to the target buffer.
source: The UTF-8 string to case-fold.target: The target buffer to write the case-folded string.The C kernel writes without a capacity, so target must hold the worst-case 3:1 expansion of
source.len() * 3 bytes whatever the input, or Status::UnexpectedDimensions is returned.
use stringzilla as sz;
let source = "HELLO WORLD";
let mut dest = [0u8; 33];
let len = sz::utf8_uncased_fold(source, &mut dest).unwrap();
assert_eq!(&dest[..len], b"hello world");