# Case Folding and Normalization > StringZilla for Rust `utf8_uncased_fold` case-folds into a target buffer, `utf8_norm` applies a Unicode normal form, and `utf8_find_denormalized` checks conformance without rewriting. ```rust pub fn utf8_uncased_fold, T: AsMut<[u8]>>(source: S, target: &mut T) -> Result; pub fn utf8_norm, T: AsMut<[u8]>>(source: S, form: NormalForm, target: &mut T) -> Result; pub fn utf8_find_denormalized>(source: T, form: NormalForm) -> Option; ``` `utf8_uncased_fold` applies Unicode case folding, for example `ß` → `ss`, returning the number of bytes written. `utf8_norm` normalizes to one of the five `NormalForm` variants: `Nfd`, `Nfc`, `Nfkd`, `Nfkc`, and `NfkcCasefold`, the Unicode `toNFKC_Casefold` that caseless search keys match on. `utf8_find_denormalized` is a fast check returning the byte offset of the first non-conforming byte, or `None` if already normalized: ```rust use stringzilla::{self as sz, NormalForm}; let mut dest = [0u8; 33]; let len = sz::utf8_uncased_fold("HELLO WORLD", &mut dest).unwrap(); assert_eq!(&dest[..len], b"hello world"); // NFC check: a decomposed "café" (e + combining acute) violates NFC. assert!(sz::utf8_find_denormalized("cafe\u{0301}", NormalForm::Nfc).is_some()); assert!(sz::utf8_find_denormalized("caf\u{00E9}", NormalForm::Nfc).is_none()); let mut out = vec![0u8; "cafe\u{0301}".len() * 18]; let n = sz::utf8_norm("cafe\u{0301}", NormalForm::Nfc, &mut out).unwrap(); assert_eq!(&out[..n], "caf\u{00E9}".as_bytes()); ``` The C kernels write without a capacity, so target buffers must hold the worst-case expansion whatever the input: `source.len() * 3` for folding and `source.len() * 18` for normalization. A shorter one is refused with `Status::UnexpectedDimensions` before anything is written. ### Uncased UTF-8 Search `utf8_uncased_search` locates a needle in a haystack under Unicode case folding. ```rust pub fn utf8_uncased_search, N: Utf8UncasedNeedleArg>(haystack: H, needle: N) -> Option<(usize, usize)>; ``` Returns `Some((offset, match_length))`, where the matched length may differ from the needle length due to case folding — `ß` matching `SS`, for instance. A `Utf8UncasedNeedle` is prepared once and only read by every search, so one needle serves many haystacks and threads, and `Utf8UncasedMatches` iterates all matches as `IndexSpan`s. Matches start on codepoint boundaries: overlapping iteration and any zero-length match advance one codepoint, so an empty needle matches at every codepoint boundary, the end included, three times in `"aé"`: ```rust use stringzilla::{self as sz, Utf8UncasedNeedle, Utf8UncasedMatches, IndexSpan}; assert_eq!(sz::utf8_uncased_search("Hello WORLD", "world"), Some((6, 5))); let needle = Utf8UncasedNeedle::new(b"hello"); assert_eq!(sz::utf8_uncased_search(b"HELLO there", &needle), Some((0, 5))); let spans: Vec = Utf8UncasedMatches::new(b"Hello hello", b"hello").collect(); assert_eq!(spans, vec![IndexSpan::new(0, 5), IndexSpan::new(6, 5)]); ``` `use stringzilla::CaseSensitivity;` `use stringzilla::NormalForm;` `use stringzilla::Utf8UncasedNeedle;` `use stringzilla::Utf8UncasedNeedleArg;` `use stringzilla::Utf8UncasedMatches;` `use stringzilla::*;` ## CaseSensitivity Whether a vocabulary matches needles byte-for-byte, or folds both sides to a shared case first. ```rust pub enum CaseSensitivity ``` Corresponds to `sz_case_sensitivity_t` in the C API. - `Cased` = `0`: Byte-exact matching; needles may be arbitrary bytes, including malformed UTF-8. - `Uncased` = `1`: Full Unicode case folding as `CaseFolding.txt` defines it; needles must be valid UTF-8. Implements: `Debug`, `Clone`, `Copy`, `StructuralPartialEq`, `PartialEq`, `Eq` ## NormalForm Unicode normalization forms for UTF-8 normalization operations. ```rust pub enum NormalForm ``` Corresponds to `sz_normal_form_t` in the C API. - `Nfd` = `0`: Canonical Decomposition. Decomposes precomposed characters into base + combining marks. - `Nfc` = `1`: Canonical Decomposition followed by Canonical Composition. The most common Unicode form. - `Nfkd` = `2`: Compatibility Decomposition. Decomposes ligatures and compatibility characters. - `Nfkc` = `3`: Compatibility Decomposition followed by Canonical Composition. - `NfkcCasefold` = `4`: NFKC with full case folding and without default-ignorable codepoints, the Unicode `toNFKC_Casefold` that caseless identifiers and search keys are matched on. Implements: `Debug`, `Clone`, `Copy`, `StructuralPartialEq`, `PartialEq`, `Eq` ## Utf8UncasedNeedle A needle prepared once for repeated uncased searches of UTF-8 text. ```rust pub struct Utf8UncasedNeedle<'a> ``` Mirrors the C `sz_utf8_uncased_needle_t`: the needle's bytes, borrowed for `'a`, and the folding analysis [`Self::new`](https://www.unum.cloud/docs/stringzilla/rust/runtime-dispatch.md#new) runs once. Searches only read it, so one needle serves any number of threads at once. Examples: ``` use stringzilla::{utf8_uncased_search, Utf8UncasedNeedle}; let needle = Utf8UncasedNeedle::new(b"hello"); assert_eq!(utf8_uncased_search(b"Hello World", &needle), Some((0, 5))); assert_eq!(utf8_uncased_search(b"say HELLO", &needle), Some((4, 5))); ``` Implements: `Send`, `Sync`, `[Utf8UncasedNeedleArg](#Utf8UncasedNeedleArg)` ## new Prepares `needle` for uncased search, analysing how it folds. ```rust pub fn new(needle: &'a [u8]) -> Self ``` ## as_bytes Returns the needle bytes. ```rust pub const fn as_bytes(&self) -> &[u8] ``` ## len Returns the length of the needle in bytes. ```rust pub const fn len(&self) -> usize ``` ## is_empty Returns true if the needle is empty. ```rust pub const fn is_empty(&self) -> bool ``` ## find_uncased_*_uncased_in Performs the uncased search in the given haystack. ```rust fn find_uncased__uncased_in(self, haystack: &[u8]) -> Option<(usize, usize)> ``` ## Utf8UncasedNeedleArg Trait for types that can be used as a uncased search needle. ```rust pub trait Utf8UncasedNeedleArg ``` This trait is implemented for: - Any type implementing `AsRef<[u8]>`, such as strings and byte slices - [`Utf8UncasedNeedle`](#Utf8UncasedNeedle) references for efficient repeated searches Implementors: - `impl> [Utf8UncasedNeedleArg](#Utf8UncasedNeedleArg) for Source` - `impl [Utf8UncasedNeedleArg](#Utf8UncasedNeedleArg) for &[Utf8UncasedNeedle](#Utf8UncasedNeedle)<'_>` ## Utf8UncasedMatches An iterator over uncased matches of a UTF-8 pattern in a string. ```rust pub struct Utf8UncasedMatches<'a, O: Overlaps = NonOverlapping> ``` This iterator yields [`IndexSpan`](https://www.unum.cloud/docs/stringzilla/rust/searching-and-counting.md#IndexSpan) values representing the byte offset and length of each match. The match length may differ from the needle length due to Unicode case folding: "ß" matches "SS", as German eszett expands to two characters. The iterator prepares the needle once, at construction. Matches start on codepoint boundaries: overlapping iteration and any zero-length match advance one codepoint, so an empty needle matches at every codepoint boundary, the end included. Examples: ``` use stringzilla::{Utf8UncasedMatches, IndexSpan}; let haystack = b"Hello WORLD, hello world"; let matches: Vec<IndexSpan> = Utf8UncasedMatches::new(haystack, b"hello").collect(); assert_eq!(matches.len(), 2); assert_eq!(matches[0], IndexSpan::new(0, 5)); assert_eq!(matches[1], IndexSpan::new(13, 5)); ``` With overlapping matches: ``` use stringzilla::{Utf8UncasedMatches, IndexSpan}; let haystack = b"aAaAa"; let matches: Vec<IndexSpan> = Utf8UncasedMatches::new(haystack, b"aA").overlapping().collect(); assert_eq!(matches.len(), 4); // Overlapping matches ``` Implements: `Iterator` ## new Creates a new iterator for non-overlapping uncased matches. ```rust pub fn new(haystack: &'a [u8], needle: &'a [u8]) -> Self ``` ## overlapping Report overlapping matches too, a compile-time policy returning the [`Overlapping`](https://www.unum.cloud/docs/stringzilla/rust/searching-and-counting.md#Overlapping) variant. ```rust pub fn overlapping(self) -> Utf8UncasedMatches<'a, Overlapping> ``` ## utf8_norm Normalizes a UTF-8 string to the requested Unicode Normal Form, writing the result to a target buffer. ```rust pub fn utf8_norm( source: Source, form: NormalForm, target: &mut Target, ) -> Result where Source: AsRef<[u8]>, Target: AsMut<[u8]> + ?Sized, ``` Covers all four standard forms: NFD, NFC, NFKD, and NFKC. NFC is the most common form on the web; NFD is useful for collation. Compatibility forms (NFKD/NFKC) additionally decompose ligatures and compatibility characters (e.g., U+FB03 ffi → "ffi"), and NFKC_Casefold folds case on top of NFKC, so "Straße" becomes "strasse". #### Returns Returns the number of bytes written to the target buffer. Arguments: - `source`: The UTF-8 string to normalize. - `form`: The Unicode normalization form to produce. - `target`: The target buffer to write the normalized string. Errors: The C kernel writes without a capacity, so `target` must hold the worst-case expansion of `source.len() * 18` bytes whatever the input, or [`Status::UnexpectedDimensions`](https://www.unum.cloud/docs/stringzilla/rust/types.md#UnexpectedDimensions) is returned. Examples: ``` use stringzilla as sz; use sz::NormalForm; let source = "caf\u{00E9}"; // "café" NFC (precomposed é) let mut dest = vec![0u8; source.len() * 18]; let len = sz::utf8_norm(source, NormalForm::Nfc, &mut dest).unwrap(); assert_eq!(&dest[..len], "caf\u{00E9}".as_bytes()); // unchanged — already NFC ``` ## utf8_find_denormalized Returns the byte offset of the first byte in `source` that violates the given Unicode Normal Form, or `None` if `source` is already in the requested form. ```rust pub fn utf8_find_denormalized(source: Source, form: NormalForm) -> Option where Source: AsRef<[u8]>, ``` This is a fast check — it does not produce the normalized output. Use it to avoid an unnecessary [`utf8_norm`](#utf8_norm) call when the input is likely already normalized. #### Returns - `None` if `source` already conforms to `form`. - `Some(offset)` with the byte offset of the first offending byte otherwise. Arguments: - `source`: The UTF-8 string to inspect. - `form`: The normalization form to check against. Examples: ``` use stringzilla as sz; use sz::NormalForm; // NFD string (decomposed): base 'e' + combining acute U+0301 let nfd = "cafe\u{0301}"; assert!(sz::utf8_find_denormalized(nfd, NormalForm::Nfc).is_some()); assert!(sz::utf8_find_denormalized("café", NormalForm::Nfc).is_none()); ``` ## utf8_uncased_search Performs uncased search for `needle` in UTF-8 `haystack`. ```rust pub fn utf8_uncased_search( haystack: Haystack, needle: Needle, ) -> Option<(usize, usize)> where Haystack: AsRef<[u8]>, Needle: Utf8UncasedNeedleArg, ``` Unlike ASCII uncased search, this handles Unicode case folding, like German ß matching "ss" and Turkish İ matching "i". #### Returns If found, returns `Some((offset, match_length))` where: - `offset` is the byte position in haystack where the match starts - `match_length` is the number of bytes matched in haystack, which may differ from the needle length Returns `None` if no match is found. Arguments: - `haystack`: The UTF-8 text to search in. - `needle`: The UTF-8 pattern to search for. Examples: Basic usage with string slices: ``` use stringzilla as sz; let haystack = "Hello WORLD"; if let Some((offset, len)) = sz::utf8_uncased_search(haystack, "world") { assert_eq!(offset, 6); assert_eq!(len, 5); } ``` With a prepared needle for repeated searches: ``` use stringzilla::{utf8_uncased_search, Utf8UncasedNeedle}; let needle = Utf8UncasedNeedle::new(b"hello"); // Analysed once in `new`, then only read by each search let result1 = utf8_uncased_search(b"Hello World", &needle); let result2 = utf8_uncased_search(b"HELLO there", &needle); assert_eq!(result1, Some((0, 5))); assert_eq!(result2, Some((0, 5))); ``` ## utf8_uncased_order Compares two UTF-8 strings in uncased manner. ```rust pub fn utf8_uncased_order(first: First, second: Second) -> Ordering where First: AsRef<[u8]>, Second: AsRef<[u8]>, ``` Uses Unicode case folding for comparison, handling characters like German ß, Turkish İ/ı, and other case variants. #### Returns - `Ordering::Less` if `first < second` - `Ordering::Equal` if `first == second` uncasedly - `Ordering::Greater` if `first > second` Arguments: - `first`: First UTF-8 string. - `second`: Second UTF-8 string. Examples: ``` use stringzilla as sz; use std::cmp::Ordering; assert_eq!(sz::utf8_uncased_order("Hello", "HELLO"), Ordering::Equal); assert_eq!(sz::utf8_uncased_order("abc", "ABD"), Ordering::Less); ``` ## utf8_uncased_fold Applies Unicode case folding to a UTF-8 string, writing the result to a target buffer. ```rust pub fn utf8_uncased_fold( source: Source, target: &mut Target, ) -> Result where Source: AsRef<[u8]>, Target: AsMut<[u8]> + ?Sized, ``` Case folding normalizes text for uncased comparisons by mapping uppercase letters to their lowercase equivalents and handling special cases like German U+00DF → ss expansion. #### Returns Returns the number of bytes written to the target buffer. Arguments: - `source`: The UTF-8 string to case-fold. - `target`: The target buffer to write the case-folded string. Errors: The C kernel writes without a capacity, so `target` must hold the worst-case 3:1 expansion of `source.len() * 3` bytes whatever the input, or [`Status::UnexpectedDimensions`](https://www.unum.cloud/docs/stringzilla/rust/types.md#UnexpectedDimensions) is returned. Examples: ``` use stringzilla as sz; let source = "HELLO WORLD"; let mut dest = [0u8; 33]; let len = sz::utf8_uncased_fold(source, &mut dest).unwrap(); assert_eq!(&dest[..len], b"hello world"); ```