Case Folding and Normalization

These functions operate on UTF-8 encoded buffers and apply full Unicode case folding. Each accepts an optional trailing validate boolean; when true, the input is checked for valid UTF-8 and an error is thrown otherwise.

utf8UncasedFold returns a new Buffer with the input case-folded. The result may be longer than the input because some code points expand when folded.

assert.strictEqual(sz.utf8UncasedFold(Buffer.from("Straße")).toString(), "strasse");

utf8UncasedFind performs a case-insensitive substring search. It returns an object { index, length }, both BigInt, where index is the matched byte offset, or -1n if not found, and length is the matched byte length, which can differ from the needle’s length under folding.

const { index, length } = sz.utf8UncasedFind(Buffer.from("Hello WÖRLD"), Buffer.from("wörld"));

The Utf8UncasedNeedle class prepares a needle once for repeated searches, amortizing the folding setup. It keeps its own copy of the needle bytes, so later writes to the source buffer do not affect it. An empty needle matches at index 0n with length 0n, in both forms. Construct it with the needle buffer and an optional validate flag, then call findIn(haystack, validate?), which returns the same { index, length } object.

const needle = new sz.Utf8UncasedNeedle(Buffer.from("wörld"));
needle.findIn(Buffer.from("Hello WÖRLD")); // => { index, length }

§ Unicode Normalization

utf8Norm converts a UTF-8 buffer into one of the four Unicode normal forms, passed as a Utf8NormalForm constant: NFD, NFC, NFKD, or NFKC. It returns a new Buffer and accepts the same optional trailing validate boolean as the case-folding functions.

const decomposed = Buffer.from("café"); // "e" followed by a combining acute
assert.strictEqual(sz.utf8Norm(decomposed, sz.Utf8NormalForm.NFC).toString(), "café");

utf8FindDenormalized checks a buffer against a normal form without rewriting it. It returns the byte offset of the first violation as a BigInt, or -1n when the input is already normalized.

assert.strictEqual(sz.utf8FindDenormalized(Buffer.from("café"), sz.Utf8NormalForm.NFC), -1n);
assert.strictEqual(sz.utf8FindDenormalized(decomposed, sz.Utf8NormalForm.NFC), 3n);