The binding exposes codepoint counting, Unicode normalization, and full case folding over UTF-8 bytes.
Counts are scalar values, not bytes; normalization covers all four forms (NFC, NFD, NFKC, NFKD).
sz.Utf8Count("你好世界") // 4 codepoints
sz.Utf8Count("Hello🌍") // 6 — the astral emoji is one scalar
sz.Utf8Normalize("é", sz.NFC) // "e" + U+0301 → precomposed "é"
sz.Utf8Normalize("fi", sz.NFKC) // ligature "fi" → "fi"These helpers apply full Unicode case folding.
Each takes a validate flag; when set, inputs are checked and ErrInvalidUTF8 is returned for malformed UTF-8.
var ErrInvalidUTF8 = errors.New("invalid UTF-8")
func Utf8CaseFold(str string, validate bool) (string, error)
func Utf8CaseInsensitiveFind(haystack, needle string, validate bool) (index int64, length int64, err error)Utf8CaseFold returns the case-folded string; folding may expand the output, so "ß" becomes "ss". Utf8CaseInsensitiveFind returns the byte index and matched byte length of the first case-insensitive match, or index == -1 when no match is found; an empty needle returns 0, 0.
folded, _ := sz.Utf8CaseFold("Straße", false) // "strasse"
idx, length, _ := sz.Utf8CaseInsensitiveFind("Hello WÖRLD", "wörld", false)
// idx == 6, length == 6For repeated searches with the same needle, Utf8CaseInsensitiveNeedle prepares it once, over its own copy of the bytes.
It is safe for concurrent use, since searches only read it.
type Utf8CaseInsensitiveNeedle struct { /* ... */ }
func NewUtf8CaseInsensitiveNeedle(needle string, validate bool) (*Utf8CaseInsensitiveNeedle, error)
func (n *Utf8CaseInsensitiveNeedle) FindIn(haystack string, validate bool) (index int64, length int64, err error)needle, _ := sz.NewUtf8CaseInsensitiveNeedle("wörld", false)
idx, length, _ := needle.FindIn("Hello WÖRLD", false)NormalFormNormalForm selects a Unicode normalization form for Utf8Normalize.type NormalForm intNFD | NFD is canonical decomposition. | |
|---|---|---|
NFC | NFC is canonical decomposition followed by canonical composition. | |
NFKD | NFKD is compatibility decomposition. | |
NFKC | NFKC is compatibility decomposition followed by canonical composition. |
Utf8CaseInsensitiveNeedleUtf8CaseInsensitiveNeedle is a needle prepared once for repeated case-insensitive UTF-8 searches.2type Utf8CaseInsensitiveNeedle struct {
// contains filtered or unexported fields
}It is safe for concurrent use, as searches only read it.
NewUtf8CaseInsensitiveNeedleNewUtf8CaseInsensitiveNeedle prepares a reusable case-insensitive needle over its own copy of it.func NewUtf8CaseInsensitiveNeedle(
needle string,
validate bool,
) (*Utf8CaseInsensitiveNeedle, error)If validate is true, the needle is validated as UTF-8.
FindInFindIn searches for the prepared needle in haystack and returns byte offsets.func (n *Utf8CaseInsensitiveNeedle) FindIn(
haystack string,
validate bool,
) (index int64, length int64, err error)Utf8CaseFoldUtf8CaseFold applies full Unicode case folding to a UTF-8 string.func Utf8CaseFold(str string, validate bool) (string, error)It can expand the output, as "ß" → "ss" does, so it allocates up to 3× the input byte size.
Utf8CaseInsensitiveFindUtf8CaseInsensitiveFind finds the first case-insensitive occurrence of needle in haystack using full Unicode case folding and returns byte offsets.func Utf8CaseInsensitiveFind(
haystack, needle string,
validate bool,
) (index int64, length int64, err error)Utf8NormalizeUtf8Normalize returns the string normalized to the requested Unicode form.func Utf8Normalize(str string, form NormalForm) stringMalformed bytes pass through unchanged. The destination is sized for the worst-case compatibility decomposition (18x); composing forms never exceed that bound.