Utf8UncasedMatchesIterator yielding all uncased matches of needle in haystack.class Utf8UncasedMatchesUses Unicode case folding for proper handling of international text.
Each iteration yields a String view of the matched region in the haystack. The matched length may differ from needle length due to case folding expansions, like German 'ß' matching 'SS'.
>>> len(list(sz.utf8_uncased_matches('aAaA', 'a')))
4
utf8_uncased_foldApply Unicode case folding to a UTF-8 string.def utf8_uncased_fold(
text: _StringLike,
validate: bool = False,
*,
capabilities: Capability | None = None,
) -> bytesCase folding normalizes text for uncased comparisons, handling one-to-many expansions, like German sharp S to 'ss'.
text_StringLike | The input UTF-8 string. | |
|---|---|---|
validatebool | If True, validate UTF-8 before processing. Default: False. | |
capabilitiesCapability | None | Capabilities to run, by default the CPU's enabled ones. |
>>> sz.utf8_uncased_fold('HELLO')
b'hello'
>>> sz.utf8_uncased_fold('Stra\u00dfe') # German sharp S
b'strasse'
utf8_normNormalize a UTF-8 string to a Unicode normalization form.def utf8_norm(
text: _StringLike,
form: str,
validate: bool = False,
*,
capabilities: Capability | None = None,
) -> bytesTransforms the input into NFD, NFC, NFKD, or NFKC, mirroring unicodedata.normalize but
returning raw UTF-8 bytes, or into NFKC_Casefold, the Unicode toNFKC_Casefold: NFKC with full
case folding and no default-ignorable codepoints.
text_StringLike | The input UTF-8 string. | |
|---|---|---|
formstr | One of 'NFC', 'NFD', 'NFKC', 'NFKD', 'NFKC_Casefold'. | |
validatebool | If True, validate UTF-8 before processing. Default: False. | |
capabilitiesCapability | None | Capabilities to run, by default the CPU's enabled ones. |
>>> sz.utf8_norm('caf\u00e9', 'NFD') # decompose
b'cafe\xcc\x81'
>>> sz.utf8_norm('\ufb01', 'NFKD') # fi ligature
b'fi'
>>> sz.utf8_norm('Stra\u00dfe\u00ad', 'NFKC_Casefold') # folded, soft hyphen dropped
b'strasse'
utf8_find_denormalizedReturn the byte offset of the first normalization violation, or None.def utf8_find_denormalized(
text: _StringLike,
form: str,
*,
capabilities: Capability | None = None,
) -> int | NoneScans the UTF-8 string and returns the byte offset of the first codepoint that breaks the given normalization form (first non-Yes Quick-Check result or canonical-ordering violation). Returns None when the string is already fully normalized.
text_StringLike | The input UTF-8 string. | |
|---|---|---|
formstr | One of 'NFC', 'NFD', 'NFKC', 'NFKD', 'NFKC_Casefold'. | |
capabilitiesCapability | None | Capabilities to run, by default the CPU's enabled ones. |
>>> sz.utf8_find_denormalized('cafe\u0301', 'NFC') # 'e' + combining acute
3
>>> sz.utf8_find_denormalized('caf\u00e9', 'NFC') is None # precomposed, already NFC
True
utf8_uncased_searchFind substring using Unicode uncased matching.def utf8_uncased_search(
haystack: _StringLike,
needle: _StringLike,
start: int = 0,
end: int | None = None,
validate: bool = False,
*,
capabilities: Capability | None = None,
) -> intPerforms a uncased search using Unicode case folding rules, correctly handling one-to-many expansions, like 'ß' matching 'SS'.
haystack_StringLike | The string to search in. | |
|---|---|---|
needle_StringLike | The substring to find. | |
startint | Starting index, defaulting to 0. | |
endint | None | Ending index, defaulting to length. | |
validatebool | If True, validate UTF-8 before processing. Default: False. | |
capabilitiesCapability | None | Capabilities to run, by default the CPU's enabled ones. |
>>> sz.utf8_uncased_search('Hello World', 'WORLD') # str: codepoint offset
6
>>> sz.utf8_uncased_search('Straße', 'STRASSE') # 'ß' = 1 codepoint
0
>>> sz.utf8_uncased_search(b'Stra\xc3\x9fe', b'STRASSE') # 'ß' = 2 bytes
0
utf8_uncased_orderCompare two UTF-8 strings uncasedly.def utf8_uncased_order(
a: _StringLike,
b: _StringLike,
validate: bool = False,
*,
capabilities: Capability | None = None,
) -> intPerforms lexicographical comparison using Unicode case folding, correctly handling one-to-many expansions, like 'Straße' equaling 'STRASSE'.
a_StringLike | First string to compare. | |
|---|---|---|
b_StringLike | Second string to compare. | |
validatebool | If True, validate UTF-8 before processing. Default: False. | |
capabilitiesCapability | None | Capabilities to run, by default the CPU's enabled ones. |
>>> sz.utf8_uncased_order('hello', 'HELLO')
0
>>> sz.utf8_uncased_order('apple', 'BANANA')
-1
utf8_uncased_matchesIterate over all uncased matches of needle in haystack.def utf8_uncased_matches(
haystack: _StringLike,
needle: _StringLike,
/,
include_overlapping: bool = False,
*,
capabilities: Capability | None = None,
) -> Utf8UncasedMatchesThis function uses Unicode case folding for proper handling of international text. The matched region length may differ from the needle length due to case folding expansions, like 'ß' matching 'SS'. An empty needle matches at every codepoint boundary, the end included.
haystack_StringLike | The string to search in. | |
|---|---|---|
needle_StringLike | The pattern to find. | |
include_overlappingbool | Allow overlapping matches, defaulting to False. | |
capabilitiesCapability | None | Capabilities to run, by default the CPU's enabled ones. |
Utf8UncasedMatches: Each matched region as a view into the original haystack.>>> list(sz.utf8_uncased_matches('Hello HELLO hello', 'hello'))
[sz.String('Hello'), sz.String('HELLO'), sz.String('hello')]
>>> list(sz.utf8_uncased_matches('Straße STRASSE', 'strasse'))
[sz.String('Straße'), sz.String('STRASSE')]