# Case Folding and Normalization

> StringZilla for Python

`import stringzilla`

## Utf8UncasedMatches

Iterator yielding all uncased matches of needle in haystack.

```python
class Utf8UncasedMatches
```

Uses Unicode case folding for proper handling of international text.

Each iteration yields a String view of the matched region in the haystack.
The matched length may differ from needle length due to case folding
expansions, like German 'ß' matching 'SS'.

Created by: - String.utf8_uncased_matches()

- sz.utf8_uncased_matches()

Examples: ```
>>> len(list(sz.utf8_uncased_matches('aAaA', 'a')))
4
```

## utf8_uncased_fold

Apply Unicode case folding to a UTF-8 string.

```python
def utf8_uncased_fold(
    text: _StringLike,
    validate: bool = False,
    *,
    capabilities: Capability | None = None,
) -> bytes
```

Case folding normalizes text for uncased comparisons,
handling one-to-many expansions, like German sharp S to 'ss'.

- `text`: The input UTF-8 string.
- `validate`: If True, validate UTF-8 before processing. Default: False.
- `capabilities`: Capabilities to run, by default the CPU's enabled ones.

Examples: ```
>>> sz.utf8_uncased_fold('HELLO')
b'hello'
>>> sz.utf8_uncased_fold('Stra\u00dfe')  # German sharp S
b'strasse'
```

## utf8_norm

Normalize a UTF-8 string to a Unicode normalization form.

```python
def utf8_norm(
    text: _StringLike,
    form: str,
    validate: bool = False,
    *,
    capabilities: Capability | None = None,
) -> bytes
```

Transforms the input into NFD, NFC, NFKD, or NFKC, mirroring [`unicodedata.normalize`](https://www.unum.cloud/docs/stringzilla/python/types.md#normalize) but
returning raw UTF-8 bytes, or into NFKC_Casefold, the Unicode toNFKC_Casefold: NFKC with full
case folding and no default-ignorable codepoints.

- `text`: The input UTF-8 string.
- `form`: One of 'NFC', 'NFD', 'NFKC', 'NFKD', 'NFKC_Casefold'.
- `validate`: If True, validate UTF-8 before processing. Default: False.
- `capabilities`: Capabilities to run, by default the CPU's enabled ones.

Examples: ```
>>> sz.utf8_norm('caf\u00e9', 'NFD')  # decompose
b'cafe\xcc\x81'
>>> sz.utf8_norm('\ufb01', 'NFKD')  # fi ligature
b'fi'
>>> sz.utf8_norm('Stra\u00dfe\u00ad', 'NFKC_Casefold')  # folded, soft hyphen dropped
b'strasse'
```

## utf8_find_denormalized

Return the byte offset of the first normalization violation, or None.

```python
def utf8_find_denormalized(
    text: _StringLike,
    form: str,
    *,
    capabilities: Capability | None = None,
) -> int | None
```

Scans the UTF-8 string and returns the byte offset of the first codepoint
that breaks the given normalization form (first non-Yes Quick-Check result
or canonical-ordering violation). Returns None when the string is already
fully normalized.

- `text`: The input UTF-8 string.
- `form`: One of 'NFC', 'NFD', 'NFKC', 'NFKD', 'NFKC_Casefold'.
- `capabilities`: Capabilities to run, by default the CPU's enabled ones.

Examples: ```
>>> sz.utf8_find_denormalized('cafe\u0301', 'NFC')  # 'e' + combining acute
3
>>> sz.utf8_find_denormalized('caf\u00e9', 'NFC') is None  # precomposed, already NFC
True
```

## utf8_uncased_search

Find substring using Unicode uncased matching.

```python
def utf8_uncased_search(
    haystack: _StringLike,
    needle: _StringLike,
    start: int = 0,
    end: int | None = None,
    validate: bool = False,
    *,
    capabilities: Capability | None = None,
) -> int
```

Performs a uncased search using Unicode case folding rules,
correctly handling one-to-many expansions, like 'ß' matching 'SS'.

- `haystack`: The string to search in.
- `needle`: The substring to find.
- `start`: Starting index, defaulting to 0.
- `end`: Ending index, defaulting to length.
- `validate`: If True, validate UTF-8 before processing. Default: False.
- `capabilities`: Capabilities to run, by default the CPU's enabled ones.

IMPORTANT - Type-dependent behavior: - str input:   start/end are CODEPOINT offsets, returns CODEPOINT offset

- bytes input: start/end are BYTE offsets, returns BYTE offset

Examples: ```
>>> sz.utf8_uncased_search('Hello World', 'WORLD')  # str: codepoint offset
6
>>> sz.utf8_uncased_search('Straße', 'STRASSE')  # 'ß' = 1 codepoint
0
>>> sz.utf8_uncased_search(b'Stra\xc3\x9fe', b'STRASSE')  # 'ß' = 2 bytes
0
```

## utf8_uncased_order

Compare two UTF-8 strings uncasedly.

```python
def utf8_uncased_order(
    a: _StringLike,
    b: _StringLike,
    validate: bool = False,
    *,
    capabilities: Capability | None = None,
) -> int
```

Performs lexicographical comparison using Unicode case folding, correctly handling
one-to-many expansions, like 'Straße' equaling 'STRASSE'.

- `a`: First string to compare.
- `b`: Second string to compare.
- `validate`: If True, validate UTF-8 before processing. Default: False.
- `capabilities`: Capabilities to run, by default the CPU's enabled ones.

Examples: ```
>>> sz.utf8_uncased_order('hello', 'HELLO')
0
>>> sz.utf8_uncased_order('apple', 'BANANA')
-1
```

## utf8_uncased_matches

Iterate over all uncased matches of needle in haystack.

```python
def utf8_uncased_matches(
    haystack: _StringLike,
    needle: _StringLike,
    /,
    include_overlapping: bool = False,
    *,
    capabilities: Capability | None = None,
) -> Utf8UncasedMatches
```

This function uses Unicode case folding for proper handling of
international text. The matched region length may differ from the
needle length due to case folding expansions, like 'ß' matching 'SS'.
An empty needle matches at every codepoint boundary, the end included.

- `haystack`: The string to search in.
- `needle`: The pattern to find.
- `include_overlapping`: Allow overlapping matches, defaulting to False.
- `capabilities`: Capabilities to run, by default the CPU's enabled ones.

Yields: - [`Utf8UncasedMatches`](#Utf8UncasedMatches): Each matched region as a view into the original haystack.

Examples: ```
>>> list(sz.utf8_uncased_matches('Hello HELLO hello', 'hello'))
[sz.String('Hello'), sz.String('HELLO'), sz.String('hello')]
>>> list(sz.utf8_uncased_matches('Straße STRASSE', 'strasse'))
[sz.String('Straße'), sz.String('STRASSE')]
```