# Searching and Counting

> StringZilla for C

The forward and reverse substring scans mirror libc's `memchr`/`memrchr` and `memmem`, but always take explicit lengths and report a pointer into the haystack, or `NULL` when there is no match.
Like every dispatch point, each takes the mask to pick a capability from and a stream, null on the CPU, and returns `sz_success_k`, or `sz_missing_kernel_k` when no capability in the mask has the kernel:

```c
sz_status_t sz_find_byte_best(sz_cptr_t haystack, sz_size_t haystack_length, sz_cptr_t needle, //
                              sz_cptr_t *match, sz_capability_t capabilities, sz_stream_t stream);
sz_status_t sz_rfind_byte_best(sz_cptr_t haystack, sz_size_t haystack_length, sz_cptr_t needle, //
                               sz_cptr_t *match, sz_capability_t capabilities, sz_stream_t stream);
sz_status_t sz_find_best(sz_cptr_t haystack, sz_size_t haystack_length, sz_cptr_t needle, sz_size_t needle_length,
                         sz_cptr_t *match, sz_capability_t capabilities, sz_stream_t stream);
sz_status_t sz_rfind_best(sz_cptr_t haystack, sz_size_t haystack_length, sz_cptr_t needle, sz_size_t needle_length,
                          sz_cptr_t *match, sz_capability_t capabilities, sz_stream_t stream);
```

Byte-set scans replace `strspn`/`strcspn` and take a prebuilt `sz_byteset_t`:

```c
sz_status_t sz_find_byteset_best(sz_cptr_t haystack, sz_size_t haystack_length, sz_byteset_t const *set, //
                                 sz_cptr_t *match, sz_capability_t capabilities, sz_stream_t stream);
sz_status_t sz_rfind_byteset_best(sz_cptr_t haystack, sz_size_t haystack_length, sz_byteset_t const *set, //
                                  sz_cptr_t *match, sz_capability_t capabilities, sz_stream_t stream);
```

Four header-only shortcuts build the set for you from a needle string, optionally inverting it:

```c
sz_cptr_t sz_find_byte_from(sz_cptr_t haystack, sz_size_t haystack_length, sz_cptr_t needle, sz_size_t needle_length);
sz_cptr_t sz_find_byte_not_from(sz_cptr_t haystack, sz_size_t haystack_length, sz_cptr_t needle, sz_size_t needle_length);
sz_cptr_t sz_rfind_byte_from(sz_cptr_t haystack, sz_size_t haystack_length, sz_cptr_t needle, sz_size_t needle_length);
sz_cptr_t sz_rfind_byte_not_from(sz_cptr_t haystack, sz_size_t haystack_length, sz_cptr_t needle, sz_size_t needle_length);
```

Example: find a substring, then count occurrences by re-scanning the tail.

```c
#include <stringzilla/stringzilla.h>

sz_size_t count_occurrences(sz_cptr_t text, sz_size_t length, sz_cptr_t needle, sz_size_t needle_length,
                            sz_capability_t capabilities) {
    sz_size_t total = 0;
    sz_cptr_t cursor = text, end = text + length, match = NULL;
    while (sz_find_best(cursor, (sz_size_t)(end - cursor), needle, needle_length, &match, capabilities, NULL) ==
               sz_success_k && match) {
        ++total;
        cursor = match + needle_length;      // disjoint matches; use `+1` for overlapping
    }
    return total;
}

int main(void) {
    sz_capability_t capabilities = sz_cap_serial_k;
    sz_capabilities_enabled_cpu(&capabilities);

    sz_byteset_t whitespace;
    sz_byteset_init(&whitespace);
    char const *spaces = " \t\n\r\v\f";
    for (char const *p = spaces; *p; ++p) sz_byteset_add(&whitespace, *p);

    char const *line = "  hello world";
    sz_cptr_t first_space = NULL;
    sz_find_byteset_best(line, 13, &whitespace, &first_space, capabilities, NULL);
    (void)first_space;
    return 0;
}
```

`#include <stringzilla/types.h>`

`#include <stringzilla/capabilities.h>`

`#include <stringzilla/find.h>`

`#include <stringzilla/substrings.h>`

`#include <stringzilla/regex.h>`

`#include <stringzilla/subdocuments.h>`

`#include <stringzilla/utf8_runes.h>`

## sz_match_t

One reported match, locating it by haystack, by needle, and by byte span.

```c
struct sz_match_t
```

- `haystack_index` `sz_size_t`: Which haystack of the collection this match was found in.
- `needle_index` `sz_size_t`: Which needle of the vocabulary matched: a substring, a pattern or a path.
- `byte_offset` `sz_size_t`: Where the match starts inside that haystack, in its own bytes.
- `byte_length` `sz_size_t`: Haystack bytes the match spans, which folding can make differ from the needle's length.

## sz_subdocuments_format_t

The document grammar a subdocuments engine reads, and the path language that addresses it.

```c
enum sz_subdocuments_format_t
```

- `sz_subdocuments_json_k` = `0`: RFC 8259 documents, addressed by RFC 9535 JSONPath.
- `sz_subdocuments_xml_k` = `1`: XML 1.0 documents, addressed by XPath 1.0 along forward axes.

## sz_subdocuments_options_t

How a subdocuments engine reads what RFC 8259 leaves open, combined by bitwise or.

```c
enum sz_subdocuments_options_t
```

- `sz_subdocuments_options_none_k` = `0`: Every member a name selector names is entered.
- `sz_subdocuments_duplicate_names_first_k` = `1`: A name selector enters only the first member of an object with that name, so each field a path names is at most one node.

## sz_substrings_overlap_policy_t

How matches that share bytes resolve: reported in full, or thinned to a leftmost run.

```c
enum sz_substrings_overlap_policy_t
```

- `sz_substrings_overlapping_k` = `0`: Every match of every needle, including ones that share bytes and ones nested in others.
- `sz_substrings_leftmost_longest_k` = `1`: Matches sharing no bytes: earliest start, then longest span, then lower needle index.
- `sz_substrings_leftmost_first_k` = `2`: Matches sharing no bytes: earliest start, then lower needle index, whatever the lengths.

## sz_kernel_utf8_count_t

Signature of the kernels behind [`sz_utf8_count_best`](#sz_utf8_count_best).

```c
typedef sz_status_t(*) sz_kernel_utf8_count_t(sz_cptr_t, sz_size_t, sz_size_t *, sz_stream_t);
```

## sz_kernel_utf8_count_strings_t

Signature of the kernels behind [`sz_utf8_count_strings_best`](#sz_utf8_count_strings_best).

```c
typedef sz_status_t(*) sz_kernel_utf8_count_strings_t(sz_strings_view_t const *, sz_size_t const *, sz_size_t, sz_size_t *, sz_size_t *, sz_stream_t);
```

## sz_kernel_find_byte_t

Signature of the [`sz_find_byte_best`](#sz_find_byte_best) and [`sz_rfind_byte_best`](#sz_rfind_byte_best) kernels.

```c
typedef sz_status_t(*) sz_kernel_find_byte_t(sz_cptr_t, sz_size_t, sz_cptr_t, sz_cptr_t *, sz_stream_t);
```

## sz_kernel_find_t

Signature of the [`sz_find_best`](#sz_find_best) and [`sz_rfind_best`](#sz_rfind_best) kernels.

```c
typedef sz_status_t(*) sz_kernel_find_t(sz_cptr_t, sz_size_t, sz_cptr_t, sz_size_t, sz_cptr_t *, sz_stream_t);
```

## sz_kernel_find_byteset_t

Signature of the [`sz_find_byteset_best`](#sz_find_byteset_best) and [`sz_rfind_byteset_best`](#sz_rfind_byteset_best) kernels.

```c
typedef sz_status_t(*) sz_kernel_find_byteset_t(sz_cptr_t, sz_size_t, sz_byteset_t const *, sz_cptr_t *, sz_stream_t);
```

## sz_kernel_find_strings_t

Signature of the [`sz_find_strings_best`](#sz_find_strings_best) kernels.

```c
typedef sz_status_t(*) sz_kernel_find_strings_t(sz_strings_view_t const *, sz_size_t const *, sz_size_t, sz_cptr_t, sz_size_t, sz_size_t *, sz_stream_t);
```

## sz_kernel_substrings_engine_init_t

Signature of every [`sz_substrings_engine_init`](#sz_substrings_engine_init) kernel.

```c
typedef sz_status_t(*) sz_kernel_substrings_engine_init_t(struct sz_substrings_engine_t *, sz_strings_view_t const *, sz_case_sensitivity_t, sz_substrings_overlap_policy_t, sz_size_t, sz_size_t, sz_size_t, sz_allocator_t *, sz_stream_t);
```

## sz_kernel_substrings_contains_t

Signature of every [`sz_substrings_contains`](#sz_substrings_contains) kernel.

```c
typedef sz_status_t(*) sz_kernel_substrings_contains_t(struct sz_substrings_engine_t *, sz_strings_view_t const *, sz_u1x8_t *, sz_stream_t);
```

## sz_kernel_substrings_counts_t

Signature of every [`sz_substrings_counts`](#sz_substrings_counts) kernel.

```c
typedef sz_status_t(*) sz_kernel_substrings_counts_t(struct sz_substrings_engine_t *, sz_strings_view_t const *, sz_size_t *, sz_size_t, sz_stream_t);
```

## sz_kernel_substrings_find_t

Signature of every [`sz_substrings_find`](#sz_substrings_find) kernel.

```c
typedef sz_status_t(*) sz_kernel_substrings_find_t(struct sz_substrings_engine_t *, sz_strings_view_t const *, sz_match_t *, sz_size_t, sz_size_t *, sz_stream_t);
```

## sz_kernel_substrings_replace_t

Signature of every [`sz_substrings_replace`](#sz_substrings_replace) kernel.

```c
typedef sz_status_t(*) sz_kernel_substrings_replace_t(struct sz_substrings_engine_t *, sz_strings_view_t const *, sz_strings_view_t const *, sz_ptr_t, sz_size_t, sz_size_t *, sz_stream_t);
```

## sz_kernel_substrings_bm25_scores_t

Signature of every [`sz_substrings_bm25_scores`](#sz_substrings_bm25_scores) kernel.

```c
typedef sz_status_t(*) sz_kernel_substrings_bm25_scores_t(struct sz_substrings_engine_t *, sz_strings_view_t const *, sz_f32_t const *, struct sz_substrings_bm25_t const *, sz_f32_t const *, sz_f32_t *, sz_size_t, sz_stream_t);
```

## sz_kernel_regex_engine_init_t

Signature of every [`sz_regex_engine_init`](#sz_regex_engine_init) kernel.

```c
typedef sz_status_t(*) sz_kernel_regex_engine_init_t(struct sz_regex_engine_t *, sz_strings_view_t const *, sz_case_sensitivity_t, sz_size_t *, sz_allocator_t *, sz_stream_t);
```

## sz_kernel_regex_contains_t

Signature of every [`sz_regex_contains`](#sz_regex_contains) kernel.

```c
typedef sz_status_t(*) sz_kernel_regex_contains_t(struct sz_regex_engine_t *, sz_strings_view_t const *, sz_u1x8_t *, sz_stream_t);
```

## sz_kernel_regex_counts_t

Signature of every [`sz_regex_counts`](#sz_regex_counts) kernel.

```c
typedef sz_status_t(*) sz_kernel_regex_counts_t(struct sz_regex_engine_t *, sz_strings_view_t const *, sz_size_t *, sz_size_t, sz_stream_t);
```

## sz_kernel_regex_find_t

Signature of every [`sz_regex_find`](#sz_regex_find) kernel.

```c
typedef sz_status_t(*) sz_kernel_regex_find_t(struct sz_regex_engine_t *, sz_strings_view_t const *, sz_match_t *, sz_size_t, sz_size_t *, sz_stream_t);
```

## sz_kernel_regex_replace_t

Signature of every [`sz_regex_replace`](#sz_regex_replace) kernel.

```c
typedef sz_status_t(*) sz_kernel_regex_replace_t(struct sz_regex_engine_t *, sz_strings_view_t const *, sz_strings_view_t const *, sz_ptr_t, sz_size_t, sz_size_t *, sz_stream_t);
```

## sz_kernel_regex_extract_t

Signature of every [`sz_regex_extract`](#sz_regex_extract) kernel.

```c
typedef sz_status_t(*) sz_kernel_regex_extract_t(struct sz_regex_engine_t *, sz_strings_view_t const *, sz_size_t, sz_match_t *, sz_size_t, sz_size_t *, sz_stream_t);
```

## sz_kernel_subdocuments_engine_init_t

Signature of every [`sz_subdocuments_engine_init`](#sz_subdocuments_engine_init) kernel.

```c
typedef sz_status_t(*) sz_kernel_subdocuments_engine_init_t(struct sz_subdocuments_engine_t *, sz_strings_view_t const *, sz_subdocuments_format_t, sz_subdocuments_options_t, sz_size_t *, sz_allocator_t *, sz_stream_t);
```

## sz_kernel_subdocuments_contains_t

Signature of every [`sz_subdocuments_contains`](#sz_subdocuments_contains) kernel.

```c
typedef sz_status_t(*) sz_kernel_subdocuments_contains_t(struct sz_subdocuments_engine_t *, sz_strings_view_t const *, sz_u1x8_t *, sz_size_t *, sz_stream_t);
```

## sz_kernel_subdocuments_counts_t

Signature of every [`sz_subdocuments_counts`](#sz_subdocuments_counts) kernel.

```c
typedef sz_status_t(*) sz_kernel_subdocuments_counts_t(struct sz_subdocuments_engine_t *, sz_strings_view_t const *, sz_size_t *, sz_size_t, sz_size_t *, sz_stream_t);
```

## sz_kernel_subdocuments_find_t

Signature of every [`sz_subdocuments_find`](#sz_subdocuments_find) kernel.

```c
typedef sz_status_t(*) sz_kernel_subdocuments_find_t(struct sz_subdocuments_engine_t *, sz_strings_view_t const *, sz_match_t *, sz_size_t, sz_size_t *, sz_size_t *, sz_stream_t);
```

## sz_subdocuments_slice_t

The array positions one selector picks, RFC 9535's `[begin:end:step]`, an index `n` being `[n:n+1:1]`, or `[-1::1]` for the last element; negative bounds count from the array's end.

```c
struct sz_subdocuments_slice_t
```

- `begin` `sz_i64_t`: The first position, or 2⁵³ past the end that a negative step walks from when omitted.
- `end` `sz_i64_t`: The position the slice stops before, or 2⁵³ past the end its step walks to when omitted.
- `step` `sz_i64_t`: Distance between selected positions, negative walking backward; zero selects none.

## sz_find_byte_best

Locates the first byte equal to the one at `needle` in `haystack`, like `memchr`.

```c
sz_status_t sz_find_byte_best(
    sz_cptr_t haystack,
    sz_size_t haystack_length,
    sz_cptr_t needle,
    sz_cptr_t *match,
    sz_capability_t capabilities,
    sz_stream_t stream);
```

- `sz_find_byte` backends: serial, westmere, haswell, skylake, neon, sve, rvv, v128, v128relaxed, loongsonasx, powervsx

- `haystack` (in): Haystack - the string to search in.
- `haystack_length` (in): Number of bytes in the haystack.
- `needle` (in): Needle - single-byte substring to find.
- `match` (out): Address of the first match, or NULL if not found.
- `capabilities` (in): One device's capabilities, like `sz_capabilities_enabled_cpu` reports.
- `stream` (in): Null on the CPU, or the GPU stream of that device to queue on.

See also: x86_64 implementation in glibc: [https://github.com/lattera/glibc/blob/master/sysdeps/x86_64/memchr.S](https://github.com/lattera/glibc/blob/master/sysdeps/x86_64/memchr.S)

See also: AArch64 implementation in glibc: [https://github.com/lattera/glibc/blob/master/sysdeps/aarch64/memchr.S](https://github.com/lattera/glibc/blob/master/sysdeps/aarch64/memchr.S)

Statuses: - [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) No capability in `capabilities` has it.

## sz_rfind_byte_best

Locates the last byte equal to the one at `needle` in `haystack`, like `memrchr`.

```c
sz_status_t sz_rfind_byte_best(
    sz_cptr_t haystack,
    sz_size_t haystack_length,
    sz_cptr_t needle,
    sz_cptr_t *match,
    sz_capability_t capabilities,
    sz_stream_t stream);
```

- `sz_rfind_byte` backends: serial, westmere, haswell, skylake, neon, sve, rvv, v128, v128relaxed, loongsonasx, powervsx

There is no AArch64 reference implementation to link here.

- `haystack` (in): Haystack - the string to search in.
- `haystack_length` (in): Number of bytes in the haystack.
- `needle` (in): Needle - single-byte substring to find.
- `match` (out): Address of the last match, or NULL if not found.
- `capabilities` (in): One device's capabilities, like `sz_capabilities_enabled_cpu` reports.
- `stream` (in): Null on the CPU, or the GPU stream of that device to queue on.

See also: x86_64 implementation in glibc: [https://github.com/lattera/glibc/blob/master/sysdeps/x86_64/memrchr.S](https://github.com/lattera/glibc/blob/master/sysdeps/x86_64/memrchr.S)

Statuses: - [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) No capability in `capabilities` has it.

## sz_find_best

Locates the first occurrence of `needle` in `haystack`, like `memmem` in LibC, or like `strstr` for strings of known length.

```c
sz_status_t sz_find_best(
    sz_cptr_t haystack,
    sz_size_t haystack_length,
    sz_cptr_t needle,
    sz_size_t needle_length,
    sz_cptr_t *match,
    sz_capability_t capabilities,
    sz_stream_t stream);
```

- `sz_find` backends: serial, westmere, haswell, skylake, neon, sve, rvv, v128, v128relaxed, loongsonasx, powervsx

- `haystack` (in): Haystack - the string to search in.
- `haystack_length` (in): Number of bytes in the haystack.
- `needle` (in): Needle - substring to find.
- `needle_length` (in): Number of bytes in the needle.
- `match` (out): Address of the first match, or NULL if not found.
- `capabilities` (in): One device's capabilities, like `sz_capabilities_enabled_cpu` reports.
- `stream` (in): Null on the CPU, or the GPU stream of that device to queue on.

Statuses: - [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) No capability in `capabilities` has it.

## sz_rfind_best

Locates the last matching substring.

```c
sz_status_t sz_rfind_best(
    sz_cptr_t haystack,
    sz_size_t haystack_length,
    sz_cptr_t needle,
    sz_size_t needle_length,
    sz_cptr_t *match,
    sz_capability_t capabilities,
    sz_stream_t stream);
```

- `sz_rfind` backends: serial, westmere, haswell, skylake, neon, sve, rvv, v128, v128relaxed, loongsonasx, powervsx

- `haystack` (in): Haystack - the string to search in.
- `haystack_length` (in): Number of bytes in the haystack.
- `needle` (in): Needle - substring to find.
- `needle_length` (in): Number of bytes in the needle.
- `match` (out): Address of the last match, or NULL if not found.
- `capabilities` (in): One device's capabilities, like `sz_capabilities_enabled_cpu` reports.
- `stream` (in): Null on the CPU, or the GPU stream of that device to queue on.

Statuses: - [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) No capability in `capabilities` has it.

## sz_find_byteset_best

Finds the first character present from the `set`, present in `haystack`.

```c
sz_status_t sz_find_byteset_best(
    sz_cptr_t haystack,
    sz_size_t haystack_length,
    sz_byteset_t const *set,
    sz_cptr_t *match,
    sz_capability_t capabilities,
    sz_stream_t stream);
```

- `sz_find_byteset` backends: serial, haswell, icelake, neon, sve2, rvv, v128, v128relaxed, loongsonasx, powervsx

Equivalent to `strspn(haystack, accepted)` and `strcspn(haystack, rejected)` in LibC. May have identical implementation and performance to [`sz_rfind_byteset_best`](#sz_rfind_byteset_best). Useful for parsing, when we want to skip a set of characters, such as:

- 6 whitespaces: " \t\n\r\v\f".
- 16 digits forming a float number: "0123456789,.eE+-".
- 5 HTML reserved characters: "\"'&<>", of which "<>" can be useful for parsing.
- 2 JSON string special characters useful to locate the end of the string: "\"\\".

- `haystack` (in): Haystack - the string to scan.
- `haystack_length` (in): Number of bytes in the haystack.
- `set` (in): Set of relevant characters.
- `match` (out): Address of the first matching character from `set`, or NULL if not found.
- `capabilities` (in): One device's capabilities, like `sz_capabilities_enabled_cpu` reports.
- `stream` (in): Null on the CPU, or the GPU stream of that device to queue on.

Statuses: - [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) No capability in `capabilities` has it.

## sz_rfind_byteset_best

Finds the last character present from the `set`, present in `haystack`.

```c
sz_status_t sz_rfind_byteset_best(
    sz_cptr_t haystack,
    sz_size_t haystack_length,
    sz_byteset_t const *set,
    sz_cptr_t *match,
    sz_capability_t capabilities,
    sz_stream_t stream);
```

- `sz_rfind_byteset` backends: serial, haswell, icelake, neon, sve2, rvv, v128, v128relaxed, loongsonasx, powervsx

Equivalent to `strspn(haystack, accepted)` and `strcspn(haystack, rejected)` in LibC. May have identical implementation and performance to [`sz_find_byteset_best`](#sz_find_byteset_best). Useful for parsing, when we want to skip a set of characters, such as:

- 6 whitespaces: " \t\n\r\v\f".
- 16 digits forming a float number: "0123456789,.eE+-".
- 5 HTML reserved characters: "\"'&<>", of which "<>" can be useful for parsing.
- 2 JSON string special characters useful to locate the end of the string: "\"\\".

- `haystack` (in): Haystack - the string to scan.
- `haystack_length` (in): Number of bytes in the haystack.
- `set` (in): Set of relevant characters.
- `match` (out): Address of the last matching character from `set`, or NULL if not found.
- `capabilities` (in): One device's capabilities, like `sz_capabilities_enabled_cpu` reports.
- `stream` (in): Null on the CPU, or the GPU stream of that device to queue on.

Statuses: - [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) No capability in `capabilities` has it.

## sz_find_strings_best

Finds `needle` in every one of `strings[order[i]]`, or of the first `count` strings, like a `LIKE` 'needle' filter over a column.

```c
sz_status_t sz_find_strings_best(
    sz_strings_view_t const *strings,
    sz_size_t const *order,
    sz_size_t count,
    sz_cptr_t needle,
    sz_size_t needle_length,
    sz_size_t *positions,
    sz_capability_t capabilities,
    sz_stream_t stream);
```

- `sz_find_strings` backends: serial, haswell, skylake, neon

A tape read in its own order is searched as one run of bytes, 64 starts at a time, so a string costs no call of its own, and a match crossing into the next string counts for neither. Other layouts and orders search string by string.

- `strings` (in): The strings to search, in any layout.
- `order` (in): Indices into `strings`, possibly repeated, or null for the first `count`.
- `count` (in): Number of strings to search.
- `needle` (in): The substring to find.
- `needle_length` (in): Number of bytes in the needle, zero matching at every start.
- `positions` (out): The offset of each first match within its string, or `STRINGZILLA_SIZE_MAX` where a string holds none, `count` of them.
- `capabilities` (in): One device's capabilities, like `sz_capabilities_enabled_cpu` reports.
- `stream` (in): Null on the CPU, or the GPU stream of that device to queue on.

Statuses: - [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) No capability in `capabilities` has it.

## sz_find_find_kernel

Finds the search kernel of `kind`, from the best of `capabilities`.

```c
sz_status_t sz_find_find_kernel(
    sz_kernel_kind_t kind,
    sz_capability_t capabilities,
    sz_kernel_punned_t *kernel,
    sz_capability_t *capability);
```

- `kind` (in): The verb whose kernel to find.
- `capabilities` (in): The mask to pick the best capability from.
- `kernel` (out): The kernel, or null when none of `capabilities` has it.
- `capability` (out): The capability the kernel belongs to, or zero.

Statuses: - [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k)
- [`sz_missing_library_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_library_k) Header-only.

## sz_substrings_engine_init

Compiles `needles` for the best capability of `capabilities` that has an init kernel.

```c
sz_status_t sz_substrings_engine_init(
    sz_substrings_engine_t *engine,
    sz_strings_view_t const *needles,
    sz_case_sensitivity_t case_sensitivity,
    sz_substrings_overlap_policy_t overlap_policy,
    sz_size_t hot_states,
    sz_size_t matches_budget,
    sz_size_t haystacks_budget,
    sz_capability_t capabilities,
    sz_allocator_t *allocator,
    sz_stream_t stream);
```

- `sz_substrings_engine_init` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, hopper, rocm, metal

- `engine` (out): Left untouched unless the call succeeds.
- `needles` (in): The vocabulary; an empty needle is refused rather than skipped, as it would match at every position and dropping it would shift every later needle's reported index. Read on the host.
- `case_sensitivity` (in): Whether both sides are folded before they meet, or compared as is.
- `overlap_policy` (in): The cover every round runs, as the arena is sized for that one alone.
- `hot_states` (in): States to keep in the dense hot rows, or STRINGZILLA_SUBSTRINGS_HOT_STATES_AUTO to fill a fixed byte budget, which holds more states the fewer classes the vocabulary spells.
- `matches_budget` (in): Matches one round may emit, read by a device tier only; a host tier walks straight into the caller's output and ignores it. Zero asks for a tier-chosen default.
- `haystacks_budget` (in): Haystacks one round may carry, read by a device tier only, zero asking for its default. The arena of counts, find and replace is sized for it here, so they allocate nothing; [sz_substrings_bm25_scores](#sz_substrings_bm25_scores) grows it for its tables on its first call only. A round carrying more is refused with [`sz_unexpected_dimensions_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_unexpected_dimensions_k).
- `capabilities` (in): One device's capabilities, like `sz_capabilities_enabled_cpu` or [`sz_capabilities_enabled_cuda`](https://www.unum.cloud/docs/stringzilla/c/runtime-dispatch.md#sz_capabilities_enabled_cuda) report; its group picks the CPU or a GPU vendor.
- `allocator` (in): Where both blocks come from, or [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) for [sz_allocator_init_unified_best](https://www.unum.cloud/docs/stringzilla/c/memory-operations.md#sz_allocator_init_unified_best) of `capabilities`. Stored by value, so [sz_substrings_engine_free](#sz_substrings_engine_free) needs none and cannot get the wrong one; on a device it must be host-writable, as the host writes the automaton there.
- `stream` (in): Null on the CPU. On a GPU, the stream to queue on, also naming the device: a `cudaStream_t`, a `hipStream_t`, or an `id<MTLCommandQueue>`; null for the default.

Notes: May join `stream`; no compute verb ever does.

Statuses: - [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) No capability of the mask has an init.
- [`sz_overflow_risk_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_overflow_risk_k) The vocabulary exceeds a 32-bit state id.
- [`sz_invalid_utf8_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_invalid_utf8_k) A malformed needle under [sz_uncased_k](https://www.unum.cloud/docs/stringzilla/c/case-folding-and-normalization.md#sz_uncased_k).
- [`sz_unexpected_dimensions_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_unexpected_dimensions_k) The vocabulary or one of its needles was empty.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) On a device, a block or a `stream` it cannot use.

## sz_*_engine_free

Returns both of the engine's blocks to the allocator that built them, once the work queued on `stream` is done with them, emptying `engine`.

```c
void sz_substrings_engine_free(
    sz_substrings_engine_t *engine,
    sz_stream_t stream);
```

- `sz_substrings_engine_free`: engine `sz_substrings_engine_t`
- `sz_regex_engine_free`: engine `sz_regex_engine_t`
- `sz_subdocuments_engine_free`: engine `sz_subdocuments_engine_t`

## sz_substrings_contains

Marks which haystacks hold at least one match of any needle, stopping each walk at its first match, the lightest verb of the engine.

```c
sz_status_t sz_substrings_contains(
    sz_substrings_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_u1x8_t *contained,
    sz_stream_t stream);
```

- `sz_substrings_contains` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, rocm

- `engine` (in): A compiled vocabulary; a match under any overlap policy is a match.
- `haystacks` (in): The texts to search; their bytes are read in place and never copied.
- `contained` (out): One bit per haystack, eight to a byte, the lowest bit first: set when that haystack matches, with the bits past the last haystack written as zero.
- `stream` (in): As [sz_substrings_counts](#sz_substrings_counts) takes it.

Statuses: - [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) A device got an argument or a `stream` that it has no way to use.

## sz_substrings_counts

Counts the matches of every needle in every haystack, one count per haystack.

```c
sz_status_t sz_substrings_counts(
    sz_substrings_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_size_t *counts,
    sz_size_t counts_stride,
    sz_stream_t stream);
```

- `sz_substrings_counts` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, hopper, rocm, metal

Every verb runs the kernel of the capability `engine` was compiled for. A device one enqueues on `stream` and returns; every round of one engine runs out of its one arena and writes its one report, so the caller orders them - one stream, or events between two - and reads the report only after joining the round that wrote it.

- `engine` (in): A compiled vocabulary, whose `overlap_policy` decides whether matches sharing bytes are all counted or thinned to a leftmost run.
- `haystacks` (in): The texts to search; their bytes are read in place and never copied. Any layout on the CPU, and on a device a tape [sz_strings_to_tape_best](https://www.unum.cloud/docs/stringzilla/c/sorting-and-sets.md#sz_strings_to_tape_best) wrote into a unified allocator, as every verb takes its haystacks and replacements.
- `counts` (out): The per-haystack counts, haystack h at `counts[h * stride]`.
- `counts_stride` (in): Entries from one haystack's count to the next, at least one, so a strided call writes one column of a ____[haystacks, vocabularies] feature matrix.
- `stream` (in): Null on the CPU. On a GPU, the stream to queue on, which also names the device the round runs on; null for the default.

Statuses: - [`sz_unexpected_dimensions_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_unexpected_dimensions_k) `counts_stride` is zero.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) A device got an argument or a `stream` it cannot use.

## sz_substrings_find

Reports every match of every needle in every haystack.

```c
sz_status_t sz_substrings_find(
    sz_substrings_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_match_t *matches,
    sz_size_t matches_capacity,
    sz_size_t *matches_offsets,
    sz_stream_t stream);
```

- `sz_substrings_find` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, hopper, rocm, metal

A capacity too small is not an error: the report's `matches_emitted` names the true total and `shortfall` names what did not fit, so a sizing call and a filling call need no walk between.

- `engine` (in): A compiled vocabulary.
- `haystacks` (in): The texts to search; their bytes are read in place and never copied.
- `matches` (out): Room for `matches_capacity` matches, ascending by haystack; [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) together with a zero capacity makes the call a pure size query.
- `matches_capacity` (in): Entries `matches` holds.
- `matches_offsets` (out): One boundary per haystack into `matches` plus one, the last being the total; filled whether or not the matches fit, which is what sizes the next call.
- `stream` (in): As [sz_substrings_counts](#sz_substrings_counts) takes it.

Statuses: - [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) A device got an unaddressable argument.

## sz_substrings_replace

Rewrites every haystack, substituting one replacement per needle, into one `target`.

```c
sz_status_t sz_substrings_replace(
    sz_substrings_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_strings_view_t const *replacements,
    sz_ptr_t target,
    sz_size_t target_capacity,
    sz_size_t *offsets,
    sz_stream_t stream);
```

- `sz_substrings_replace` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, hopper, rocm, metal

A target too small is not an error: the report's `target_length` names the bytes the rewrite needs and `shortfall` names what did not fit, leaving the contents of `target` unspecified.

- `engine` (in): A vocabulary compiled under a leftmost policy; [sz_substrings_overlapping_k](#sz_substrings_overlapping_k) is refused, since a substitution over matches that share bytes is not a function.
- `haystacks` (in): The texts to rewrite; their bytes are read in place and never copied.
- `replacements` (in): One replacement per needle, indexed by needle; an empty one deletes.
- `target` (out): Room for `target_capacity` bytes, or [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) with zero capacity to size only.
- `target_capacity` (in): Bytes `target` holds.
- `offsets` (out): One rewritten boundary per haystack plus one, the last being the total; filled whether or not the target held the result, which is what sizes the next call.
- `stream` (in): As [sz_substrings_counts](#sz_substrings_counts) takes it.

Statuses: - [`sz_unexpected_dimensions_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_unexpected_dimensions_k) `replacements` lacks one entry per needle.
- [`sz_status_unknown_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_status_unknown_k) The policy leaves no cover.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) An argument no device kernel can address.

## sz_substrings_bm25_scores

Scores every haystack against the vocabulary as one BM25 query, one score per haystack.

```c
sz_status_t sz_substrings_bm25_scores(
    sz_substrings_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_f32_t const *document_lengths,
    sz_substrings_bm25_t const *parameters,
    sz_f32_t const *needle_weights,
    sz_f32_t *scores,
    sz_size_t scores_stride,
    sz_stream_t stream);
```

- `sz_substrings_bm25_scores` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, hopper, rocm, metal

The vocabulary is the query: `needle_weights` holds each needle's IDF or boost. Term frequencies are raw overlapping counts, since a leftmost cover would suppress genuine occurrences of a needle nested in another, so the engine's own policy does not apply here. A CPU backend sums in ascending needle order and a CUDA backend in fixed-point integers, so each is bit-stable across its own runs, and the two agree.

The arguments are invalid when `needle_weights` is [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL), `scores_stride` is zero, or `length_normalization` is positive while `average_document_length` is not.

- `engine` (in): A compiled vocabulary.
- `haystacks` (in): The documents to score; their bytes are read in place and never copied.
- `document_lengths` (in): One length per haystack to normalize by, in the unit of `average_document_length`, or [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) to use each haystack's byte length.
- `parameters` (in): BM25's continuous parameters.
- `needle_weights` (in): One weight per needle of the engine.
- `scores` (out): The per-haystack scores, haystack h at `scores[h * stride]`.
- `scores_stride` (in): Entries from one haystack's score to the next, at least one, so a strided call writes one column of a ____[haystacks, vocabularies] feature matrix.
- `stream` (in): As [sz_substrings_counts](#sz_substrings_counts) takes it.

Statuses: - [`sz_unexpected_dimensions_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_unexpected_dimensions_k) Invalid arguments, as above.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) An argument or grown arena a device cannot address.

## sz_substrings_engine_init_serial

Compiles `needles` for the serial kernels.

```c
sz_status_t sz_substrings_engine_init_serial(
    sz_substrings_engine_t *engine,
    sz_strings_view_t const *needles,
    sz_case_sensitivity_t case_sensitivity,
    sz_substrings_overlap_policy_t overlap_policy,
    sz_size_t hot_states,
    sz_size_t matches_budget,
    sz_size_t haystacks_budget,
    sz_allocator_t *allocator,
    sz_stream_t stream);
```

- `sz_substrings_engine_init` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, hopper, rocm, metal

- `engine` (out): Left untouched unless the call succeeds.
- `needles` (in): The vocabulary; an empty needle is refused rather than skipped, as it would match at every position and dropping it would shift every later needle's reported index. Read on the host.
- `case_sensitivity` (in): Whether both sides are folded before they meet, or compared as is.
- `overlap_policy` (in): The cover every round runs, as the arena is sized for that one alone.
- `hot_states` (in): States to keep in the dense hot rows, or STRINGZILLA_SUBSTRINGS_HOT_STATES_AUTO to fill a fixed byte budget, which holds more states the fewer classes the vocabulary spells.
- `matches_budget` (in): Matches one round may emit, read by a device tier only; the serial walk goes straight into the caller's output and ignores it.
- `haystacks_budget` (in): Haystacks one round may carry, read by a device tier only; the serial walk ignores it.
- `allocator` (in): Where both blocks come from, or [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) for `sz_allocator_init_unified_serial`. Stored by value, so [sz_substrings_engine_free](#sz_substrings_engine_free) needs none and cannot get the wrong one.
- `stream` (in): Null on the CPU.

Statuses: - [`sz_overflow_risk_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_overflow_risk_k) The vocabulary exceeds a 32-bit state id.
- [`sz_invalid_utf8_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_invalid_utf8_k) A malformed needle under [sz_uncased_k](https://www.unum.cloud/docs/stringzilla/c/case-folding-and-normalization.md#sz_uncased_k).
- [`sz_unexpected_dimensions_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_unexpected_dimensions_k) The vocabulary or one of its needles was empty.

## sz_substrings_contains_serial

Marks which haystacks hold at least one match of any needle, stopping each walk at its first match, the lightest verb of the engine.

```c
sz_status_t sz_substrings_contains_serial(
    sz_substrings_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_u1x8_t *contained,
    sz_stream_t stream);
```

- `sz_substrings_contains` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, rocm

- `engine` (in): A compiled vocabulary; a match under any overlap policy is a match.
- `haystacks` (in): The texts to search; their bytes are read in place and never copied.
- `contained` (out): One bit per haystack, eight to a byte, the lowest bit first: set when that haystack matches, with the bits past the last haystack written as zero.
- `stream` (in): As [sz_substrings_counts](#sz_substrings_counts) takes it.

Statuses: - [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) A device got an argument or a `stream` that it has no way to use.

## sz_substrings_counts_serial

Counts the matches of every needle in every haystack, one count per haystack.

```c
sz_status_t sz_substrings_counts_serial(
    sz_substrings_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_size_t *counts,
    sz_size_t counts_stride,
    sz_stream_t stream);
```

- `sz_substrings_counts` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, hopper, rocm, metal

Every verb runs the kernel of the capability `engine` was compiled for. A device one enqueues on `stream` and returns; every round of one engine runs out of its one arena and writes its one report, so the caller orders them - one stream, or events between two - and reads the report only after joining the round that wrote it.

- `engine` (in): A compiled vocabulary, whose `overlap_policy` decides whether matches sharing bytes are all counted or thinned to a leftmost run.
- `haystacks` (in): The texts to search; their bytes are read in place and never copied. Any layout on the CPU, and on a device a tape [sz_strings_to_tape_best](https://www.unum.cloud/docs/stringzilla/c/sorting-and-sets.md#sz_strings_to_tape_best) wrote into a unified allocator, as every verb takes its haystacks and replacements.
- `counts` (out): The per-haystack counts, haystack h at `counts[h * stride]`.
- `counts_stride` (in): Entries from one haystack's count to the next, at least one, so a strided call writes one column of a ____[haystacks, vocabularies] feature matrix.
- `stream` (in): Null on the CPU. On a GPU, the stream to queue on, which also names the device the round runs on; null for the default.

Statuses: - [`sz_unexpected_dimensions_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_unexpected_dimensions_k) `counts_stride` is zero.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) A device got an argument or a `stream` it cannot use.

## sz_substrings_find_serial

Reports every match of every needle in every haystack.

```c
sz_status_t sz_substrings_find_serial(
    sz_substrings_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_match_t *matches,
    sz_size_t matches_capacity,
    sz_size_t *matches_offsets,
    sz_stream_t stream);
```

- `sz_substrings_find` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, hopper, rocm, metal

A capacity too small is not an error: the report's `matches_emitted` names the true total and `shortfall` names what did not fit, so a sizing call and a filling call need no walk between.

- `engine` (in): A compiled vocabulary.
- `haystacks` (in): The texts to search; their bytes are read in place and never copied.
- `matches` (out): Room for `matches_capacity` matches, ascending by haystack; [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) together with a zero capacity makes the call a pure size query.
- `matches_capacity` (in): Entries `matches` holds.
- `matches_offsets` (out): One boundary per haystack into `matches` plus one, the last being the total; filled whether or not the matches fit, which is what sizes the next call.
- `stream` (in): As [sz_substrings_counts](#sz_substrings_counts) takes it.

Statuses: - [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) A device got an unaddressable argument.

## sz_substrings_replace_serial

Rewrites every haystack, substituting one replacement per needle, into one `target`.

```c
sz_status_t sz_substrings_replace_serial(
    sz_substrings_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_strings_view_t const *replacements,
    sz_ptr_t target,
    sz_size_t target_capacity,
    sz_size_t *offsets,
    sz_stream_t stream);
```

- `sz_substrings_replace` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, hopper, rocm, metal

A target too small is not an error: the report's `target_length` names the bytes the rewrite needs and `shortfall` names what did not fit, leaving the contents of `target` unspecified.

- `engine` (in): A vocabulary compiled under a leftmost policy; [sz_substrings_overlapping_k](#sz_substrings_overlapping_k) is refused, since a substitution over matches that share bytes is not a function.
- `haystacks` (in): The texts to rewrite; their bytes are read in place and never copied.
- `replacements` (in): One replacement per needle, indexed by needle; an empty one deletes.
- `target` (out): Room for `target_capacity` bytes, or [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) with zero capacity to size only.
- `target_capacity` (in): Bytes `target` holds.
- `offsets` (out): One rewritten boundary per haystack plus one, the last being the total; filled whether or not the target held the result, which is what sizes the next call.
- `stream` (in): As [sz_substrings_counts](#sz_substrings_counts) takes it.

Statuses: - [`sz_unexpected_dimensions_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_unexpected_dimensions_k) `replacements` lacks one entry per needle.
- [`sz_status_unknown_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_status_unknown_k) The policy leaves no cover.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) An argument no device kernel can address.

## sz_substrings_bm25_scores_serial

Scores every haystack against the vocabulary as one BM25 query, one score per haystack.

```c
sz_status_t sz_substrings_bm25_scores_serial(
    sz_substrings_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_f32_t const *document_lengths,
    sz_substrings_bm25_t const *parameters,
    sz_f32_t const *needle_weights,
    sz_f32_t *scores,
    sz_size_t scores_stride,
    sz_stream_t stream);
```

- `sz_substrings_bm25_scores` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, hopper, rocm, metal

The vocabulary is the query: `needle_weights` holds each needle's IDF or boost. Term frequencies are raw overlapping counts, since a leftmost cover would suppress genuine occurrences of a needle nested in another, so the engine's own policy does not apply here. A CPU backend sums in ascending needle order and a CUDA backend in fixed-point integers, so each is bit-stable across its own runs, and the two agree.

The arguments are invalid when `needle_weights` is [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL), `scores_stride` is zero, or `length_normalization` is positive while `average_document_length` is not.

- `engine` (in): A compiled vocabulary.
- `haystacks` (in): The documents to score; their bytes are read in place and never copied.
- `document_lengths` (in): One length per haystack to normalize by, in the unit of `average_document_length`, or [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) to use each haystack's byte length.
- `parameters` (in): BM25's continuous parameters.
- `needle_weights` (in): One weight per needle of the engine.
- `scores` (out): The per-haystack scores, haystack h at `scores[h * stride]`.
- `scores_stride` (in): Entries from one haystack's score to the next, at least one, so a strided call writes one column of a ____[haystacks, vocabularies] feature matrix.
- `stream` (in): As [sz_substrings_counts](#sz_substrings_counts) takes it.

Statuses: - [`sz_unexpected_dimensions_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_unexpected_dimensions_k) Invalid arguments, as above.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) An argument or grown arena a device cannot address.

## sz_*_find_kernel

Finds the substrings kernel of `kind`, from the best of `capabilities`.

```c
sz_status_t sz_substrings_find_kernel(
    sz_kernel_kind_t kind,
    sz_capability_t capabilities,
    sz_kernel_punned_t *kernel,
    sz_capability_t *capability);
```

- `sz_substrings_find_kernel`
- `sz_regex_find_kernel`
- `sz_subdocuments_find_kernel`

- `sz_substrings_find` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, hopper, rocm, metal
- `sz_regex_find` backends: serial, haswell, icelake, neon, cuda, rocm, metal
- `sz_subdocuments_find` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, rocm, metal

- `kind` (in): The verb whose kernel to find.
- `capabilities` (in): The mask to pick the best capability from.
- `kernel` (out): The kernel, or null when none of `capabilities` has it.
- `capability` (out): The capability the kernel belongs to, or zero.

Statuses: - [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k)
- [`sz_missing_library_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_library_k) Header-only.

## sz_regex_engine_init

Compiles `patterns` for the best capability of `capabilities` that has an init kernel.

```c
sz_status_t sz_regex_engine_init(
    sz_regex_engine_t *engine,
    sz_strings_view_t const *patterns,
    sz_case_sensitivity_t case_sensitivity,
    sz_size_t *malformed_index,
    sz_capability_t capabilities,
    sz_allocator_t *allocator,
    sz_stream_t stream);
```

- `sz_regex_engine_init` backends: serial, haswell, icelake, neon, cuda, rocm, metal

- `engine` (out): Left untouched unless the call succeeds.
- `patterns` (in): The vocabulary, read on the host; a pattern's index is the `needle_index` its matches report, and the lower index wins between patterns matching at one position.
- `case_sensitivity` (in): Whether patterns match bytes as written, or both sides are folded first, as the section on case folding describes.
- `malformed_index` (out): Where the index of the first refused pattern is written, only when the call returns [`sz_invalid_syntax_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_invalid_syntax_k); may be [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL).
- `capabilities` (in): One device's capabilities, like `sz_capabilities_enabled_cpu` reports.
- `allocator` (in): Where both blocks come from, or [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) for [sz_allocator_init_unified_best](https://www.unum.cloud/docs/stringzilla/c/memory-operations.md#sz_allocator_init_unified_best) of `capabilities`. Stored by value, so [sz_regex_engine_free](#sz_regex_engine_free) needs none and cannot get the wrong one. Scratch grows to the widest call seen, so a warm engine allocates nothing.
- `stream` (in): Null on the CPU, or the device stream the call is enqueued on.

Statuses: - [`sz_invalid_syntax_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_invalid_syntax_k) An empty vocabulary or a pattern outside the syntax above.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) No capability of the mask has an init.
- [`sz_overflow_risk_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_overflow_risk_k) An NFA exceeds a 32-bit state id.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) A `stream` or `allocator` its capability cannot use.

## sz_regex_contains

Marks which haystacks hold at least one match of any pattern, stopping each walk at its first match, the lightest verb of the engine.

```c
sz_status_t sz_regex_contains(
    sz_regex_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_u1x8_t *contained,
    sz_stream_t stream);
```

- `sz_regex_contains` backends: serial, haswell, icelake, neon, cuda, rocm

Device rounds walk every byte at once, as the counts' do, and keep only which haystacks matched.

- `engine` (in): A compiled vocabulary, whose round scratch grows to the longest haystack.
- `haystacks` (in): The texts to search, in any layout; their bytes are read in place.
- `contained` (out): One bit per haystack, eight to a byte, the lowest bit first: set when that haystack matches, with the bits past the last haystack written as zero.
- `stream` (in): Null on the CPU, or the device stream the call is enqueued on.

Statuses: - [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) A `stream` the engine's capability cannot use.

## sz_regex_counts

Counts the vocabulary's leftmost-first matches in every haystack, one count per haystack.

```c
sz_status_t sz_regex_counts(
    sz_regex_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_size_t *counts,
    sz_size_t counts_stride,
    sz_stream_t stream);
```

- `sz_regex_counts` backends: serial, haswell, icelake, neon, cuda, rocm, metal

- `engine` (in): A compiled vocabulary, whose round scratch grows to the longest haystack.
- `haystacks` (in): The texts to search, in any layout; their bytes are read in place. Malformed UTF-8 is searched as is: no class or literal matches a malformed byte.
- `counts` (out): The per-haystack counts, haystack h at `counts[h * stride]`.
- `counts_stride` (in): Entries from one haystack's count to the next, at least one, so a strided call writes one column of a ____[haystacks, vocabularies] feature matrix.
- `stream` (in): Null on the CPU, or the device stream the call is enqueued on.

Statuses: - [`sz_unexpected_dimensions_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_unexpected_dimensions_k) `counts_stride` is zero.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) A `stream` the engine's capability cannot use.

## sz_regex_find

Reports every leftmost-first match of the vocabulary in every haystack.

```c
sz_status_t sz_regex_find(
    sz_regex_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_match_t *matches,
    sz_size_t matches_capacity,
    sz_size_t *matches_offsets,
    sz_stream_t stream);
```

- `sz_regex_find` backends: serial, haswell, icelake, neon, cuda, rocm, metal

Within a haystack, matches share no bytes and come in ascending order of start; an empty match is never reported where the previous match ended, as Rust's `find_iter` does, nor before a UTF-8 continuation byte, so none splits a codepoint.

- `engine` (in): A compiled vocabulary.
- `haystacks` (in): The texts to search; their bytes are read in place and never copied.
- `matches` (out): Room for `matches_capacity` matches, ascending by haystack, then by start; [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) together with a zero capacity makes the call a pure size query.
- `matches_capacity` (in): Entries `matches` holds; only the first that many are written.
- `matches_offsets` (out): One boundary per haystack into the full list of matches plus one, the last being the total; true whether or not the matches fit, which sizes the next call.
- `stream` (in): Null on the CPU, or the device stream the call is enqueued on.

Statuses: - [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) A `stream` the engine's capability cannot use.

## sz_regex_replace

Rewrites every haystack, substituting each match with its pattern's replacement, into one `target`.

```c
sz_status_t sz_regex_replace(
    sz_regex_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_strings_view_t const *replacements,
    sz_ptr_t target,
    sz_size_t target_capacity,
    sz_size_t *offsets,
    sz_stream_t stream);
```

- `sz_regex_replace` backends: serial, haswell, icelake, neon, cuda, rocm, metal

A replacement is copied as written, except that `$$` stands for one dollar and `$N`, `${N}` and `${name}` for the text of that group of the match's pattern, `$0` being the whole match; a group that took no part in the match or does not exist is empty, and a dollar starting no reference stands for itself, so `$1a` is group one followed by `a`. Groups are read only for the matches whose replacement names them, by a PikeVM over each such match's own span.

- `engine` (in): A compiled vocabulary.
- `haystacks` (in): The texts to rewrite; their bytes are read in place and never copied.
- `replacements` (in): One replacement per pattern, indexed by pattern; an empty one deletes.
- `target` (out): Room for `target_capacity` bytes, or [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) with zero capacity to size only.
- `target_capacity` (in): Bytes `target` holds; only the first that many are written.
- `offsets` (out): One rewritten boundary per haystack plus one, the last being the total; true whether or not the target held the result, which sizes the next call.
- `stream` (in): Null on the CPU, or the device stream the call is enqueued on.

Statuses: - [`sz_unexpected_dimensions_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_unexpected_dimensions_k) `replacements` lacks one entry per pattern.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) A `stream` the engine's capability cannot use.

## sz_regex_extract

Locates the groups of each haystack's first match of `pattern_index`, as Pandas `str.extract` reads them.

```c
sz_status_t sz_regex_extract(
    sz_regex_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_size_t pattern_index,
    sz_match_t *groups,
    sz_size_t groups_capacity,
    sz_size_t *groups_offsets,
    sz_stream_t stream);
```

- `sz_regex_extract` backends: serial, haswell, icelake, neon, cuda, rocm, metal

A haystack with such a match gets one row for the whole match, `needle_index` zero, then one row per group that took part, `needle_index` naming the group; a haystack without one gets no rows, so a group that took no part is told apart from an empty one. The first match of a pattern is the first the vocabulary reports with that `needle_index`.

- `engine` (in): A compiled vocabulary.
- `haystacks` (in): The texts to search; their bytes are read in place and never copied.
- `pattern_index` (in): The pattern whose groups to locate.
- `groups` (out): Room for `groups_capacity` rows, or [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) with zero capacity to size only.
- `groups_capacity` (in): Rows `groups` holds; only the first that many are written.
- `groups_offsets` (out): One boundary per haystack plus one, the last being the total; true whether or not `groups` held them, which sizes the next call.
- `stream` (in): Null on the CPU, or the device stream the call is enqueued on.

Statuses: - [`sz_unexpected_dimensions_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_unexpected_dimensions_k) `pattern_index` is past the vocabulary.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) A `stream` the engine's capability cannot use.

## sz_regex_engine_init_serial

Compiles `patterns` for the serial kernels.

```c
sz_status_t sz_regex_engine_init_serial(
    sz_regex_engine_t *engine,
    sz_strings_view_t const *patterns,
    sz_case_sensitivity_t case_sensitivity,
    sz_size_t *malformed_index,
    sz_allocator_t *allocator,
    sz_stream_t stream);
```

- `sz_regex_engine_init` backends: serial, haswell, icelake, neon, cuda, rocm, metal

- `engine` (out): Left untouched unless the call succeeds.
- `patterns` (in): The vocabulary, read on the host; a pattern's index is the `needle_index` its matches report, and the lower index wins between patterns matching at one position.
- `case_sensitivity` (in): Whether patterns match bytes as written, or both sides are folded first, as the section on case folding describes.
- `malformed_index` (out): Where the index of the first refused pattern is written, only when the call returns [`sz_invalid_syntax_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_invalid_syntax_k); may be [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL).
- `allocator` (in): Where both blocks come from, or [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) for `sz_allocator_init_unified_serial`. Stored by value, so [sz_regex_engine_free](#sz_regex_engine_free) needs none and cannot get the wrong one. Scratch grows to the widest call seen, so a warm engine allocates nothing.
- `stream` (in): Null on the CPU.

Statuses: - [`sz_invalid_syntax_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_invalid_syntax_k) An empty vocabulary or a pattern outside the syntax above.
- [`sz_overflow_risk_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_overflow_risk_k) An NFA exceeds a 32-bit state id.

## sz_regex_contains_serial

Marks which haystacks hold at least one match of any pattern, stopping each walk at its first match, the lightest verb of the engine.

```c
sz_status_t sz_regex_contains_serial(
    sz_regex_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_u1x8_t *contained,
    sz_stream_t stream);
```

- `sz_regex_contains` backends: serial, haswell, icelake, neon, cuda, rocm

Device rounds walk every byte at once, as the counts' do, and keep only which haystacks matched.

- `engine` (in): A compiled vocabulary, whose round scratch grows to the longest haystack.
- `haystacks` (in): The texts to search, in any layout; their bytes are read in place.
- `contained` (out): One bit per haystack, eight to a byte, the lowest bit first: set when that haystack matches, with the bits past the last haystack written as zero.
- `stream` (in): Null on the CPU, or the device stream the call is enqueued on.

Statuses: - [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) A `stream` the engine's capability cannot use.

## sz_regex_counts_serial

Counts the vocabulary's leftmost-first matches in every haystack, one count per haystack.

```c
sz_status_t sz_regex_counts_serial(
    sz_regex_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_size_t *counts,
    sz_size_t counts_stride,
    sz_stream_t stream);
```

- `sz_regex_counts` backends: serial, haswell, icelake, neon, cuda, rocm, metal

- `engine` (in): A compiled vocabulary, whose round scratch grows to the longest haystack.
- `haystacks` (in): The texts to search, in any layout; their bytes are read in place. Malformed UTF-8 is searched as is: no class or literal matches a malformed byte.
- `counts` (out): The per-haystack counts, haystack h at `counts[h * stride]`.
- `counts_stride` (in): Entries from one haystack's count to the next, at least one, so a strided call writes one column of a ____[haystacks, vocabularies] feature matrix.
- `stream` (in): Null on the CPU, or the device stream the call is enqueued on.

Statuses: - [`sz_unexpected_dimensions_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_unexpected_dimensions_k) `counts_stride` is zero.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) A `stream` the engine's capability cannot use.

## sz_regex_find_serial

Reports every leftmost-first match of the vocabulary in every haystack.

```c
sz_status_t sz_regex_find_serial(
    sz_regex_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_match_t *matches,
    sz_size_t matches_capacity,
    sz_size_t *matches_offsets,
    sz_stream_t stream);
```

- `sz_regex_find` backends: serial, haswell, icelake, neon, cuda, rocm, metal

Within a haystack, matches share no bytes and come in ascending order of start; an empty match is never reported where the previous match ended, as Rust's `find_iter` does, nor before a UTF-8 continuation byte, so none splits a codepoint.

- `engine` (in): A compiled vocabulary.
- `haystacks` (in): The texts to search; their bytes are read in place and never copied.
- `matches` (out): Room for `matches_capacity` matches, ascending by haystack, then by start; [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) together with a zero capacity makes the call a pure size query.
- `matches_capacity` (in): Entries `matches` holds; only the first that many are written.
- `matches_offsets` (out): One boundary per haystack into the full list of matches plus one, the last being the total; true whether or not the matches fit, which sizes the next call.
- `stream` (in): Null on the CPU, or the device stream the call is enqueued on.

Statuses: - [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) A `stream` the engine's capability cannot use.

## sz_regex_replace_serial

Rewrites every haystack, substituting each match with its pattern's replacement, into one `target`.

```c
sz_status_t sz_regex_replace_serial(
    sz_regex_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_strings_view_t const *replacements,
    sz_ptr_t target,
    sz_size_t target_capacity,
    sz_size_t *offsets,
    sz_stream_t stream);
```

- `sz_regex_replace` backends: serial, haswell, icelake, neon, cuda, rocm, metal

A replacement is copied as written, except that `$$` stands for one dollar and `$N`, `${N}` and `${name}` for the text of that group of the match's pattern, `$0` being the whole match; a group that took no part in the match or does not exist is empty, and a dollar starting no reference stands for itself, so `$1a` is group one followed by `a`. Groups are read only for the matches whose replacement names them, by a PikeVM over each such match's own span.

- `engine` (in): A compiled vocabulary.
- `haystacks` (in): The texts to rewrite; their bytes are read in place and never copied.
- `replacements` (in): One replacement per pattern, indexed by pattern; an empty one deletes.
- `target` (out): Room for `target_capacity` bytes, or [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) with zero capacity to size only.
- `target_capacity` (in): Bytes `target` holds; only the first that many are written.
- `offsets` (out): One rewritten boundary per haystack plus one, the last being the total; true whether or not the target held the result, which sizes the next call.
- `stream` (in): Null on the CPU, or the device stream the call is enqueued on.

Statuses: - [`sz_unexpected_dimensions_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_unexpected_dimensions_k) `replacements` lacks one entry per pattern.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) A `stream` the engine's capability cannot use.

## sz_regex_extract_serial

Locates the groups of each haystack's first match of `pattern_index`, as Pandas `str.extract` reads them.

```c
sz_status_t sz_regex_extract_serial(
    sz_regex_engine_t *engine,
    sz_strings_view_t const *haystacks,
    sz_size_t pattern_index,
    sz_match_t *groups,
    sz_size_t groups_capacity,
    sz_size_t *groups_offsets,
    sz_stream_t stream);
```

- `sz_regex_extract` backends: serial, haswell, icelake, neon, cuda, rocm, metal

A haystack with such a match gets one row for the whole match, `needle_index` zero, then one row per group that took part, `needle_index` naming the group; a haystack without one gets no rows, so a group that took no part is told apart from an empty one. The first match of a pattern is the first the vocabulary reports with that `needle_index`.

- `engine` (in): A compiled vocabulary.
- `haystacks` (in): The texts to search; their bytes are read in place and never copied.
- `pattern_index` (in): The pattern whose groups to locate.
- `groups` (out): Room for `groups_capacity` rows, or [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) with zero capacity to size only.
- `groups_capacity` (in): Rows `groups` holds; only the first that many are written.
- `groups_offsets` (out): One boundary per haystack plus one, the last being the total; true whether or not `groups` held them, which sizes the next call.
- `stream` (in): Null on the CPU, or the device stream the call is enqueued on.

Statuses: - [`sz_unexpected_dimensions_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_unexpected_dimensions_k) `pattern_index` is past the vocabulary.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_device_memory_mismatch_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_device_memory_mismatch_k) A `stream` the engine's capability cannot use.

## sz_subdocuments_engine_init

Compiles `paths` for the best capability of `capabilities` that has an init kernel.

```c
sz_status_t sz_subdocuments_engine_init(
    sz_subdocuments_engine_t *engine,
    sz_strings_view_t const *paths,
    sz_subdocuments_format_t format,
    sz_subdocuments_options_t options,
    sz_size_t *malformed_index,
    sz_capability_t capabilities,
    sz_allocator_t *allocator,
    sz_stream_t stream);
```

- `sz_subdocuments_engine_init` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, rocm, metal

JSON paths follow RFC 9535: the root `$`, child segments `.name` and `.*`, and bracketed unions of names `'name'` or `"name"`, wildcards `*`, indices `n`, slices `begin:end:step` and filters `?expression`, the same after a descendant `..`, with blank space where the RFC allows it. Negative indices and bounds count from the array's end, and a negative step walks backward, though matches still come in document order. Filters hold comparisons, existence tests of relative queries, `&&`, `||`, `!` and parentheses, and the functions `length()`, `count()`, `value()`, `match()` and `search()`, whose I-Regexp patterns compile into one regex engine owned by this one. Each path compiles into one step per segment, numbered path after path, each filter's queries taking steps of their own. A path may also be a bare filter `?expression`, which tests the document's root as `$[?expression]` tests an array's elements.

XML paths follow XPath 1.0 along forward axes: absolute paths of `/` and `//` steps, each a name test, `*`, `@name`, `@*` or `text()` with an optional position `[n]`, which counts the siblings passing the same test. Name tests compare QNames as the document spells them, prefix included, and namespace declarations are no attributes. No blank space is allowed.

What no walk evaluates is an absolute query inside a filter, a comparison with no literal, `length()` or `count()` on either side, a pattern other than a string literal or one the regex family refuses, and in XML a reverse axis or a predicate other than a position.

- `engine` (out): Left untouched unless the call succeeds.
- `paths` (in): The vocabulary, one path per string, read on the host.
- `format` (in): The grammar of the documents every verb of this engine reads.
- `options` (in): How JSON walks read what RFC 8259 leaves open, or [`sz_subdocuments_options_none_k`](#sz_subdocuments_options_none_k) for the default.
- `malformed_index` (out): The first path to break the grammar or leave the subset, on failure.
- `capabilities` (in): One device's capabilities, like `sz_capabilities_enabled_cpu` reports.
- `allocator` (in): Where both blocks come from, or [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) for [sz_allocator_init_unified_best](https://www.unum.cloud/docs/stringzilla/c/memory-operations.md#sz_allocator_init_unified_best) of `capabilities`. Stored by value, so [sz_subdocuments_engine_free](#sz_subdocuments_engine_free) needs none and cannot get the wrong one. Scratch grows to the widest call seen, so a warm engine allocates nothing.
- `stream` (in): Null on the CPU.

Statuses: - [`sz_invalid_syntax_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_invalid_syntax_k) Anything outside the grammar, and what no walk evaluates, as above.
- [`sz_missing_library_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_library_k) `match()` or `search()` in a header-only build, whose regex dispatch links no library.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) No capability of the mask has an init, a `format` no kernel reads yet, or on a GPU non-zero `options` or a bare filter, which `malformed_index` then names.
- [`sz_unexpected_dimensions_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_unexpected_dimensions_k) An empty vocabulary.
- [`sz_overflow_risk_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_overflow_risk_k) Past 32-bit step or name offsets.

## sz_subdocuments_contains

Marks which documents hold at least one node some path selects, stopping each walk at its first match, the lightest verb of the engine.

```c
sz_status_t sz_subdocuments_contains(
    sz_subdocuments_engine_t *engine,
    sz_strings_view_t const *documents,
    sz_u1x8_t *contained,
    sz_size_t *malformed_index,
    sz_stream_t stream);
```

- `sz_subdocuments_contains` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, rocm

A walk validates nothing past the match it stops at, so a document malformed only after its first match is marked rather than refused. Device walks run whole, as the counts' do, so on a device that document fails the call.

- `engine` (in): Compiled paths, serving one call at a time.
- `documents` (in): One document per string; their bytes are read in place and never copied.
- `contained` (out): One bit per document, eight to a byte, the lowest bit first: set when that document matches, with the bits past the last document written as zero.
- `malformed_index` (out): The first malformed document, on failure.
- `stream` (in): Null on the CPU.

Statuses: - [`sz_invalid_syntax_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_invalid_syntax_k) A malformed document, as [sz_subdocuments_counts](#sz_subdocuments_counts) defines it, leaving `contained` unspecified.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.

## sz_subdocuments_counts

Counts the nodes of every document that some path selects, one count per document.

```c
sz_status_t sz_subdocuments_counts(
    sz_subdocuments_engine_t *engine,
    sz_strings_view_t const *documents,
    sz_size_t *counts,
    sz_size_t counts_stride,
    sz_size_t *malformed_index,
    sz_stream_t stream);
```

- `sz_subdocuments_counts` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, rocm, metal

A node several paths select counts once per path, as [sz_subdocuments_find](#sz_subdocuments_find) reports it.

A document is malformed with unbalanced brackets or tags, an unterminated string, attribute value, comment or CDATA section, a `/` outside the strings of JSON, broken structure where the walk reads it, or an XML DOCTYPE with an internal subset, and leaves the counts unspecified. A container or element no live step can enter is skipped by counting its nesting, so nothing else it holds is validated.

- `engine` (in): Compiled paths; the walk grows the engine's depth stack, so one engine serves one call at a time.
- `documents` (in): One document per string; their bytes are read in place and never copied.
- `counts` (out): The per-document counts, document d at `counts[d * stride]`.
- `counts_stride` (in): Entries from one document's count to the next, at least one.
- `malformed_index` (out): The first malformed document, on failure.
- `stream` (in): Null on the CPU.

Statuses: - [`sz_unexpected_dimensions_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_unexpected_dimensions_k) `counts_stride` is zero.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_invalid_syntax_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_invalid_syntax_k) A malformed document.

## sz_subdocuments_find

Reports every node of every document that some path selects.

```c
sz_status_t sz_subdocuments_find(
    sz_subdocuments_engine_t *engine,
    sz_strings_view_t const *documents,
    sz_match_t *matches,
    sz_size_t matches_capacity,
    sz_size_t *matches_offsets,
    sz_size_t *malformed_index,
    sz_stream_t stream);
```

- `sz_subdocuments_find` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, rocm, metal

Matches come by document, then by start offset, then by path, so a container comes before the nodes inside it, and a node is reported once per path that selects it, however many of the path's selectors or descendant routes reach it. Each match spans the node's exact bytes: in JSON a string with its quotes, a number, `true`, `false` or `null` without the blank space around it, or an object or array through its closing bracket; in XML an element through its end tag, an attribute's value without its quotes, or a text node, whose CDATA sections stay as written.

- `engine` (in): Compiled paths, serving one call at a time.
- `documents` (in): One document per string; their bytes are read in place and never copied.
- `matches` (out): Room for `matches_capacity` matches; [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) together with a zero capacity makes the call a pure size query.
- `matches_capacity` (in): Entries `matches` holds.
- `matches_offsets` (out): One boundary per document into the matches plus one, the last being the total; filled whether or not the matches fit, which is what sizes the next call.
- `malformed_index` (out): The first malformed document, on failure.
- `stream` (in): Null on the CPU.

Statuses: - [`sz_invalid_syntax_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_invalid_syntax_k) A malformed document, as [sz_subdocuments_counts](#sz_subdocuments_counts) defines it, leaving the outputs unspecified.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.

## sz_subdocuments_engine_init_serial

Compiles `paths` for the serial kernels, with the grammar [sz_subdocuments_engine_init](#sz_subdocuments_engine_init) lists.

```c
sz_status_t sz_subdocuments_engine_init_serial(
    sz_subdocuments_engine_t *engine,
    sz_strings_view_t const *paths,
    sz_subdocuments_format_t format,
    sz_subdocuments_options_t options,
    sz_size_t *malformed_index,
    sz_allocator_t *allocator,
    sz_stream_t stream);
```

- `sz_subdocuments_engine_init` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, rocm, metal

- `engine` (out): Left untouched unless the call succeeds.
- `paths` (in): The vocabulary, one path per string, read on the host.
- `format` (in): The grammar of the documents every verb of this engine reads.
- `options` (in): How JSON walks read what RFC 8259 leaves open, or [`sz_subdocuments_options_none_k`](#sz_subdocuments_options_none_k) for the default.
- `malformed_index` (out): The first path to break the grammar or leave the subset, on failure.
- `allocator` (in): Where both blocks come from, or [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) for `sz_allocator_init_unified_serial`. Stored by value, so [sz_subdocuments_engine_free](#sz_subdocuments_engine_free) needs none and cannot get the wrong one. Scratch grows to the widest call seen, so a warm engine allocates nothing.
- `stream` (in): Null on the CPU.

Statuses: - [`sz_invalid_syntax_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_invalid_syntax_k) Anything outside the grammar or what no walk evaluates.
- [`sz_missing_library_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_library_k) `match()` or `search()` in a header-only build.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) A `format` no kernel reads yet.
- [`sz_unexpected_dimensions_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_unexpected_dimensions_k) An empty vocabulary.
- [`sz_overflow_risk_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_overflow_risk_k) Past 32-bit step or name offsets.

## sz_subdocuments_contains_serial

Marks which documents hold at least one node some path selects, stopping each walk at its first match, the lightest verb of the engine.

```c
sz_status_t sz_subdocuments_contains_serial(
    sz_subdocuments_engine_t *engine,
    sz_strings_view_t const *documents,
    sz_u1x8_t *contained,
    sz_size_t *malformed_index,
    sz_stream_t stream);
```

- `sz_subdocuments_contains` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, rocm

A walk validates nothing past the match it stops at, so a document malformed only after its first match is marked rather than refused. Device walks run whole, as the counts' do, so on a device that document fails the call.

- `engine` (in): Compiled paths, serving one call at a time.
- `documents` (in): One document per string; their bytes are read in place and never copied.
- `contained` (out): One bit per document, eight to a byte, the lowest bit first: set when that document matches, with the bits past the last document written as zero.
- `malformed_index` (out): The first malformed document, on failure.
- `stream` (in): Null on the CPU.

Statuses: - [`sz_invalid_syntax_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_invalid_syntax_k) A malformed document, as [sz_subdocuments_counts](#sz_subdocuments_counts) defines it, leaving `contained` unspecified.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.

## sz_subdocuments_counts_serial

Counts the nodes of every document that some path selects, one count per document.

```c
sz_status_t sz_subdocuments_counts_serial(
    sz_subdocuments_engine_t *engine,
    sz_strings_view_t const *documents,
    sz_size_t *counts,
    sz_size_t counts_stride,
    sz_size_t *malformed_index,
    sz_stream_t stream);
```

- `sz_subdocuments_counts` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, rocm, metal

A node several paths select counts once per path, as [sz_subdocuments_find](#sz_subdocuments_find) reports it.

A document is malformed with unbalanced brackets or tags, an unterminated string, attribute value, comment or CDATA section, a `/` outside the strings of JSON, broken structure where the walk reads it, or an XML DOCTYPE with an internal subset, and leaves the counts unspecified. A container or element no live step can enter is skipped by counting its nesting, so nothing else it holds is validated.

- `engine` (in): Compiled paths; the walk grows the engine's depth stack, so one engine serves one call at a time.
- `documents` (in): One document per string; their bytes are read in place and never copied.
- `counts` (out): The per-document counts, document d at `counts[d * stride]`.
- `counts_stride` (in): Entries from one document's count to the next, at least one.
- `malformed_index` (out): The first malformed document, on failure.
- `stream` (in): Null on the CPU.

Statuses: - [`sz_unexpected_dimensions_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_unexpected_dimensions_k) `counts_stride` is zero.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.
- [`sz_invalid_syntax_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_invalid_syntax_k) A malformed document.

## sz_subdocuments_find_serial

Reports every node of every document that some path selects.

```c
sz_status_t sz_subdocuments_find_serial(
    sz_subdocuments_engine_t *engine,
    sz_strings_view_t const *documents,
    sz_match_t *matches,
    sz_size_t matches_capacity,
    sz_size_t *matches_offsets,
    sz_size_t *malformed_index,
    sz_stream_t stream);
```

- `sz_subdocuments_find` backends: serial, haswell, icelake, neon, sve2, rvv, v128, loongsonasx, powervsx, cuda, rocm, metal

Matches come by document, then by start offset, then by path, so a container comes before the nodes inside it, and a node is reported once per path that selects it, however many of the path's selectors or descendant routes reach it. Each match spans the node's exact bytes: in JSON a string with its quotes, a number, `true`, `false` or `null` without the blank space around it, or an object or array through its closing bracket; in XML an element through its end tag, an attribute's value without its quotes, or a text node, whose CDATA sections stay as written.

- `engine` (in): Compiled paths, serving one call at a time.
- `documents` (in): One document per string; their bytes are read in place and never copied.
- `matches` (out): Room for `matches_capacity` matches; [`STRINGZILLA_NULL`](https://www.unum.cloud/docs/stringzilla/c/types.md#STRINGZILLA_NULL) together with a zero capacity makes the call a pure size query.
- `matches_capacity` (in): Entries `matches` holds.
- `matches_offsets` (out): One boundary per document into the matches plus one, the last being the total; filled whether or not the matches fit, which is what sizes the next call.
- `malformed_index` (out): The first malformed document, on failure.
- `stream` (in): Null on the CPU.

Statuses: - [`sz_invalid_syntax_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_invalid_syntax_k) A malformed document, as [sz_subdocuments_counts](#sz_subdocuments_counts) defines it, leaving the outputs unspecified.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) An empty engine.

## sz_utf8_count_best

Count the number of UTF-8 characters in a string.

```c
sz_status_t sz_utf8_count_best(
    sz_cptr_t text,
    sz_size_t length,
    sz_size_t *count,
    sz_capability_t capabilities,
    sz_stream_t stream);
```

- `sz_utf8_count` backends: serial, haswell, icelake, neon, sve2, rvv, v128, v128relaxed, loongsonasx, powervsx

The logic is to count the number of "continuation bytes" matching the 10xxxxxx pattern, and then subtract that from the total byte length to get the number of "start bytes" - coinciding with the number of UTF-8 characters. Counting the characters of a string:

```
sz_capability_t capabilities;
sz_capabilities_enabled_cpu(&capabilities);
sz_size_t char_count;
sz_utf8_count_best(text, length, &char_count, capabilities, NULL);
printf("String has %zu characters\n", char_count);
```

- `text` (in): String to be scanned.
- `length` (in): Number of bytes in the string.
- `count` (out): Number of UTF-8 characters in the string.
- `capabilities` (in): One device's capabilities, like `sz_capabilities_enabled_cpu` reports.
- `stream` (in): Null on the CPU, or the GPU stream of that device to queue on.

Statuses: - [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) No capability in `capabilities` has it.

## sz_utf8_count_strings_best

Checks that every one of `strings[order[i]]`, or of the first `count` strings, is well-formed UTF-8 and, unless `runes` is null, counts each one's runes.

```c
sz_status_t sz_utf8_count_strings_best(
    sz_strings_view_t const *strings,
    sz_size_t const *order,
    sz_size_t count,
    sz_size_t *runes,
    sz_size_t *malformed_index,
    sz_capability_t capabilities,
    sz_stream_t stream);
```

- `sz_utf8_count_strings` backends: serial, haswell, icelake, neon

Well-formed is RFC 3629: no overlong forms, surrogates, codepoints past U+10FFFF, stray continuation bytes or truncated sequences, and no sequence crossing from one string into the next. A tape read in its own order is validated as one run of bytes, resolving each string at its end in one step. Other layouts and orders go string by string.

- `strings` (in): The strings to check, in any layout.
- `order` (in): Indices into `strings`, possibly repeated, or null for the first `count`.
- `count` (in): Number of strings to check.
- `runes` (out): Null, or the runes of each string, `count` of them, unspecified from the first ill-formed string on.
- `malformed_index` (out): The first ill-formed string, or `count` when all are well-formed.
- `capabilities` (in): One device's capabilities, like `sz_capabilities_enabled_cpu` reports.
- `stream` (in): Null on the CPU, or the GPU stream of that device to queue on.

Statuses: - [`sz_invalid_utf8_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_invalid_utf8_k) A string is ill-formed.
- [`sz_missing_kernel_k`](https://www.unum.cloud/docs/stringzilla/c/types.md#sz_missing_kernel_k) No capability in `capabilities` has it.