# Searching and Counting > StringZilla for Rust All search functions accept any `AsRef<[u8]>` and return byte offsets as `Option`. They are available both as `sz::` free functions and as `StringZillableBinary` trait methods. Free functions: ```rust pub fn find, N: AsRef<[u8]>>(haystack: H, needle: N) -> Option; pub fn rfind, N: AsRef<[u8]>>(haystack: H, needle: N) -> Option; pub fn find_byteset>(haystack: H, needles: Byteset) -> Option; pub fn rfind_byteset>(haystack: H, needles: Byteset) -> Option; pub fn find_byte_from(haystack: H, needles: N) -> Option; // first byte in the set pub fn rfind_byte_from(haystack: H, needles: N) -> Option; // last byte in the set pub fn find_byte_not_from(haystack: H, needles: N) -> Option; // first byte not in the set pub fn rfind_byte_not_from(haystack: H, needles: N) -> Option; // last byte not in the set ``` ```rust use stringzilla as sz; assert_eq!(sz::find("Hello, world!", "world"), Some(7)); assert_eq!(sz::rfind("Hello, world, world!", "world"), Some(14)); // First/last byte from a set of delimiters, or the first/last byte NOT in a set. assert_eq!(sz::find_byte_from("Hello, world!", "aeiou"), Some(1)); // 'e' assert_eq!(sz::rfind_byte_from("Hello, world!", "aeiou"), Some(8)); // 'o' assert_eq!(sz::find_byte_not_from("Hello, world!", "aeiou"), Some(0)); // 'H' assert_eq!(sz::rfind_byte_not_from("Hello, world!", "aeiou"), Some(12));// '!' ``` The `StringZillableBinary<'a, N>` trait provides the same operations as methods on the haystack. The blanket impl covers every `T: AsRef<[u8]> + ?Sized`, so they work on `&str`, `String`, `&[u8]`, `Cow`, and more: ```rust use stringzilla::StringZillableBinary; let haystack = "Hello, world!"; assert_eq!(haystack.sz_find("world".as_bytes()), Some(7)); assert_eq!(haystack.sz_rfind("world".as_bytes()), Some(7)); assert_eq!(haystack.sz_find_byte_from("aeiou".as_bytes()), Some(1)); assert_eq!(haystack.sz_rfind_byte_from("aeiou".as_bytes()), Some(8)); assert_eq!(haystack.sz_find_byte_not_from("aeiou".as_bytes()), Some(0)); assert_eq!(haystack.sz_rfind_byte_not_from("aeiou".as_bytes()), Some(12)); ``` Trait methods: `sz_find`, `sz_rfind`, `sz_find_byte_from`, `sz_rfind_byte_from`, `sz_find_byte_not_from`, `sz_rfind_byte_not_from`. ### Counting UTF-8 Characters These two functions reason about codepoint boundaries without fully decoding the string. ```rust pub fn count_utf8>(text: T) -> usize; pub fn find_nth_utf8>(text: T, n: usize) -> Option; ``` `count_utf8` counts codepoint start bytes with SIMD; `find_nth_utf8` returns the byte offset of the Nth, zero-indexed, codepoint without decoding the whole string: ```rust use stringzilla as sz; assert_eq!(sz::count_utf8("你好世界"), 4); assert_eq!(sz::count_utf8("Hello🌍"), 6); assert_eq!(sz::find_nth_utf8("Hello🌍", 5), Some(5)); // 🌍 starts at byte 5 assert_eq!(sz::find_nth_utf8("Hello", 5), None); ``` ### Multi-Pattern Search `SubstringsEngine` compiles a whole needle set into one Aho-Corasick automaton and walks every haystack against all of them at once, so a dictionary of thousands of terms costs one pass rather than thousands. Building it is the expensive half and the engine is reusable, so a long-lived one amortizes that across every later batch. ```rust fn new>(needles: &[N], case_sensitivity: CaseSensitivity, overlap_policy: SubstringsOverlapPolicy, hot_states: usize, matches_budget: usize, haystacks_budget: usize, stream: &Stream) -> Result; fn counts<'scope>(&'scope mut self, scope: &'scope Scope<'scope, '_>, haystacks: StringsView<'scope>, counts: &'scope mut [usize], counts_stride: usize) -> Result<(), Status>; fn find<'scope>(&'scope mut self, scope: &'scope Scope<'scope, '_>, haystacks: StringsView<'scope>, matches: &'scope mut [Match], matches_offsets: &'scope mut [usize]) -> Result<(), Status>; fn replace<'scope>(&'scope mut self, scope: &'scope Scope<'scope, '_>, haystacks: StringsView<'scope>, replacements: StringsView<'scope>, target: &'scope mut [u8], offsets: &'scope mut [usize]) -> Result<(), Status>; fn bm25_scores<'scope>(&'scope mut self, scope: &'scope Scope<'scope, '_>, haystacks: StringsView<'scope>, document_lengths: Option<&'scope [f32]>, parameters: &'scope Bm25Params, needle_weights: &'scope [f32], scores: &'scope mut [f32], scores_stride: usize) -> Result<(), Status>; fn report(&self) -> SubstringsReport; ``` `hot_states` sizes the automaton's dense tier, or takes `SUBSTRINGS_HOT_STATES_AUTO` to fill a fixed byte budget instead, which holds more states the fewer byte classes the vocabulary spells. `matches_budget` bounds what one round may emit and `haystacks_budget` how many haystacks it may carry, and both are read by a device tier alone, so a host engine takes `SUBSTRINGS_MATCHES_BUDGET_AUTO` and zero and walks straight into the caller's output. `CaseSensitivity::Cased` matches bytes exactly and accepts arbitrary needles, while `CaseSensitivity::Uncased` folds both sides under full Unicode case folding and requires valid UTF-8. Folding is not a byte-length-preserving operation, so a 1-byte needle can match a 3-byte span — the Kelvin sign `U+212A` folds to `k` — which is why every `Match` carries its own `byte_length` rather than borrowing the needle's. `SubstringsOverlapPolicy` decides what a walk reports, and it sizes the engine's arena, so it is fixed at construction and one engine runs exactly one of the three: - `Overlapping` — every match of every needle, nested and overlapping ones included. - `LeftmostLongest` — a cover taking the widest match at the earliest start. - `LeftmostFirst` — a cover taking the lowest needle index at the earliest start. A capacity shortfall is not an error. The sizing walk always runs, so `report()` names `matches_emitted`, the true total; `matches_stored`, what was written; `target_length`, what a rewrite needs; and `shortfall`, what did not fit. That is what lets one call with an empty output size the next one, with no walk in between. ```rust use stringzilla::{ Capabilities, CaseSensitivity, Stream, StringsView, SubstringsEngine, Match, SubstringsOverlapPolicy, SUBSTRINGS_HOT_STATES_AUTO, SUBSTRINGS_MATCHES_BUDGET_AUTO, }; let cpu = Stream::default(Capabilities::cpu_enabled()); let mut engine = SubstringsEngine::new( &["cat", "catalog"], CaseSensitivity::Cased, SubstringsOverlapPolicy::Overlapping, SUBSTRINGS_HOT_STATES_AUTO, SUBSTRINGS_MATCHES_BUDGET_AUTO, 0, &cpu, ) .unwrap(); let documents = StringsView::from_slices(&["a catalog of cats", "nothing here"]); let mut counts = [0usize; 2]; cpu.scope(|scope| engine.counts(scope, documents, &mut counts, 1)).unwrap(); assert_eq!(counts, [3, 0]); // "catalog", the "cat" inside it, and the "cat" of "cats" // `find` fills one boundary per haystack plus a final total, whether or not the matches fit. let mut matches = [Match::default(); 3]; let mut offsets = [0usize; 3]; cpu.scope(|scope| engine.find(scope, documents, &mut matches, &mut offsets)).unwrap(); assert_eq!(offsets, [0, 3, 3]); assert_eq!(engine.report().matches_emitted, 3); assert_eq!(engine.report().shortfall, 0); ``` #### Scoring and Rewriting The same automaton scores documents with BM25 and rewrites them, each in a single walk. `bm25_scores` treats the dictionary itself as the query — `needle_weights[i]` is needle `i`'s IDF or boost — and writes one score per haystack, so a many-term query over a large corpus never materializes per-term frequency rows. Term frequencies are raw overlapping counts, which is classic BM25, so the engine's own policy does not apply here, and `document_lengths` falls back to each haystack's byte length when `None`. `Bm25Params` has no `Default`, because a corpus mean has no correct default value. Its two constructors name the two configurations that exist: `Bm25Params::normalized(mean)` is the literature's `k1 = 1.2` and `b = 0.75` against a corpus whose mean document length you know, and `Bm25Params::unnormalized()` switches length normalization off and leaves `document_lengths` unread. `replace` takes one replacement per needle, inserted verbatim, an empty one deleting its match, and writes every rewritten haystack into one `target` buffer beside their boundaries. Rewriting is defined only under a cover, so an engine built with `SubstringsOverlapPolicy::Overlapping` is refused there — an overlapping rewrite is not a function. A `target` too small is not an error either: `report().target_length` names the bytes the rewrite needed and the buffer's contents are then unspecified, so an empty `target` is how the next call is sized. ```rust use stringzilla::{Bm25Params, Capabilities, CaseSensitivity, Stream, StringsView, SubstringsEngine, SubstringsOverlapPolicy, SUBSTRINGS_HOT_STATES_AUTO, SUBSTRINGS_MATCHES_BUDGET_AUTO}; let cpu = Stream::default(Capabilities::cpu_enabled()); let mut engine = SubstringsEngine::new( &["cat", "dog"], CaseSensitivity::Cased, SubstringsOverlapPolicy::LeftmostLongest, SUBSTRINGS_HOT_STATES_AUTO, SUBSTRINGS_MATCHES_BUDGET_AUTO, 0, &cpu, ) .unwrap(); let documents = StringsView::from_slices(&["cat and dog", "nothing here"]); let mut scores = [0.0f32; 2]; let parameters = Bm25Params::normalized(10.0); cpu.scope(|scope| engine.bm25_scores(scope, documents, None, ¶meters, &[1.0, 1.0], &mut scores, 1)) .unwrap(); assert_eq!(scores[1], 0.0); // One replacement per needle: an empty target sizes the rewrite, a second scope performs it. let replacements = StringsView::from_slices(&["feline", "canine"]); let mut offsets = [0usize; 3]; cpu.scope(|scope| engine.replace(scope, documents, replacements, &mut [], &mut offsets)).unwrap(); let mut target = vec![0u8; engine.report().target_length]; cpu.scope(|scope| engine.replace(scope, documents, replacements, &mut target, &mut offsets)).unwrap(); assert_eq!(&target[offsets[0]..offsets[1]], b"feline and canine"); ``` `use stringzilla::Matcher;` `use stringzilla::FindMatches;` `use stringzilla::RFindMatches;` `use stringzilla::StringZillableBinary;` `use stringzilla::RegexEngine;` `use stringzilla::SubdocumentsFormat;` `use stringzilla::SubdocumentsEngine;` `use stringzilla::SubstringsOverlapPolicy;` `use stringzilla::Match;` `use stringzilla::SubstringsReport;` `use stringzilla::Bm25Params;` `use stringzilla::SubstringsEngine;` `use stringzilla::IndexSpan;` `use stringzilla::Overlaps;` `use stringzilla::NonOverlapping;` `use stringzilla::Overlapping;` `use stringzilla::*;` ## Matcher What a match or split iterator looks for; the iterator type picks the direction. ```rust pub enum Matcher<'a> ``` - `Substring` = `(&'a [u8])`: An exact byte sequence. - `Bytes` = `(Byteset)`: Any one byte of the set; invert the set to match the bytes outside it. Implements: `Debug`, `Clone`, `Copy` ## find Offset of the first match in `haystack`. ```rust pub fn find(&self, haystack: &[u8]) -> Option ``` ## rfind Offset of the last match in `haystack`. ```rust pub fn rfind(&self, haystack: &[u8]) -> Option ``` ## width Bytes every match spans: the needle's length, or one for a byteset. ```rust pub fn width(&self) -> usize ``` ## FindMatches An iterator over the matches of a pattern in a byte slice, front to back. ```rust pub struct FindMatches<'a, Overlap: Overlaps = NonOverlapping> ``` #### Empty needle An empty needle matches at every offset from 0 to `haystack.len()`, so an `n`-byte haystack yields `n + 1` empty matches, mirroring `"abc".matches("").count() == 4`. Examples: ``` use stringzilla::{FindMatches, Matcher}; let matches: Vec<&[u8]> = FindMatches::new(b"abababa", Matcher::Substring(b"aba")).collect(); assert_eq!(matches, vec![b"aba", b"aba"]); ``` Implements: `Iterator` ## new Iterates the non-overlapping matches of `matcher` in `haystack`, front to back. ```rust pub fn new(haystack: &'a [u8], matcher: Matcher<'a>) -> Self ``` ## * Report overlapping matches too, a compile-time policy returning the [`Overlapping`](#Overlapping) variant. ```rust pub fn overlapping(self) -> FindMatches<'a, Overlapping> ``` ## RFindMatches An iterator over the matches of a pattern in a byte slice, back to front. ```rust pub struct RFindMatches<'a, Overlap: Overlaps = NonOverlapping> ``` #### Empty needle An empty needle matches at every offset from `haystack.len()` down to 0, so an `n`-byte haystack yields `n + 1` empty matches. Examples: ``` use stringzilla::{Matcher, RFindMatches}; let matches: Vec<&[u8]> = RFindMatches::new(b"abababa", Matcher::Substring(b"aba")).collect(); assert_eq!(matches, vec![b"aba", b"aba"]); ``` Implements: `Iterator` ## new Iterates the non-overlapping matches of `matcher` in `haystack`, back to front. ```rust pub fn new(haystack: &'a [u8], matcher: Matcher<'a>) -> Self ``` ## StringZillableBinary Trait for binary string operations that take a needle parameter. ```rust pub trait StringZillableBinary<'a, Needle> where Needle: AsRef<[u8]> + 'a, ``` These operations include searching, splitting, and pattern matching. Examples: Basic usage on a string slice: ``` use stringzilla::StringZillableBinary; let haystack = "Hello, world!"; assert_eq!(haystack.sz_find("world".as_bytes()), Some(7)); ``` Implementors: - `impl<'a, Source, Needle> [StringZillableBinary](#StringZillableBinary)<'a, Needle> for Source where Source: AsRef<[u8]> + ?Sized, Needle: AsRef<[u8]> + 'a` ## sz_find Searches for the first occurrence of `needle` in `self`. ```rust fn sz_find(&self, needle: Needle) -> Option ``` Examples: ``` use stringzilla::StringZillableBinary; let haystack = "Hello, world!"; assert_eq!(haystack.sz_find("world".as_bytes()), Some(7)); ``` ## sz_rfind Searches for the last occurrence of `needle` in `self`. ```rust fn sz_rfind(&self, needle: Needle) -> Option ``` Examples: ``` use stringzilla::StringZillableBinary; let haystack = "Hello, world, world!"; assert_eq!(haystack.sz_rfind("world".as_bytes()), Some(14)); ``` ## sz_find_byte_from Finds the index of the first character in `self` that is also present in `needles`. ```rust fn sz_find_byte_from(&self, needles: Needle) -> Option ``` Examples: ``` use stringzilla::StringZillableBinary; let haystack = "Hello, world!"; assert_eq!(haystack.sz_find_byte_from("aeiou".as_bytes()), Some(1)); ``` ## sz_rfind_byte_from Finds the index of the last character in `self` that is also present in `needles`. ```rust fn sz_rfind_byte_from(&self, needles: Needle) -> Option ``` Examples: ``` use stringzilla::StringZillableBinary; let haystack = "Hello, world!"; assert_eq!(haystack.sz_rfind_byte_from("aeiou".as_bytes()), Some(8)); ``` ## sz_find_byte_not_from Finds the index of the first character in `self` that is not present in `needles`. ```rust fn sz_find_byte_not_from(&self, needles: Needle) -> Option ``` Examples: ``` use stringzilla::StringZillableBinary; let haystack = "Hello, world!"; assert_eq!(haystack.sz_find_byte_not_from("aeiou".as_bytes()), Some(0)); ``` ## sz_rfind_byte_not_from Finds the index of the last character in `self` that is not present in `needles`. ```rust fn sz_rfind_byte_not_from(&self, needles: Needle) -> Option ``` Examples: ``` use stringzilla::StringZillableBinary; let haystack = "Hello, world!"; assert_eq!(haystack.sz_rfind_byte_not_from("aeiou".as_bytes()), Some(12)); ``` ## sz_matches Returns an iterator over all non-overlapping matches of the given `needle` in `self`. ```rust fn sz_matches(&'a self, needle: &'a Needle) -> FindMatches<'a> ``` Arguments: - `needle`: The byte slice to search for within `self`. Examples: ``` use stringzilla::StringZillableBinary; let haystack = b"abababa"; let needle = b"aba"; let matches: Vec<&[u8]> = haystack.sz_matches(needle).collect(); assert_eq!(matches, vec![b"aba", b"aba"]); // non-overlapping by default (like str::matches) let overlapping: Vec<&[u8]> = haystack.sz_matches(needle).overlapping().collect(); assert_eq!(overlapping, vec![b"aba", b"aba", b"aba"]); // opt in with .overlapping() ``` ## sz_rmatches Returns an iterator over all non-overlapping matches of the given `needle` in `self`, searching from the end. ```rust fn sz_rmatches(&'a self, needle: &'a Needle) -> RFindMatches<'a> ``` Arguments: - `needle`: The byte slice to search for within `self`. Examples: ``` use stringzilla::StringZillableBinary; let haystack = b"abababa"; let needle = b"aba"; let matches: Vec<&[u8]> = haystack.sz_rmatches(needle).collect(); assert_eq!(matches, vec![b"aba", b"aba"]); // non-overlapping by default let overlapping: Vec<&[u8]> = haystack.sz_rmatches(needle).overlapping().collect(); assert_eq!(overlapping, vec![b"aba", b"aba", b"aba"]); // opt in with .overlapping() ``` ## sz_splits Returns an iterator over the substrings of `self` that are separated by the given `needle`. ```rust fn sz_splits(&'a self, needle: &'a Needle) -> FindSplits<'a> ``` Arguments: - `needle`: The byte slice to split `self` by. Examples: ``` use stringzilla::StringZillableBinary; let haystack = b"a,b,c,d"; let needle = b","; let splits: Vec<&[u8]> = haystack.sz_splits(needle).collect(); assert_eq!(splits, vec![b"a", b"b", b"c", b"d"]); ``` ## sz_rsplits Returns an iterator over the substrings of `self` that are separated by the given `needle`, searching from the end. ```rust fn sz_rsplits(&'a self, needle: &'a Needle) -> RFindSplits<'a> ``` Arguments: - `needle`: The byte slice to split `self` by. Examples: ``` use stringzilla::StringZillableBinary; let haystack = b"a,b,c,d"; let needle = b","; let splits: Vec<&[u8]> = haystack.sz_rsplits(needle).collect(); assert_eq!(splits, vec![b"d", b"c", b"b", b"a"]); ``` ## sz_find_first_of Returns an iterator over all non-overlapping matches of any of the bytes in `needles` within `self`. ```rust fn sz_find_first_of(&'a self, needles: &'a Needle) -> FindMatches<'a> ``` Arguments: - `needles`: The set of bytes to search for within `self`. Examples: ``` use stringzilla::StringZillableBinary; let haystack = b"Hello, world!"; let needles = b"aeiou"; let matches: Vec<&[u8]> = haystack.sz_find_first_of(needles).collect(); assert_eq!(matches, vec![b"e", b"o", b"o"]); ``` ## sz_find_last_of Returns an iterator over all non-overlapping matches of any of the bytes in `needles` within `self`, searching from the end. ```rust fn sz_find_last_of(&'a self, needles: &'a Needle) -> RFindMatches<'a> ``` Arguments: - `needles`: The set of bytes to search for within `self`. Examples: ``` use stringzilla::StringZillableBinary; let haystack = b"Hello, world!"; let needles = b"aeiou"; let matches: Vec<&[u8]> = haystack.sz_find_last_of(needles).collect(); assert_eq!(matches, vec![b"o", b"o", b"e"]); ``` ## sz_find_first_not_of Returns an iterator over all non-overlapping matches of any byte not in `needles` within `self`. ```rust fn sz_find_first_not_of(&'a self, needles: &'a Needle) -> FindMatches<'a> ``` Arguments: - `needles`: The set of bytes that should not be matched within `self`. Examples: ``` use stringzilla::StringZillableBinary; let haystack = b"Hello, world!"; let needles = b"aeiou"; let matches: Vec<&[u8]> = haystack.sz_find_first_not_of(needles).collect(); assert_eq!(matches, vec![b"H", b"l", b"l", b",", b" ", b"w", b"r", b"l", b"d", b"!"]); ``` ## sz_find_last_not_of Returns an iterator over all non-overlapping matches of any byte not in `needles` within `self`, searching from the end. ```rust fn sz_find_last_not_of(&'a self, needles: &'a Needle) -> RFindMatches<'a> ``` Arguments: - `needles`: The set of bytes that should not be matched within `self`. Examples: ``` use stringzilla::StringZillableBinary; let haystack = b"Hello, world!"; let needles = b"aeiou"; let matches: Vec<&[u8]> = haystack.sz_find_last_not_of(needles).collect(); assert_eq!(matches, vec![b"!", b"d", b"l", b"r", b"w", b" ", b",", b"l", b"l", b"H"]); ``` ## RegexEngine A vocabulary of RE2-syntax patterns compiled into two automata, walked over as many batches of haystacks as a caller has. ```rust pub struct RegexEngine ``` Matches are leftmost-first across the union of patterns, ties going to the lower pattern index, which is what Rust's `regex`, RE2 and Python's `re` report; a [`Match`](#Match) names its pattern as `needle_index`. Every haystack streams through the automata once whatever the pattern count. The fields mirror `sz_regex_engine_t` one for one and only [`patterns_count`](#patterns_count) is read from Rust, so the layout is load-bearing and the engine travels to C by pointer. Every verb grows the engine's round scratch, so it takes `&mut self`. Examples: ``` use stringzilla::{Capabilities, CaseSensitivity, Match, RegexEngine, Stream, StringsView}; let cpu = Stream::default(Capabilities::cpu_enabled()); let mut engine = RegexEngine::new(&["[0-9]+", "[a-z]+"], CaseSensitivity::Cased, &cpu).unwrap(); let documents = StringsView::from_slices(&["abc 123 de", "!"]); let (mut matches, mut offsets) = ([Match::default(); 4], [0usize; 3]); cpu.scope(|scope| engine.find(scope, documents, &mut matches, &mut offsets)).unwrap(); assert_eq!(offsets, [0, 3, 3]); assert_eq!((matches[1].needle_index, matches[1].byte_offset, matches[1].byte_length), (0, 4, 3)); ``` Implements: `Drop` ## new Compiles `patterns` with the capabilities of `stream`, like [`Capabilities::cpu_enabled`](https://www.unum.cloud/docs/stringzilla/rust/runtime-dispatch.md#cpu_enabled), on the device `stream` names. ```rust pub fn new( patterns: &[Pattern], case_sensitivity: CaseSensitivity, stream: &Stream, ) -> Result where Pattern: AsRef<[u8]>, ``` A pattern outside the supported syntax, like a backreference, a capturing group inside a lookaround or an inline `(?i)`, is refused as [`StringsError::Malformed`](https://www.unum.cloud/docs/stringzilla/rust/types.md#Malformed) with its index; case folding is the engine-wide `case_sensitivity` instead. ## patterns_count Patterns the vocabulary holds, which bounds every reported `needle_index`. ```rust pub fn patterns_count(&self) -> usize ``` ## counts Queues on the stream of [`scope`](https://www.unum.cloud/docs/stringzilla/rust/runtime-dispatch.md#scope) the count of every haystack's leftmost-first matches, one count per haystack. ```rust pub fn counts<'scope>( &'scope mut self, scope: &'scope Scope<'scope, '_>, haystacks: StringsView<'scope>, counts: &'scope mut [usize], counts_stride: usize, ) -> Result<(), Status> ``` [`counts`](#counts) receives haystack `h` at `counts[h * counts_stride]`, the stride counting entries rather than bytes and being at least one. A stream of another group than the engine's is refused with [`Status::DeviceMemoryMismatch`](https://www.unum.cloud/docs/stringzilla/rust/types.md#DeviceMemoryMismatch), here and by every verb below. ## find Queues on the stream of [`scope`](https://www.unum.cloud/docs/stringzilla/rust/runtime-dispatch.md#scope) a report of every leftmost-first match, ascending by haystack, then by start. ```rust pub fn find<'scope>( &'scope mut self, scope: &'scope Scope<'scope, '_>, haystacks: StringsView<'scope>, matches: &'scope mut [Match], matches_offsets: &'scope mut [usize], ) -> Result<(), Status> ``` `matches_offsets` holds one boundary per haystack plus a final total, filled whether or not the matches fit, which sizes the next call; an empty `matches` makes the call a size query. ## replace Rewrites every haystack into one `target` buffer, substituting each match with its pattern's replacement, on the CPU only. ```rust pub fn replace<'scope>( &'scope mut self, scope: &'scope Scope<'scope, '_>, haystacks: StringsView<'scope>, replacements: StringsView<'scope>, target: &'scope mut [u8], offsets: &'scope mut [usize], ) -> Result<(), Status> ``` `replacements` is indexed by pattern, so it holds exactly [`RegexEngine::patterns_count`](#patterns_count) entries; a replacement names groups as `$1`, `${1}` or `${name}`, `$0` being the whole match and `$$` one dollar. `offsets` holds one boundary per haystack plus a final total and is filled whether or not `target` held the result, which sizes the next call. ## extract Queues on the stream of [`scope`](https://www.unum.cloud/docs/stringzilla/rust/runtime-dispatch.md#scope) the groups of each haystack's first match of `pattern_index`, as Pandas [`str.extract`](#extract) reads them. ```rust pub fn extract<'scope>( &'scope mut self, scope: &'scope Scope<'scope, '_>, haystacks: StringsView<'scope>, pattern_index: usize, groups: &'scope mut [Match], groups_offsets: &'scope mut [usize], ) -> Result<(), Status> ``` A haystack with such a match gets a [`Match`](#Match) row for the whole match, `needle_index` zero, then one per group that took part, `needle_index` naming the group, so a group left out differs from an empty one; a haystack without one gets no rows. `groups_offsets` holds one boundary per haystack plus a final total, filled whether or not the rows fit, which sizes the next call; an empty `groups` makes the call a size query. ``` use stringzilla::{Capabilities, CaseSensitivity, Match, RegexEngine, Stream, StringsView}; let cpu = Stream::default(Capabilities::cpu_enabled()); let mut engine = RegexEngine::new(&["(\\d+)-(\\d+)"], CaseSensitivity::Cased, &cpu).unwrap(); let documents = StringsView::from_slices(&["a 12-34", "none"]); let (mut groups, mut offsets) = ([Match::default(); 3], [0usize; 3]); cpu.scope(|scope| engine.extract(scope, documents, 0, &mut groups, &mut offsets)).unwrap(); assert_eq!(offsets, [0, 3, 3]); assert_eq!((groups[2].needle_index, groups[2].byte_offset, groups[2].byte_length), (2, 5, 2)); ``` ## SubdocumentsFormat The grammar of every document an engine reads, and the path language that addresses it. ```rust pub enum SubdocumentsFormat ``` Corresponds to `sz_subdocuments_format_t` in the C API. - `Json` = `0`: RFC 8259 documents, addressed by RFC 9535 JSONPath. - `Xml` = `1`: XML 1.0 documents, addressed by XPath 1.0 location paths along forward axes. Implements: `Debug`, `Clone`, `Copy`, `StructuralPartialEq`, `PartialEq`, `Eq` ## SubdocumentsEngine A vocabulary of paths compiled into one automaton over keys, walked over as many batches of documents as a caller has. ```rust pub struct SubdocumentsEngine ``` Every document is walked once whatever the path count, and no tree is built: a container no path can enter is skipped by counting its brackets. A [`Match`](#Match) spans the selected node's exact bytes inside its document, in JSON a string with its quotes, a number or literal, or a container through its closing bracket, and in XML an element, an attribute's value or a text node, which a caller reads in place or hands to another parser. The fields mirror `sz_subdocuments_engine_t` one for one and only [`paths_count`](#paths_count) and [`format`](#format) are read from Rust, so the layout is load-bearing and the engine travels to C by pointer. The regex engine of `match()` and `search()` filters is embedded by value and freed by the C side, hence `ManuallyDrop`. Every verb grows the engine's depth stack, so it takes `&mut self`. Examples: ``` use stringzilla::{Capabilities, Match, Stream, StringsView, SubdocumentsEngine, SubdocumentsFormat}; let cpu = Stream::default(Capabilities::cpu_enabled()); let paths = ["$.store.book[?@.price < 10].title"]; let mut engine = SubdocumentsEngine::new(&paths, SubdocumentsFormat::Json, &cpu).unwrap(); let document = r#"{"store": {"book": [{"title": "A", "price": 8}, {"title": "B", "price": 12}]}}"#; let documents = [document]; let documents = StringsView::from_slices(&documents); let (mut matches, mut offsets, mut malformed_index) = ([Match::default(); 4], [0usize; 2], 0); cpu.scope(|scope| engine.find(scope, documents, &mut matches, &mut offsets, &mut malformed_index)).unwrap(); assert_eq!(offsets[1], 1); let title = &matches[0]; assert_eq!(&document[title.byte_offset..title.byte_offset + title.byte_length], "\"A\""); ``` Implements: `Drop` ## new Compiles `paths` for [`format`](#format) with the capabilities of `stream`, like [`Capabilities::cpu_enabled`](https://www.unum.cloud/docs/stringzilla/rust/runtime-dispatch.md#cpu_enabled), on the device `stream` names. ```rust pub fn new( paths: &[Path], format: SubdocumentsFormat, stream: &Stream, ) -> Result where Path: AsRef<[u8]>, ``` JSON paths follow RFC 9535, filters and their functions included, and XML paths follow XPath 1.0 along forward axes. A path that breaks its grammar, or that no single walk can evaluate, like an absolute query inside a filter, is refused as [`StringsError::Malformed`](https://www.unum.cloud/docs/stringzilla/rust/types.md#Malformed) with its index. ## paths_count Paths the vocabulary holds, which bounds every reported `needle_index`. ```rust pub fn paths_count(&self) -> usize ``` ## format The grammar every document of every verb follows. ```rust pub fn format(&self) -> SubdocumentsFormat ``` ## counts Queues on the stream of [`scope`](https://www.unum.cloud/docs/stringzilla/rust/runtime-dispatch.md#scope) the count of every document's nodes that some path selects, once per path, one count per document. ```rust pub fn counts<'scope>( &'scope mut self, scope: &'scope Scope<'scope, '_>, documents: StringsView<'scope>, counts: &'scope mut [usize], counts_stride: usize, malformed_index: &'scope mut usize, ) -> Result<(), Status> ``` [`counts`](#counts) receives document `d` at `counts[d * counts_stride]`, the stride counting entries rather than bytes and being at least one. A malformed document fails the call, and `malformed_index` then names the first one once the scope joins. ## find Queues on the stream of [`scope`](https://www.unum.cloud/docs/stringzilla/rust/runtime-dispatch.md#scope) a report of every node of every document that some path selects, by document, then by start offset, then by path, so a container precedes what it holds. ```rust pub fn find<'scope>( &'scope mut self, scope: &'scope Scope<'scope, '_>, documents: StringsView<'scope>, matches: &'scope mut [Match], matches_offsets: &'scope mut [usize], malformed_index: &'scope mut usize, ) -> Result<(), Status> ``` `matches_offsets` holds one boundary per document plus a final total, and is filled whether or not the matches fit, which is what sizes the next call; an empty `matches` makes the call a pure size query. A malformed document fails the call, and `malformed_index` then names the first one once the scope joins. ## SubstringsOverlapPolicy How matches that share bytes resolve: reported in full, or thinned to a leftmost run. ```rust pub enum SubstringsOverlapPolicy ``` The policy sizes the engine's arena, so it is fixed at construction rather than travelling per call, and one engine runs exactly one of the three. Corresponds to `sz_substrings_overlap_policy_t` in the C API. - `Overlapping` = `0`: Every match of every needle, including ones that share bytes and ones nested in others. - `LeftmostLongest` = `1`: Matches sharing no bytes: earliest start, then longest span, then lower needle index. - `LeftmostFirst` = `2`: Matches sharing no bytes: earliest start, then lower needle index, however long the rival. Implements: `Debug`, `Clone`, `Copy`, `StructuralPartialEq`, `PartialEq`, `Eq` ## Match One reported match, locating it by haystack, by needle, and by byte span. ```rust pub struct Match ``` Under case folding a needle's own byte length is not the length of every match - needle `k` matches both the 1-byte `k` and the 3-byte Kelvin sign `U+212A` - so the span travels per match. Corresponds to `sz_match_t` in the C API. - `haystack_index` `usize`: Which haystack of the sequence this match was found in. - `needle_index` `usize`: Which needle of the vocabulary matched. - `byte_offset` `usize`: Where the match starts inside that haystack, in its own bytes. - `byte_length` `usize`: Haystack bytes the match spans, which folding can make differ from the needle's own length. Implements: `Debug`, `Clone`, `Copy`, `Default`, `StructuralPartialEq`, `PartialEq`, `Eq` ## SubstringsReport What a round found, which is how a capacity shortfall is reported rather than as an error. ```rust pub struct SubstringsReport ``` The sizing walk always runs, so `matches_emitted` is the truth whatever the caller's output could hold, and a nonzero `shortfall` is the one signal that an output is incomplete rather than wrong. Corresponds to `sz_substrings_report_t` in the C API. - `matches_emitted` `usize`: Matches the sizing walk found, which is the truth whatever the output held. - `matches_stored` `usize`: Matches written out, which is `matches_emitted` clipped at the capacity. - `target_length` `usize`: Bytes a rewrite needs, which is the truth whatever the target held. - `shortfall` `usize`: Matches or bytes the round could not hold, zero when everything fit. Implements: `Debug`, `Clone`, `Copy`, `Default`, `StructuralPartialEq`, `PartialEq`, `Eq` ## Bm25Params BM25's continuous parameters. ```rust pub struct Bm25Params ``` There is no correct default for the corpus mean, so there is no `Default`: reach for [`Bm25Params::normalized`](#normalized) or [`Bm25Params::unnormalized`](#unnormalized), which name the two configurations that exist. A positive `length_normalization` beside a non-positive `average_document_length` is refused, since it would divide by a mean that is not there. Corresponds to `sz_substrings_bm25_t` in the C API. - `term_frequency_saturation` `f32`: The literature's `k1`: how slowly repeated occurrences stop adding score; 1.2 is customary. - `length_normalization` `f32`: The literature's `b`, in `[0, 1]`: 0 ignores document length, 1 normalizes it fully. - `average_document_length` `f32`: The corpus-wide mean document length, in the unit of the lengths scored; read only when `length_normalization` is positive. Implements: `Debug`, `Clone`, `Copy`, `StructuralPartialEq`, `PartialEq` ## normalized Classic BM25 with the literature's customary `k1 = 1.2` and `b = 0.75`, normalized against a corpus whose mean document length is `average_document_length`, in the unit the per-document lengths use. ```rust pub const fn normalized(average_document_length: f32) -> Self ``` ## unnormalized BM25 with length normalization switched off, for a corpus whose mean length is unknown or whose documents are uniform enough not to need it; the per-document lengths then go unread. ```rust pub const fn unnormalized() -> Self ``` ## SubstringsEngine A vocabulary compiled into one automaton, walked over as many batches of haystacks as a caller has. ```rust pub struct SubstringsEngine ``` Building the automaton is the expensive half and every verb below reuses it, so one long-lived engine amortizes that across every later batch: a dictionary of thousands of terms costs one pass over a haystack rather than thousands. The fields mirror `sz_substrings_engine_t` one for one and only [`needles_count`](#needles_count), [`overlap_policy`](#overlap_policy) and [`report`](#report) are read from Rust, so the layout is load-bearing and the engine travels to C by pointer. Owning raw pointers makes it neither `Send` nor `Sync`, which is what the C contract wants: a compute verb writes the engine's round arena, so it mutates, and every verb below takes `&mut self` for that reason. Examples: ``` use stringzilla::{ Capabilities, CaseSensitivity, Stream, StringsView, SubstringsEngine, SubstringsOverlapPolicy, SUBSTRINGS_HOT_STATES_AUTO, SUBSTRINGS_MATCHES_BUDGET_AUTO, }; let cpu = Stream::default(Capabilities::cpu_enabled()); let mut engine = SubstringsEngine::new( &["cat", "catalog"], CaseSensitivity::Cased, SubstringsOverlapPolicy::Overlapping, SUBSTRINGS_HOT_STATES_AUTO, SUBSTRINGS_MATCHES_BUDGET_AUTO, 0, &cpu, ) .unwrap(); let documents = StringsView::from_slices(&["a catalog of cats", "nothing here"]); let mut counts = [0usize; 2]; cpu.scope(|scope| engine.counts(scope, documents, &mut counts, 1)).unwrap(); assert_eq!(counts, [3, 0]); // "catalog", the "cat" inside it, and the "cat" of "cats" ``` Implements: `Drop` ## new Compiles `needles` into an automaton the matching verbs read, with the capabilities of `stream`, like [`Capabilities::cpu_enabled`](https://www.unum.cloud/docs/stringzilla/rust/runtime-dispatch.md#cpu_enabled) or a GPU's [`Capabilities::cuda_enabled`](https://www.unum.cloud/docs/stringzilla/rust/runtime-dispatch.md#cuda_enabled), on the device `stream` names, which it may join. ```rust pub fn new( needles: &[Needle], case_sensitivity: CaseSensitivity, overlap_policy: SubstringsOverlapPolicy, hot_states: usize, matches_budget: usize, haystacks_budget: usize, stream: &Stream, ) -> Result where Needle: AsRef<[u8]>, ``` An empty needle is refused rather than skipped, since it would match at every position and dropping it would shift every later needle's reported index. `hot_states` is the count kept in the dense hot rows, or [`SUBSTRINGS_HOT_STATES_AUTO`](#SUBSTRINGS_HOT_STATES_AUTO) to fill a fixed byte budget instead. The budgets are read by a device kernel alone, so a host engine takes [`SUBSTRINGS_MATCHES_BUDGET_AUTO`](#SUBSTRINGS_MATCHES_BUDGET_AUTO) and zero and walks straight into the caller's output. `matches_budget` is what a device round may emit before every later kernel retires, and `haystacks_budget` how many haystacks it may carry; the round's arena is sized for both here, zero asking for a default, so no verb allocates, a round carrying more haystacks is refused, and matches past the budget surface as the report's `shortfall`. A GPU engine's verbs take haystacks, replacements and every buffer, weights and lengths included, in memory the device reaches, such as a [`Strings`](https://www.unum.cloud/docs/stringzilla/rust/types.md#Strings) and a `Vec` in a [`UnifiedAllocator`](https://www.unum.cloud/docs/stringzilla/rust/memory-operations.md#UnifiedAllocator). ## needles_count Needles the vocabulary holds, which bounds every reported `needle_index`. ```rust pub fn needles_count(&self) -> usize ``` ## overlap_policy The cover every round runs, fixed when the arena was sized. ```rust pub fn overlap_policy(&self) -> SubstringsOverlapPolicy ``` ## report What the last round found, which is how a capacity shortfall surfaces instead of as an error. ```rust pub fn report(&self) -> SubstringsReport ``` A verb borrows the engine until its scope joins, so the record a device wrote is complete. ## counts Queues on the stream of [`scope`](https://www.unum.cloud/docs/stringzilla/rust/runtime-dispatch.md#scope) the count of every needle's matches in every haystack, one count per haystack. ```rust pub fn counts<'scope>( &'scope mut self, scope: &'scope Scope<'scope, '_>, haystacks: StringsView<'scope>, counts: &'scope mut [usize], counts_stride: usize, ) -> Result<(), Status> ``` [`counts`](#counts) receives haystack `h` at `counts[h * counts_stride]`, the stride counting entries rather than bytes and being at least one, so a strided call writes one column of a `[haystacks, vocabularies]` feature matrix. A stream of another group than the engine's is refused with [`Status::DeviceMemoryMismatch`](https://www.unum.cloud/docs/stringzilla/rust/types.md#DeviceMemoryMismatch), here and by every verb below. ## find Queues on the stream of [`scope`](https://www.unum.cloud/docs/stringzilla/rust/runtime-dispatch.md#scope) a report of every match of every needle in every haystack, ascending by haystack. ```rust pub fn find<'scope>( &'scope mut self, scope: &'scope Scope<'scope, '_>, haystacks: StringsView<'scope>, matches: &'scope mut [Match], matches_offsets: &'scope mut [usize], ) -> Result<(), Status> ``` `matches_offsets` holds one boundary per haystack plus a final total, and is filled whether or not the matches fit, which is what sizes the next call. A capacity too small is not an error: [`SubstringsEngine::report`](#report) names the true total and what did not fit, so a sizing call with an empty `matches` followed by one filling call needs no walk in between. ## replace Queues on the stream of [`scope`](https://www.unum.cloud/docs/stringzilla/rust/runtime-dispatch.md#scope) a rewrite of every haystack into one `target` buffer, substituting one replacement per needle. ```rust pub fn replace<'scope>( &'scope mut self, scope: &'scope Scope<'scope, '_>, haystacks: StringsView<'scope>, replacements: StringsView<'scope>, target: &'scope mut [u8], offsets: &'scope mut [usize], ) -> Result<(), Status> ``` `replacements` is indexed by needle and an empty one deletes the match, so it holds exactly [`SubstringsEngine::needles_count`](#needles_count) entries. `offsets` holds one boundary per haystack plus a final total and is filled whether or not `target` held the result, which is what sizes the next call; a `target` too small leaves its contents unspecified rather than failing, and [`SubstringsEngine::report`](#report) names the bytes the rewrite needed. The engine's policy must be a cover, since a substitution over matches that share bytes is not a function. ## bm25_scores Queues on the stream of [`scope`](https://www.unum.cloud/docs/stringzilla/rust/runtime-dispatch.md#scope) the score of every haystack against the whole vocabulary as one BM25 query, one score per haystack. ```rust pub fn bm25_scores<'scope>( &'scope mut self, scope: &'scope Scope<'scope, '_>, haystacks: StringsView<'scope>, document_lengths: Option<&'scope [f32]>, parameters: &'scope Bm25Params, needle_weights: &'scope [f32], scores: &'scope mut [f32], scores_stride: usize, ) -> Result<(), Status> ``` The vocabulary is the query and `needle_weights` holds each needle's IDF or boost, so a many-term query over a large corpus never materializes per-term frequency rows. Term frequencies are raw overlapping counts, since a cover would suppress genuine occurrences of a needle nested in another, so the engine's own policy does not apply here. `document_lengths` is `None` to normalize by each haystack's byte length instead. [`scores`](https://www.unum.cloud/docs/stringzilla/rust/fingerprints.md#scores) receives haystack `h` at `scores[h * scores_stride]`, on the same convention as [`SubstringsEngine::counts`](#counts). ## IndexSpan Represents a byte span with offset and length. ```rust pub struct IndexSpan ``` Used for matches of UTF-8 characters, substrings, or any byte-level operations. Stores the byte offset from the start of the text and the length in bytes. - `offset` `usize`: Byte offset from the start of the text - `length` `usize`: Length in bytes of the matched span Examples: ``` use stringzilla::IndexSpan; let text = "Hello\nWorld"; let span = IndexSpan::new(5, 1); assert_eq!(span.offset, 5); assert_eq!(span.length, 1); let matched = span.extract(text.as_bytes()); assert_eq!(matched, Some(&b"\n"[..])); ``` Implements: `Debug`, `Clone`, `Copy`, `StructuralPartialEq`, `PartialEq`, `Eq`, `Hash` ## new Creates a new IndexSpan with the given offset and length. ```rust pub const fn new(offset: usize, length: usize) -> Self ``` ## range Returns the range of bytes covered by this span, ending at [`Self::end`](#end). ```rust pub const fn range(&self) -> Range ``` Examples: ``` use stringzilla::IndexSpan; let span = IndexSpan::new(5, 3); assert_eq!(span.range(), 5..8); ``` ## extract Extracts the matched bytes from the source text, or `None` if the span runs past its end. ```rust pub fn extract<'a>(&self, text: &'a [u8]) -> Option<&'a [u8]> ``` Examples: ``` use stringzilla::IndexSpan; let text = b"Hello World"; assert_eq!(IndexSpan::new(6, 5).extract(text), Some(&b"World"[..])); assert_eq!(IndexSpan::new(6, 6).extract(text), None); ``` ## end Returns the end offset, `offset + length`, saturating at `usize::MAX`. ```rust pub const fn end(&self) -> usize ``` Examples: ``` use stringzilla::IndexSpan; let span = IndexSpan::new(5, 3); assert_eq!(span.end(), 8); ``` ## Overlaps Compile-time policy for whether overlapping matches are reported - a named marker, not a raw `bool`. ```rust pub trait Overlaps ``` Implementors: - `impl [Overlaps](#Overlaps) for [NonOverlapping](#NonOverlapping)` - `impl [Overlaps](#Overlaps) for [Overlapping](#Overlapping)` ## OVERLAP Whether overlapping matches are included. ```rust const OVERLAP: bool; ``` ## NonOverlapping Report only non-overlapping matches, the default, like `str::matches`. ```rust pub struct NonOverlapping; ``` Implements: `[Overlaps](#Overlaps)` ## Overlapping Report overlapping matches too, via `.overlapping()`. ```rust pub struct Overlapping; ``` Implements: `[Overlaps](#Overlaps)` ## find Locates the first matching substring within `haystack` that equals `needle`. ```rust pub fn find(haystack: Haystack, needle: Needle) -> Option where Haystack: AsRef<[u8]>, Needle: AsRef<[u8]>, ``` This function is similar to the `memmem()` function in LibC, but, unlike `strstr()`, it requires the length of both haystack and needle to be known beforehand. #### Returns An `Option` representing the starting index of the first occurrence of `needle` within `haystack` if found, otherwise `None`. #### Empty needle The C core returns the start of `haystack` for an empty needle, like `strstr`, so `find(haystack, b"")` is always `Some(0)`, matching `"abc".find("") == Some(0)`. This holds even for an empty `haystack`. Arguments: - `haystack`: The byte slice to search. - `needle`: The byte slice to find within the haystack. ## rfind Locates the last matching substring within `haystack` that equals `needle`. ```rust pub fn rfind(haystack: Haystack, needle: Needle) -> Option where Haystack: AsRef<[u8]>, Needle: AsRef<[u8]>, ``` This function is useful for finding the most recent or last occurrence of a pattern within a byte slice. #### Returns An `Option` representing the starting index of the last occurrence of `needle` within `haystack` if found, otherwise `None`. #### Empty needle The C core returns the end of `haystack` for an empty needle, the reverse mirror of `strstr`, so `rfind(haystack, b"")` is always `Some(haystack.len())`, matching `"abc".rfind("") == Some(3)`. This holds even for an empty `haystack`. Arguments: - `haystack`: The byte slice to search. - `needle`: The byte slice to find within the haystack. ## contains Checks whether `needle` occurs anywhere within `haystack`. ```rust pub fn contains(haystack: Haystack, needle: Needle) -> bool where Haystack: AsRef<[u8]>, Needle: AsRef<[u8]>, ``` #### Returns `true` if `needle` occurs within `haystack`, `false` otherwise. #### Empty needle Mirrors [`str::contains`](https://www.unum.cloud/docs/stringzilla/rust/runtime-dispatch.md#contains): an empty needle is always present, so `contains(haystack, b"")` is always `true`, matching `"abc".contains("") == true`, even for an empty `haystack`. Arguments: - `haystack`: The byte slice to search. - `needle`: The byte slice to look for within the haystack. ## find_byteset Finds the index of the first character in `haystack` that is also present in `needles`. ```rust pub fn find_byteset(haystack: Haystack, needles: Byteset) -> Option where Haystack: AsRef<[u8]>, ``` This function is particularly useful for parsing and tokenization tasks where a set of delimiter characters is used. #### Returns An `Option` representing the index of the first occurrence of any byte from `needles` within `haystack`, if found, otherwise `None`. Arguments: - `haystack`: The byte slice to search. - `needles`: The set of bytes to search for within the haystack. ## rfind_byteset Finds the index of the last character in `haystack` that is also present in `needles`. ```rust pub fn rfind_byteset(haystack: Haystack, needles: Byteset) -> Option where Haystack: AsRef<[u8]>, ``` This can be used to find the last occurrence of any character from a specified set, useful in parsing scenarios such as finding the last delimiter in a string. #### Returns An `Option` representing the index of the last occurrence of any byte from `needles` within `haystack`, if found, otherwise `None`. Arguments: - `haystack`: The byte slice to search. - `needles`: The set of bytes to search for within the haystack. ## find_byte_from Finds the index of the first character in `haystack` that is also present in `needles`. ```rust pub fn find_byte_from(haystack: Haystack, needles: Needle) -> Option where Haystack: AsRef<[u8]>, Needle: AsRef<[u8]>, ``` This function is particularly useful for parsing and tokenization tasks where a set of delimiter characters is used. #### Returns An `Option` representing the index of the first occurrence of any byte from `needles` within `haystack`, if found, otherwise `None`. Arguments: - `haystack`: The byte slice to search. - `needles`: The set of bytes to search for within the haystack. ## rfind_byte_from Finds the index of the last character in `haystack` that is also present in `needles`. ```rust pub fn rfind_byte_from(haystack: Haystack, needles: Needle) -> Option where Haystack: AsRef<[u8]>, Needle: AsRef<[u8]>, ``` This can be used to find the last occurrence of any character from a specified set, useful in parsing scenarios such as finding the last delimiter in a string. #### Returns An `Option` representing the index of the last occurrence of any byte from `needles` within `haystack`, if found, otherwise `None`. Arguments: - `haystack`: The byte slice to search. - `needles`: The set of bytes to search for within the haystack. ## find_byte_not_from Finds the index of the first character in `haystack` that is not present in `needles`. ```rust pub fn find_byte_not_from(haystack: Haystack, needles: Needle) -> Option where Haystack: AsRef<[u8]>, Needle: AsRef<[u8]>, ``` This function is useful for skipping over a known set of characters and finding the first character that does not belong to that set. #### Returns An `Option` representing the index of the first occurrence of any byte not in `needles` within `haystack`, if found, otherwise `None`. Arguments: - `haystack`: The byte slice to search. - `needles`: The set of bytes that should not be matched within the haystack. ## rfind_byte_not_from Finds the index of the last character in `haystack` that is not present in `needles`. ```rust pub fn rfind_byte_not_from(haystack: Haystack, needles: Needle) -> Option where Haystack: AsRef<[u8]>, Needle: AsRef<[u8]>, ``` Useful for text processing tasks such as trimming trailing characters that belong to a specified set. #### Returns An `Option` representing the index of the last occurrence of any byte not in `needles` within `haystack`, if found, otherwise `None`. Arguments: - `haystack`: The byte slice to search. - `needles`: The set of bytes that should not be matched within the haystack. ## count_utf8 Counts the number of UTF-8 characters in the text. ```rust pub fn count_utf8(text: Text) -> usize where Text: AsRef<[u8]>, ``` This function efficiently counts UTF-8 characters by identifying character start bytes, the non-continuation bytes. Uses SIMD acceleration when available. #### Returns The number of UTF-8 characters, or codepoints, in the text. Arguments: - `text`: The UTF-8 encoded byte slice to count characters in. Examples: ``` use stringzilla as sz; let text = "Hello"; assert_eq!(sz::count_utf8(text), 5); let text_unicode = "Hello🌍"; assert_eq!(sz::count_utf8(text_unicode), 6); let text_cjk = "你好世界"; assert_eq!(sz::count_utf8(text_cjk), 4); ``` ## find_nth_utf8 Finds the byte offset of the Nth UTF-8 character (0-indexed). ```rust pub fn find_nth_utf8(text: Text, n: usize) -> Option where Text: AsRef<[u8]>, ``` This function efficiently locates the Nth UTF-8 character without decoding the entire string. Uses SIMD acceleration when available. #### Returns An `Option` containing the byte offset of the Nth character. Returns `None` if the string has fewer than N+1 characters. Arguments: - `text`: The UTF-8 encoded byte slice to search. - `n`: The 0-based index of the character to find. Examples: ``` use stringzilla as sz; let text = "Hello"; assert_eq!(sz::find_nth_utf8(text, 0), Some(0)); // 'H' assert_eq!(sz::find_nth_utf8(text, 4), Some(4)); // 'o' assert_eq!(sz::find_nth_utf8(text, 5), None); let text_unicode = "Hello🌍"; assert_eq!(sz::find_nth_utf8(text_unicode, 5), Some(5)); // 🌍 starts at byte 5 assert_eq!(sz::find_nth_utf8(text_unicode, 6), None); ``` ## SUBSTRINGS_HOT_STATES_AUTO The `hot_states` argument's "size the hot tier yourself" value, so zero stays a real all-cold request. ```rust pub const SUBSTRINGS_HOT_STATES_AUTO: usize = usize::MAX; ``` ## SUBSTRINGS_MATCHES_BUDGET_AUTO The `matches_budget` argument's "let the tier choose" value; a host tier reads no budget at all. ```rust pub const SUBSTRINGS_MATCHES_BUDGET_AUTO: usize = 0; ```