Rust · Searching and Counting

Searching and Counting

use stringzilla::Matcher;use stringzilla::FindMatches;use stringzilla::RFindMatches;use stringzilla::StringZillableBinary;use stringzilla::RegexEngine;use stringzilla::SubdocumentsFormat;use stringzilla::SubdocumentsEngine;use stringzilla::SubstringsOverlapPolicy;use stringzilla::Match;use stringzilla::SubstringsReport;use stringzilla::Bm25Params;use stringzilla::SubstringsEngine;use stringzilla::IndexSpan;use stringzilla::Overlaps;use stringzilla::NonOverlapping;use stringzilla::Overlapping;use stringzilla::*; 6.0.0-rc.1 · 91a27bdd

All search functions accept any AsRef<[u8]> and return byte offsets as Option<usize>. They are available both as sz:: free functions and as StringZillableBinary trait methods.

Free functions:

pub fn find<H: AsRef<[u8]>, N: AsRef<[u8]>>(haystack: H, needle: N) -> Option<usize>;
pub fn rfind<H: AsRef<[u8]>, N: AsRef<[u8]>>(haystack: H, needle: N) -> Option<usize>;

pub fn find_byteset<H: AsRef<[u8]>>(haystack: H, needles: Byteset) -> Option<usize>;
pub fn rfind_byteset<H: AsRef<[u8]>>(haystack: H, needles: Byteset) -> Option<usize>;

pub fn find_byte_from<H, N>(haystack: H, needles: N) -> Option<usize>;      // first byte in the set
pub fn rfind_byte_from<H, N>(haystack: H, needles: N) -> Option<usize>;     // last  byte in the set
pub fn find_byte_not_from<H, N>(haystack: H, needles: N) -> Option<usize>;  // first byte not in the set
pub fn rfind_byte_not_from<H, N>(haystack: H, needles: N) -> Option<usize>; // last  byte not in the set
use stringzilla as sz;

assert_eq!(sz::find("Hello, world!", "world"), Some(7));
assert_eq!(sz::rfind("Hello, world, world!", "world"), Some(14));

// First/last byte from a set of delimiters, or the first/last byte NOT in a set.
assert_eq!(sz::find_byte_from("Hello, world!", "aeiou"), Some(1));      // 'e'
assert_eq!(sz::rfind_byte_from("Hello, world!", "aeiou"), Some(8));     // 'o'
assert_eq!(sz::find_byte_not_from("Hello, world!", "aeiou"), Some(0));  // 'H'
assert_eq!(sz::rfind_byte_not_from("Hello, world!", "aeiou"), Some(12));// '!'

The StringZillableBinary<'a, N> trait provides the same operations as methods on the haystack. The blanket impl covers every T: AsRef<[u8]> + ?Sized, so they work on &str, String, &[u8], Cow<str>, and more:

use stringzilla::StringZillableBinary;

let haystack = "Hello, world!";
assert_eq!(haystack.sz_find("world".as_bytes()), Some(7));
assert_eq!(haystack.sz_rfind("world".as_bytes()), Some(7));
assert_eq!(haystack.sz_find_byte_from("aeiou".as_bytes()), Some(1));
assert_eq!(haystack.sz_rfind_byte_from("aeiou".as_bytes()), Some(8));
assert_eq!(haystack.sz_find_byte_not_from("aeiou".as_bytes()), Some(0));
assert_eq!(haystack.sz_rfind_byte_not_from("aeiou".as_bytes()), Some(12));

Trait methods: sz_find, sz_rfind, sz_find_byte_from, sz_rfind_byte_from, sz_find_byte_not_from, sz_rfind_byte_not_from.

§ Counting UTF-8 Characters

These two functions reason about codepoint boundaries without fully decoding the string.

pub fn count_utf8<T: AsRef<[u8]>>(text: T) -> usize;
pub fn find_nth_utf8<T: AsRef<[u8]>>(text: T, n: usize) -> Option<usize>;

count_utf8 counts codepoint start bytes with SIMD; find_nth_utf8 returns the byte offset of the Nth, zero-indexed, codepoint without decoding the whole string:

use stringzilla as sz;

assert_eq!(sz::count_utf8("你好世界"), 4);
assert_eq!(sz::count_utf8("Hello🌍"), 6);
assert_eq!(sz::find_nth_utf8("Hello🌍", 5), Some(5)); // 🌍 starts at byte 5
assert_eq!(sz::find_nth_utf8("Hello", 5), None);

SubstringsEngine compiles a whole needle set into one Aho-Corasick automaton and walks every haystack against all of them at once, so a dictionary of thousands of terms costs one pass rather than thousands. Building it is the expensive half and the engine is reusable, so a long-lived one amortizes that across every later batch.

fn new<N: AsRef<[u8]>>(needles: &[N], case_sensitivity: CaseSensitivity,
    overlap_policy: SubstringsOverlapPolicy, hot_states: usize, matches_budget: usize,
    haystacks_budget: usize, stream: &Stream) -> Result<SubstringsEngine, Status>;

fn counts<'scope>(&'scope mut self, scope: &'scope Scope<'scope, '_>,
    haystacks: StringsView<'scope>, counts: &'scope mut [usize], counts_stride: usize) -> Result<(), Status>;
fn find<'scope>(&'scope mut self, scope: &'scope Scope<'scope, '_>,
    haystacks: StringsView<'scope>, matches: &'scope mut [Match],
    matches_offsets: &'scope mut [usize]) -> Result<(), Status>;
fn replace<'scope>(&'scope mut self, scope: &'scope Scope<'scope, '_>,
    haystacks: StringsView<'scope>, replacements: StringsView<'scope>,
    target: &'scope mut [u8], offsets: &'scope mut [usize]) -> Result<(), Status>;
fn bm25_scores<'scope>(&'scope mut self, scope: &'scope Scope<'scope, '_>,
    haystacks: StringsView<'scope>, document_lengths: Option<&'scope [f32]>, parameters: &'scope Bm25Params,
    needle_weights: &'scope [f32], scores: &'scope mut [f32], scores_stride: usize) -> Result<(), Status>;

fn report(&self) -> SubstringsReport;

hot_states sizes the automaton’s dense tier, or takes SUBSTRINGS_HOT_STATES_AUTO to fill a fixed byte budget instead, which holds more states the fewer byte classes the vocabulary spells. matches_budget bounds what one round may emit and haystacks_budget how many haystacks it may carry, and both are read by a device tier alone, so a host engine takes SUBSTRINGS_MATCHES_BUDGET_AUTO and zero and walks straight into the caller’s output.

CaseSensitivity::Cased matches bytes exactly and accepts arbitrary needles, while CaseSensitivity::Uncased folds both sides under full Unicode case folding and requires valid UTF-8. Folding is not a byte-length-preserving operation, so a 1-byte needle can match a 3-byte span — the Kelvin sign U+212A folds to k — which is why every Match carries its own byte_length rather than borrowing the needle’s.

SubstringsOverlapPolicy decides what a walk reports, and it sizes the engine’s arena, so it is fixed at construction and one engine runs exactly one of the three:

  • Overlapping — every match of every needle, nested and overlapping ones included.
  • LeftmostLongest — a cover taking the widest match at the earliest start.
  • LeftmostFirst — a cover taking the lowest needle index at the earliest start.

A capacity shortfall is not an error. The sizing walk always runs, so report() names matches_emitted, the true total; matches_stored, what was written; target_length, what a rewrite needs; and shortfall, what did not fit. That is what lets one call with an empty output size the next one, with no walk in between.

use stringzilla::{
    Capabilities, CaseSensitivity, Stream, StringsView, SubstringsEngine, Match, SubstringsOverlapPolicy,
    SUBSTRINGS_HOT_STATES_AUTO, SUBSTRINGS_MATCHES_BUDGET_AUTO,
};

let cpu = Stream::default(Capabilities::cpu_enabled());
let mut engine = SubstringsEngine::new(
    &["cat", "catalog"],
    CaseSensitivity::Cased,
    SubstringsOverlapPolicy::Overlapping,
    SUBSTRINGS_HOT_STATES_AUTO,
    SUBSTRINGS_MATCHES_BUDGET_AUTO,
    0,
    &cpu,
)
.unwrap();

let documents = StringsView::from_slices(&["a catalog of cats", "nothing here"]);
let mut counts = [0usize; 2];
cpu.scope(|scope| engine.counts(scope, documents, &mut counts, 1)).unwrap();
assert_eq!(counts, [3, 0]); // "catalog", the "cat" inside it, and the "cat" of "cats"

// `find` fills one boundary per haystack plus a final total, whether or not the matches fit.
let mut matches = [Match::default(); 3];
let mut offsets = [0usize; 3];
cpu.scope(|scope| engine.find(scope, documents, &mut matches, &mut offsets)).unwrap();
assert_eq!(offsets, [0, 3, 3]);
assert_eq!(engine.report().matches_emitted, 3);
assert_eq!(engine.report().shortfall, 0);

§ Scoring and Rewriting

The same automaton scores documents with BM25 and rewrites them, each in a single walk. bm25_scores treats the dictionary itself as the query — needle_weights[i] is needle i’s IDF or boost — and writes one score per haystack, so a many-term query over a large corpus never materializes per-term frequency rows. Term frequencies are raw overlapping counts, which is classic BM25, so the engine’s own policy does not apply here, and document_lengths falls back to each haystack’s byte length when None.

Bm25Params has no Default, because a corpus mean has no correct default value. Its two constructors name the two configurations that exist: Bm25Params::normalized(mean) is the literature’s k1 = 1.2 and b = 0.75 against a corpus whose mean document length you know, and Bm25Params::unnormalized() switches length normalization off and leaves document_lengths unread.

replace takes one replacement per needle, inserted verbatim, an empty one deleting its match, and writes every rewritten haystack into one target buffer beside their boundaries. Rewriting is defined only under a cover, so an engine built with SubstringsOverlapPolicy::Overlapping is refused there — an overlapping rewrite is not a function. A target too small is not an error either: report().target_length names the bytes the rewrite needed and the buffer’s contents are then unspecified, so an empty target is how the next call is sized.

use stringzilla::{Bm25Params, Capabilities, CaseSensitivity, Stream, StringsView, SubstringsEngine,
                      SubstringsOverlapPolicy, SUBSTRINGS_HOT_STATES_AUTO, SUBSTRINGS_MATCHES_BUDGET_AUTO};

let cpu = Stream::default(Capabilities::cpu_enabled());
let mut engine = SubstringsEngine::new(
    &["cat", "dog"],
    CaseSensitivity::Cased,
    SubstringsOverlapPolicy::LeftmostLongest,
    SUBSTRINGS_HOT_STATES_AUTO,
    SUBSTRINGS_MATCHES_BUDGET_AUTO,
    0,
    &cpu,
)
.unwrap();

let documents = StringsView::from_slices(&["cat and dog", "nothing here"]);
let mut scores = [0.0f32; 2];
let parameters = Bm25Params::normalized(10.0);
cpu.scope(|scope| engine.bm25_scores(scope, documents, None, &parameters, &[1.0, 1.0], &mut scores, 1))
    .unwrap();
assert_eq!(scores[1], 0.0);

// One replacement per needle: an empty target sizes the rewrite, a second scope performs it.
let replacements = StringsView::from_slices(&["feline", "canine"]);
let mut offsets = [0usize; 3];
cpu.scope(|scope| engine.replace(scope, documents, replacements, &mut [], &mut offsets)).unwrap();
let mut target = vec![0u8; engine.report().target_length];
cpu.scope(|scope| engine.replace(scope, documents, replacements, &mut target, &mut offsets)).unwrap();
assert_eq!(&target[offsets[0]..offsets[1]], b"feline and canine");

API

EnumMatcherWhat a match or split iterator looks for; the iterator type picks the direction.3
find.rs:309 ↗
pub enum Matcher<'a>

Values

Substring(&amp;'a [u8])An exact byte sequence.
Bytes(<a data-ref="Byteset">Byteset</a>)Any one byte of the set; invert the set to match the bytes outside it.

Implements

Debug, Clone, Copy
findOffset of the first match in haystack.
find.rs:318 ↗
pub fn find(&self, haystack: &[u8]) -> Option<usize>
rfindOffset of the last match in haystack.
find.rs:326 ↗
pub fn rfind(&self, haystack: &[u8]) -> Option<usize>
widthBytes every match spans: the needle's length, or one for a byteset.
find.rs:334 ↗
pub fn width(&self) -> usize
StructFindMatchesAn iterator over the matches of a pattern in a byte slice, front to back.2
find.rs:357 ↗
pub struct FindMatches<'a, Overlap: Overlaps = NonOverlapping>

Empty needle

An empty needle matches at every offset from 0 to haystack.len(), so an n-byte haystack yields n + 1 empty matches, mirroring "abc".matches("").count() == 4.

Examples

use stringzilla::{FindMatches, Matcher};

let matches: Vec<&[u8]> = FindMatches::new(b"abababa", Matcher::Substring(b"aba")).collect();
assert_eq!(matches, vec![b"aba", b"aba"]);

Implements

Iterator
newIterates the non-overlapping matches of matcher in haystack, front to back.
find.rs:365 ↗
pub fn new(haystack: &'a [u8], matcher: Matcher<'a>) -> Self
*× 2Report overlapping matches too, a compile-time policy returning the Overlapping variant.
find.rs:374 ↗
pub fn overlapping(self) -> FindMatches<'a, Overlapping>
StructRFindMatchesAn iterator over the matches of a pattern in a byte slice, back to front.1
find.rs:415 ↗
pub struct RFindMatches<'a, Overlap: Overlaps = NonOverlapping>

Empty needle

An empty needle matches at every offset from haystack.len() down to 0, so an n-byte haystack yields n + 1 empty matches.

Examples

use stringzilla::{Matcher, RFindMatches};

let matches: Vec<&[u8]> = RFindMatches::new(b"abababa", Matcher::Substring(b"aba")).collect();
assert_eq!(matches, vec![b"aba", b"aba"]);

Implements

Iterator
newIterates the non-overlapping matches of matcher in haystack, back to front.
find.rs:423 ↗
pub fn new(haystack: &'a [u8], matcher: Matcher<'a>) -> Self
TraitStringZillableBinaryTrait for binary string operations that take a needle parameter.14
find.rs:606 ↗
pub trait StringZillableBinary<'a, Needle>
where
    Needle: AsRef<[u8]> + 'a,

These operations include searching, splitting, and pattern matching.

Examples

Basic usage on a string slice:

use stringzilla::StringZillableBinary;

let haystack = "Hello, world!";
assert_eq!(haystack.sz_find("world".as_bytes()), Some(7));

Implementors

  • impl<'a, Source, Needle> StringZillableBinary<'a, Needle> for Source where Source: AsRef<[u8]> + ?Sized, Needle: AsRef<[u8]> + 'a
sz_findSearches for the first occurrence of needle in self.
find.rs:620 ↗
fn sz_find(&self, needle: Needle) -> Option<usize>

Examples

use stringzilla::StringZillableBinary;

let haystack = "Hello, world!";
assert_eq!(haystack.sz_find("world".as_bytes()), Some(7));
sz_rfindSearches for the last occurrence of needle in self.
find.rs:632 ↗
fn sz_rfind(&self, needle: Needle) -> Option<usize>

Examples

use stringzilla::StringZillableBinary;

let haystack = "Hello, world, world!";
assert_eq!(haystack.sz_rfind("world".as_bytes()), Some(14));
sz_find_byte_fromFinds the index of the first character in self that is also present in needles.
find.rs:644 ↗
fn sz_find_byte_from(&self, needles: Needle) -> Option<usize>

Examples

use stringzilla::StringZillableBinary;

let haystack = "Hello, world!";
assert_eq!(haystack.sz_find_byte_from("aeiou".as_bytes()), Some(1));
sz_rfind_byte_fromFinds the index of the last character in self that is also present in needles.
find.rs:656 ↗
fn sz_rfind_byte_from(&self, needles: Needle) -> Option<usize>

Examples

use stringzilla::StringZillableBinary;

let haystack = "Hello, world!";
assert_eq!(haystack.sz_rfind_byte_from("aeiou".as_bytes()), Some(8));
sz_find_byte_not_fromFinds the index of the first character in self that is not present in needles.
find.rs:668 ↗
fn sz_find_byte_not_from(&self, needles: Needle) -> Option<usize>

Examples

use stringzilla::StringZillableBinary;

let haystack = "Hello, world!";
assert_eq!(haystack.sz_find_byte_not_from("aeiou".as_bytes()), Some(0));
sz_rfind_byte_not_fromFinds the index of the last character in self that is not present in needles.
find.rs:680 ↗
fn sz_rfind_byte_not_from(&self, needles: Needle) -> Option<usize>

Examples

use stringzilla::StringZillableBinary;

let haystack = "Hello, world!";
assert_eq!(haystack.sz_rfind_byte_not_from("aeiou".as_bytes()), Some(12));
sz_matchesReturns an iterator over all non-overlapping matches of the given needle in self.
find.rs:700 ↗
fn sz_matches(&'a self, needle: &'a Needle) -> FindMatches<'a>

Arguments

  • needle: The byte slice to search for within self.

Examples

use stringzilla::StringZillableBinary;

let haystack = b"abababa";
let needle = b"aba";
let matches: Vec<&[u8]> = haystack.sz_matches(needle).collect();
assert_eq!(matches, vec![b"aba", b"aba"]); // non-overlapping by default (like str::matches)
let overlapping: Vec<&[u8]> = haystack.sz_matches(needle).overlapping().collect();
assert_eq!(overlapping, vec![b"aba", b"aba", b"aba"]); // opt in with .overlapping()
sz_rmatchesReturns an iterator over all non-overlapping matches of the given needle in self, searching from the end.
find.rs:721 ↗
fn sz_rmatches(&'a self, needle: &'a Needle) -> RFindMatches<'a>

Arguments

  • needle: The byte slice to search for within self.

Examples

use stringzilla::StringZillableBinary;

let haystack = b"abababa";
let needle = b"aba";
let matches: Vec<&[u8]> = haystack.sz_rmatches(needle).collect();
assert_eq!(matches, vec![b"aba", b"aba"]); // non-overlapping by default
let overlapping: Vec<&[u8]> = haystack.sz_rmatches(needle).overlapping().collect();
assert_eq!(overlapping, vec![b"aba", b"aba", b"aba"]); // opt in with .overlapping()
sz_splitsReturns an iterator over the substrings of self that are separated by the given needle.
find.rs:739 ↗
fn sz_splits(&'a self, needle: &'a Needle) -> FindSplits<'a>

Arguments

  • needle: The byte slice to split self by.

Examples

use stringzilla::StringZillableBinary;

let haystack = b"a,b,c,d";
let needle = b",";
let splits: Vec<&[u8]> = haystack.sz_splits(needle).collect();
assert_eq!(splits, vec![b"a", b"b", b"c", b"d"]);
sz_rsplitsReturns an iterator over the substrings of self that are separated by the given needle, searching from the end.
find.rs:758 ↗
fn sz_rsplits(&'a self, needle: &'a Needle) -> RFindSplits<'a>

Arguments

  • needle: The byte slice to split self by.

Examples

use stringzilla::StringZillableBinary;

let haystack = b"a,b,c,d";
let needle = b",";
let splits: Vec<&[u8]> = haystack.sz_rsplits(needle).collect();
assert_eq!(splits, vec![b"d", b"c", b"b", b"a"]);
sz_find_first_ofReturns an iterator over all non-overlapping matches of any of the bytes in needles within self.
find.rs:777 ↗
fn sz_find_first_of(&'a self, needles: &'a Needle) -> FindMatches<'a>

Arguments

  • needles: The set of bytes to search for within self.

Examples

use stringzilla::StringZillableBinary;

let haystack = b"Hello, world!";
let needles = b"aeiou";
let matches: Vec<&[u8]> = haystack.sz_find_first_of(needles).collect();
assert_eq!(matches, vec![b"e", b"o", b"o"]);
sz_find_last_ofReturns an iterator over all non-overlapping matches of any of the bytes in needles within self, searching from the end.
find.rs:796 ↗
fn sz_find_last_of(&'a self, needles: &'a Needle) -> RFindMatches<'a>

Arguments

  • needles: The set of bytes to search for within self.

Examples

use stringzilla::StringZillableBinary;

let haystack = b"Hello, world!";
let needles = b"aeiou";
let matches: Vec<&[u8]> = haystack.sz_find_last_of(needles).collect();
assert_eq!(matches, vec![b"o", b"o", b"e"]);
sz_find_first_not_ofReturns an iterator over all non-overlapping matches of any byte not in needles within self.
find.rs:815 ↗
fn sz_find_first_not_of(&'a self, needles: &'a Needle) -> FindMatches<'a>

Arguments

  • needles: The set of bytes that should not be matched within self.

Examples

use stringzilla::StringZillableBinary;

let haystack = b"Hello, world!";
let needles = b"aeiou";
let matches: Vec<&[u8]> = haystack.sz_find_first_not_of(needles).collect();
assert_eq!(matches, vec![b"H", b"l", b"l", b",", b" ", b"w", b"r", b"l", b"d", b"!"]);
sz_find_last_not_ofReturns an iterator over all non-overlapping matches of any byte not in needles within self, searching from the end.
find.rs:834 ↗
fn sz_find_last_not_of(&'a self, needles: &'a Needle) -> RFindMatches<'a>

Arguments

  • needles: The set of bytes that should not be matched within self.

Examples

use stringzilla::StringZillableBinary;

let haystack = b"Hello, world!";
let needles = b"aeiou";
let matches: Vec<&[u8]> = haystack.sz_find_last_not_of(needles).collect();
assert_eq!(matches, vec![b"!", b"d", b"l", b"r", b"w", b" ", b",", b"l", b"l", b"H"]);
StructRegexEngineA vocabulary of RE2-syntax patterns compiled into two automata, walked over as many batches of haystacks as a caller has.6
regex.rs:85 ↗
pub struct RegexEngine

Matches are leftmost-first across the union of patterns, ties going to the lower pattern index, which is what Rust's regex, RE2 and Python's re report; a Match names its pattern as needle_index. Every haystack streams through the automata once whatever the pattern count.

The fields mirror sz_regex_engine_t one for one and only patterns_count is read from Rust, so the layout is load-bearing and the engine travels to C by pointer. Every verb grows the engine's round scratch, so it takes &mut self.

Examples

use stringzilla::{Capabilities, CaseSensitivity, Match, RegexEngine, Stream, StringsView};

let cpu = Stream::default(Capabilities::cpu_enabled());
let mut engine = RegexEngine::new(&["[0-9]+", "[a-z]+"], CaseSensitivity::Cased, &cpu).unwrap();

let documents = StringsView::from_slices(&["abc 123 de", "!"]);
let (mut matches, mut offsets) = ([Match::default(); 4], [0usize; 3]);
cpu.scope(|scope| engine.find(scope, documents, &mut matches, &mut offsets)).unwrap();
assert_eq!(offsets, [0, 3, 3]);
assert_eq!((matches[1].needle_index, matches[1].byte_offset, matches[1].byte_length), (0, 4, 3));

Implements

Drop
newCompiles patterns with the capabilities of stream, like Capabilities::cpu_enabled, on the device stream names.
regex.rs:140 ↗
pub fn new<Pattern>(
    patterns: &[Pattern],
    case_sensitivity: CaseSensitivity,
    stream: &Stream,
) -> Result<Self, StringsError>
where
    Pattern: AsRef<[u8]>,

A pattern outside the supported syntax, like a backreference, a capturing group inside a lookaround or an inline (?i), is refused as StringsError::Malformed with its index; case folding is the engine-wide case_sensitivity instead.

patterns_countPatterns the vocabulary holds, which bounds every reported needle_index.
regex.rs:169 ↗
pub fn patterns_count(&self) -> usize
countsQueues on the stream of scope the count of every haystack's leftmost-first matches, one count per haystack.
regex.rs:179 ↗
pub fn counts<'scope>(
    &'scope mut self,
    scope: &'scope Scope<'scope, '_>,
    haystacks: StringsView<'scope>,
    counts: &'scope mut [usize],
    counts_stride: usize,
) -> Result<(), Status>

counts receives haystack h at counts[h * counts_stride], the stride counting entries rather than bytes and being at least one. A stream of another group than the engine's is refused with Status::DeviceMemoryMismatch, here and by every verb below.

findQueues on the stream of scope a report of every leftmost-first match, ascending by haystack, then by start.
regex.rs:197 ↗
pub fn find<'scope>(
    &'scope mut self,
    scope: &'scope Scope<'scope, '_>,
    haystacks: StringsView<'scope>,
    matches: &'scope mut [Match],
    matches_offsets: &'scope mut [usize],
) -> Result<(), Status>

matches_offsets holds one boundary per haystack plus a final total, filled whether or not the matches fit, which sizes the next call; an empty matches makes the call a size query.

replaceRewrites every haystack into one target buffer, substituting each match with its pattern's replacement, on the CPU only.
regex.rs:221 ↗
pub fn replace<'scope>(
    &'scope mut self,
    scope: &'scope Scope<'scope, '_>,
    haystacks: StringsView<'scope>,
    replacements: StringsView<'scope>,
    target: &'scope mut [u8],
    offsets: &'scope mut [usize],
) -> Result<(), Status>

replacements is indexed by pattern, so it holds exactly RegexEngine::patterns_count entries; a replacement names groups as $1, ${1} or ${name}, $0 being the whole match and $ one dollar. offsets holds one boundary per haystack plus a final total and is filled whether or not target held the result, which sizes the next call.

extractQueues on the stream of scope the groups of each haystack's first match of pattern_index, as Pandas str.extract reads them.
regex.rs:270 ↗
pub fn extract<'scope>(
    &'scope mut self,
    scope: &'scope Scope<'scope, '_>,
    haystacks: StringsView<'scope>,
    pattern_index: usize,
    groups: &'scope mut [Match],
    groups_offsets: &'scope mut [usize],
) -> Result<(), Status>

A haystack with such a match gets a Match row for the whole match, needle_index zero, then one per group that took part, needle_index naming the group, so a group left out differs from an empty one; a haystack without one gets no rows. groups_offsets holds one boundary per haystack plus a final total, filled whether or not the rows fit, which sizes the next call; an empty groups makes the call a size query.

use stringzilla::{Capabilities, CaseSensitivity, Match, RegexEngine, Stream, StringsView};

let cpu = Stream::default(Capabilities::cpu_enabled());
let mut engine = RegexEngine::new(&["(\\d+)-(\\d+)"], CaseSensitivity::Cased, &cpu).unwrap();

let documents = StringsView::from_slices(&["a 12-34", "none"]);
let (mut groups, mut offsets) = ([Match::default(); 3], [0usize; 3]);
cpu.scope(|scope| engine.extract(scope, documents, 0, &mut groups, &mut offsets)).unwrap();
assert_eq!(offsets, [0, 3, 3]);
assert_eq!((groups[2].needle_index, groups[2].byte_offset, groups[2].byte_length), (2, 5, 2));
EnumSubdocumentsFormatThe grammar of every document an engine reads, and the path language that addresses it.
subdocuments.rs:53 ↗
pub enum SubdocumentsFormat

Corresponds to sz_subdocuments_format_t in the C API.

Values

Json0RFC 8259 documents, addressed by RFC 9535 JSONPath.
Xml1XML 1.0 documents, addressed by XPath 1.0 location paths along forward axes.

Implements

Debug, Clone, Copy, StructuralPartialEq, PartialEq, Eq
StructSubdocumentsEngineA vocabulary of paths compiled into one automaton over keys, walked over as many batches of documents as a caller has.5
subdocuments.rs:94 ↗
pub struct SubdocumentsEngine

Every document is walked once whatever the path count, and no tree is built: a container no path can enter is skipped by counting its brackets. A Match spans the selected node's exact bytes inside its document, in JSON a string with its quotes, a number or literal, or a container through its closing bracket, and in XML an element, an attribute's value or a text node, which a caller reads in place or hands to another parser.

The fields mirror sz_subdocuments_engine_t one for one and only paths_count and format are read from Rust, so the layout is load-bearing and the engine travels to C by pointer. The regex engine of match() and search() filters is embedded by value and freed by the C side, hence ManuallyDrop. Every verb grows the engine's depth stack, so it takes &mut self.

Examples

use stringzilla::{Capabilities, Match, Stream, StringsView, SubdocumentsEngine, SubdocumentsFormat};

let cpu = Stream::default(Capabilities::cpu_enabled());
let paths = ["$.store.book[?@.price < 10].title"];
let mut engine = SubdocumentsEngine::new(&paths, SubdocumentsFormat::Json, &cpu).unwrap();

let document = r#"{"store": {"book": [{"title": "A", "price": 8}, {"title": "B", "price": 12}]}}"#;
let documents = [document];
let documents = StringsView::from_slices(&documents);
let (mut matches, mut offsets, mut malformed_index) = ([Match::default(); 4], [0usize; 2], 0);
cpu.scope(|scope| engine.find(scope, documents, &mut matches, &mut offsets, &mut malformed_index)).unwrap();
assert_eq!(offsets[1], 1);
let title = &matches[0];
assert_eq!(&document[title.byte_offset..title.byte_offset + title.byte_length], "\"A\"");

Implements

Drop
newCompiles paths for format with the capabilities of stream, like Capabilities::cpu_enabled, on the device stream names.
subdocuments.rs:140 ↗
pub fn new<Path>(
    paths: &[Path],
    format: SubdocumentsFormat,
    stream: &Stream,
) -> Result<Self, StringsError>
where
    Path: AsRef<[u8]>,

JSON paths follow RFC 9535, filters and their functions included, and XML paths follow XPath 1.0 along forward axes. A path that breaks its grammar, or that no single walk can evaluate, like an absolute query inside a filter, is refused as StringsError::Malformed with its index.

paths_countPaths the vocabulary holds, which bounds every reported needle_index.
subdocuments.rs:166 ↗
pub fn paths_count(&self) -> usize
formatThe grammar every document of every verb follows.
subdocuments.rs:171 ↗
pub fn format(&self) -> SubdocumentsFormat
countsQueues on the stream of scope the count of every document's nodes that some path selects, once per path, one count per document.
subdocuments.rs:181 ↗
pub fn counts<'scope>(
    &'scope mut self,
    scope: &'scope Scope<'scope, '_>,
    documents: StringsView<'scope>,
    counts: &'scope mut [usize],
    counts_stride: usize,
    malformed_index: &'scope mut usize,
) -> Result<(), Status>

counts receives document d at counts[d * counts_stride], the stride counting entries rather than bytes and being at least one. A malformed document fails the call, and malformed_index then names the first one once the scope joins.

findQueues on the stream of scope a report of every node of every document that some path selects, by document, then by start offset, then by path, so a container precedes what it holds.
subdocuments.rs:213 ↗
pub fn find<'scope>(
    &'scope mut self,
    scope: &'scope Scope<'scope, '_>,
    documents: StringsView<'scope>,
    matches: &'scope mut [Match],
    matches_offsets: &'scope mut [usize],
    malformed_index: &'scope mut usize,
) -> Result<(), Status>

matches_offsets holds one boundary per document plus a final total, and is filled whether or not the matches fit, which is what sizes the next call; an empty matches makes the call a pure size query. A malformed document fails the call, and malformed_index then names the first one once the scope joins.

EnumSubstringsOverlapPolicyHow matches that share bytes resolve: reported in full, or thinned to a leftmost run.
substrings.rs:39 ↗
pub enum SubstringsOverlapPolicy

The policy sizes the engine's arena, so it is fixed at construction rather than travelling per call, and one engine runs exactly one of the three.

Corresponds to sz_substrings_overlap_policy_t in the C API.

Values

Overlapping0Every match of every needle, including ones that share bytes and ones nested in others.
LeftmostLongest1Matches sharing no bytes: earliest start, then longest span, then lower needle index.
LeftmostFirst2Matches sharing no bytes: earliest start, then lower needle index, however long the rival.

Implements

Debug, Clone, Copy, StructuralPartialEq, PartialEq, Eq
StructMatchOne reported match, locating it by haystack, by needle, and by byte span.
substrings.rs:56 ↗
pub struct Match

Under case folding a needle's own byte length is not the length of every match - needle k matches both the 1-byte k and the 3-byte Kelvin sign U+212A - so the span travels per match.

Corresponds to sz_match_t in the C API.

Fields

haystack_indexusizeWhich haystack of the sequence this match was found in.
needle_indexusizeWhich needle of the vocabulary matched.
byte_offsetusizeWhere the match starts inside that haystack, in its own bytes.
byte_lengthusizeHaystack bytes the match spans, which folding can make differ from the needle's own length.

Implements

Debug, Clone, Copy, Default, StructuralPartialEq, PartialEq, Eq
StructSubstringsReportWhat a round found, which is how a capacity shortfall is reported rather than as an error.
substrings.rs:76 ↗
pub struct SubstringsReport

The sizing walk always runs, so matches_emitted is the truth whatever the caller's output could hold, and a nonzero shortfall is the one signal that an output is incomplete rather than wrong.

Corresponds to sz_substrings_report_t in the C API.

Fields

matches_emittedusizeMatches the sizing walk found, which is the truth whatever the output held.
matches_storedusizeMatches written out, which is matches_emitted clipped at the capacity.
target_lengthusizeBytes a rewrite needs, which is the truth whatever the target held.
shortfallusizeMatches or bytes the round could not hold, zero when everything fit.

Implements

Debug, Clone, Copy, Default, StructuralPartialEq, PartialEq, Eq
StructBm25ParamsBM25's continuous parameters.2
substrings.rs:97 ↗
pub struct Bm25Params

There is no correct default for the corpus mean, so there is no Default: reach for Bm25Params::normalized or Bm25Params::unnormalized, which name the two configurations that exist. A positive length_normalization beside a non-positive average_document_length is refused, since it would divide by a mean that is not there.

Corresponds to sz_substrings_bm25_t in the C API.

Fields

term_frequency_saturationf32The literature's k1: how slowly repeated occurrences stop adding score; 1.2 is customary.
length_normalizationf32The literature's b, in [0, 1]: 0 ignores document length, 1 normalizes it fully.
average_document_lengthf32The corpus-wide mean document length, in the unit of the lengths scored; read only when length_normalization is positive.

Implements

Debug, Clone, Copy, StructuralPartialEq, PartialEq
normalizedClassic BM25 with the literature's customary k1 = 1.2 and b = 0.75, normalized against a corpus whose mean document length is average_document_length, in the unit the per-document lengths use.
substrings.rs:111 ↗
pub const fn normalized(average_document_length: f32) -> Self
unnormalizedBM25 with length normalization switched off, for a corpus whose mean length is unknown or whose documents are uniform enough not to need it; the per-document lengths then go unread.
substrings.rs:121 ↗
pub const fn unnormalized() -> Self
StructSubstringsEngineA vocabulary compiled into one automaton, walked over as many batches of haystacks as a caller has.8
substrings.rs:170 ↗
pub struct SubstringsEngine

Building the automaton is the expensive half and every verb below reuses it, so one long-lived engine amortizes that across every later batch: a dictionary of thousands of terms costs one pass over a haystack rather than thousands.

The fields mirror sz_substrings_engine_t one for one and only needles_count, overlap_policy and report are read from Rust, so the layout is load-bearing and the engine travels to C by pointer. Owning raw pointers makes it neither Send nor Sync, which is what the C contract wants: a compute verb writes the engine's round arena, so it mutates, and every verb below takes &mut self for that reason.

Examples

use stringzilla::{
    Capabilities, CaseSensitivity, Stream, StringsView, SubstringsEngine, SubstringsOverlapPolicy,
    SUBSTRINGS_HOT_STATES_AUTO, SUBSTRINGS_MATCHES_BUDGET_AUTO,
};

let cpu = Stream::default(Capabilities::cpu_enabled());
let mut engine = SubstringsEngine::new(
    &["cat", "catalog"],
    CaseSensitivity::Cased,
    SubstringsOverlapPolicy::Overlapping,
    SUBSTRINGS_HOT_STATES_AUTO,
    SUBSTRINGS_MATCHES_BUDGET_AUTO,
    0,
    &cpu,
)
.unwrap();

let documents = StringsView::from_slices(&["a catalog of cats", "nothing here"]);
let mut counts = [0usize; 2];
cpu.scope(|scope| engine.counts(scope, documents, &mut counts, 1)).unwrap();
assert_eq!(counts, [3, 0]); // "catalog", the "cat" inside it, and the "cat" of "cats"

Implements

Drop
newCompiles needles into an automaton the matching verbs read, with the capabilities of stream, like Capabilities::cpu_enabled or a GPU's Capabilities::cuda_enabled, on the device stream names, which it may join.
substrings.rs:223 ↗
pub fn new<Needle>(
    needles: &[Needle],
    case_sensitivity: CaseSensitivity,
    overlap_policy: SubstringsOverlapPolicy,
    hot_states: usize,
    matches_budget: usize,
    haystacks_budget: usize,
    stream: &Stream,
) -> Result<Self, Status>
where
    Needle: AsRef<[u8]>,

An empty needle is refused rather than skipped, since it would match at every position and dropping it would shift every later needle's reported index. hot_states is the count kept in the dense hot rows, or SUBSTRINGS_HOT_STATES_AUTO to fill a fixed byte budget instead.

The budgets are read by a device kernel alone, so a host engine takes SUBSTRINGS_MATCHES_BUDGET_AUTO and zero and walks straight into the caller's output. matches_budget is what a device round may emit before every later kernel retires, and haystacks_budget how many haystacks it may carry; the round's arena is sized for both here, zero asking for a default, so no verb allocates, a round carrying more haystacks is refused, and matches past the budget surface as the report's shortfall. A GPU engine's verbs take haystacks, replacements and every buffer, weights and lengths included, in memory the device reaches, such as a Strings and a Vec in a UnifiedAllocator.

needles_countNeedles the vocabulary holds, which bounds every reported needle_index.
substrings.rs:255 ↗
pub fn needles_count(&self) -> usize
overlap_policyThe cover every round runs, fixed when the arena was sized.
substrings.rs:260 ↗
pub fn overlap_policy(&self) -> SubstringsOverlapPolicy
reportWhat the last round found, which is how a capacity shortfall surfaces instead of as an error.
substrings.rs:268 ↗
pub fn report(&self) -> SubstringsReport

A verb borrows the engine until its scope joins, so the record a device wrote is complete.

countsQueues on the stream of scope the count of every needle's matches in every haystack, one count per haystack.
substrings.rs:279 ↗
pub fn counts<'scope>(
    &'scope mut self,
    scope: &'scope Scope<'scope, '_>,
    haystacks: StringsView<'scope>,
    counts: &'scope mut [usize],
    counts_stride: usize,
) -> Result<(), Status>

counts receives haystack h at counts[h * counts_stride], the stride counting entries rather than bytes and being at least one, so a strided call writes one column of a [haystacks, vocabularies] feature matrix. A stream of another group than the engine's is refused with Status::DeviceMemoryMismatch, here and by every verb below.

findQueues on the stream of scope a report of every match of every needle in every haystack, ascending by haystack.
substrings.rs:299 ↗
pub fn find<'scope>(
    &'scope mut self,
    scope: &'scope Scope<'scope, '_>,
    haystacks: StringsView<'scope>,
    matches: &'scope mut [Match],
    matches_offsets: &'scope mut [usize],
) -> Result<(), Status>

matches_offsets holds one boundary per haystack plus a final total, and is filled whether or not the matches fit, which is what sizes the next call. A capacity too small is not an error: SubstringsEngine::report names the true total and what did not fit, so a sizing call with an empty matches followed by one filling call needs no walk in between.

replaceQueues on the stream of scope a rewrite of every haystack into one target buffer, substituting one replacement per needle.
substrings.rs:327 ↗
pub fn replace<'scope>(
    &'scope mut self,
    scope: &'scope Scope<'scope, '_>,
    haystacks: StringsView<'scope>,
    replacements: StringsView<'scope>,
    target: &'scope mut [u8],
    offsets: &'scope mut [usize],
) -> Result<(), Status>

replacements is indexed by needle and an empty one deletes the match, so it holds exactly SubstringsEngine::needles_count entries. offsets holds one boundary per haystack plus a final total and is filled whether or not target held the result, which is what sizes the next call; a target too small leaves its contents unspecified rather than failing, and SubstringsEngine::report names the bytes the rewrite needed.

The engine's policy must be a cover, since a substitution over matches that share bytes is not a function.

bm25_scoresQueues on the stream of scope the score of every haystack against the whole vocabulary as one BM25 query, one score per haystack.
substrings.rs:367 ↗
pub fn bm25_scores<'scope>(
    &'scope mut self,
    scope: &'scope Scope<'scope, '_>,
    haystacks: StringsView<'scope>,
    document_lengths: Option<&'scope [f32]>,
    parameters: &'scope Bm25Params,
    needle_weights: &'scope [f32],
    scores: &'scope mut [f32],
    scores_stride: usize,
) -> Result<(), Status>

The vocabulary is the query and needle_weights holds each needle's IDF or boost, so a many-term query over a large corpus never materializes per-term frequency rows. Term frequencies are raw overlapping counts, since a cover would suppress genuine occurrences of a needle nested in another, so the engine's own policy does not apply here. document_lengths is None to normalize by each haystack's byte length instead.

scores receives haystack h at scores[h * scores_stride], on the same convention as SubstringsEngine::counts.

StructIndexSpanRepresents a byte span with offset and length.4
types.rs:206 ↗
pub struct IndexSpan

Used for matches of UTF-8 characters, substrings, or any byte-level operations. Stores the byte offset from the start of the text and the length in bytes.

Fields

offsetusizeByte offset from the start of the text
lengthusizeLength in bytes of the matched span

Examples

use stringzilla::IndexSpan;

let text = "Hello\nWorld";
let span = IndexSpan::new(5, 1);
assert_eq!(span.offset, 5);
assert_eq!(span.length, 1);
let matched = span.extract(text.as_bytes());
assert_eq!(matched, Some(&b"\n"[..]));

Implements

Debug, Clone, Copy, StructuralPartialEq, PartialEq, Eq, Hash
newCreates a new IndexSpan with the given offset and length.
types.rs:216 ↗
pub const fn new(offset: usize, length: usize) -> Self
rangeReturns the range of bytes covered by this span, ending at Self::end.
types.rs:231 ↗
pub const fn range(&self) -> Range<usize>

Examples

use stringzilla::IndexSpan;

let span = IndexSpan::new(5, 3);
assert_eq!(span.range(), 5..8);
extractExtracts the matched bytes from the source text, or None if the span runs past its end.
types.rs:247 ↗
pub fn extract<'a>(&self, text: &'a [u8]) -> Option<&'a [u8]>

Examples

use stringzilla::IndexSpan;

let text = b"Hello World";
assert_eq!(IndexSpan::new(6, 5).extract(text), Some(&b"World"[..]));
assert_eq!(IndexSpan::new(6, 6).extract(text), None);
endReturns the end offset, offset + length, saturating at usize::MAX.
types.rs:262 ↗
pub const fn end(&self) -> usize

Examples

use stringzilla::IndexSpan;

let span = IndexSpan::new(5, 3);
assert_eq!(span.end(), 8);
TraitOverlapsCompile-time policy for whether overlapping matches are reported - a named marker, not a raw bool.1
types.rs:964 ↗
pub trait Overlaps

Implementors

OVERLAPWhether overlapping matches are included.
types.rs:966 ↗
const OVERLAP: bool;
StructNonOverlappingReport only non-overlapping matches, the default, like str::matches.
types.rs:969 ↗
pub struct NonOverlapping;

Implements

StructOverlappingReport overlapping matches too, via .overlapping().
types.rs:974 ↗
pub struct Overlapping;

Implements

Functions

findLocates the first matching substring within haystack that equals needle.
find.rs:33 ↗
pub fn find<Haystack, Needle>(haystack: Haystack, needle: Needle) -> Option<usize>
where
    Haystack: AsRef<[u8]>,
    Needle: AsRef<[u8]>,

This function is similar to the memmem() function in LibC, but, unlike strstr(), it requires the length of both haystack and needle to be known beforehand.

Returns

An Option<usize> representing the starting index of the first occurrence of needle within haystack if found, otherwise None.

Empty needle

The C core returns the start of haystack for an empty needle, like strstr, so find(haystack, b"") is always Some(0), matching "abc".find("") == Some(0). This holds even for an empty haystack.

Arguments

  • haystack: The byte slice to search.
  • needle: The byte slice to find within the haystack.
rfindLocates the last matching substring within haystack that equals needle.
find.rs:85 ↗
pub fn rfind<Haystack, Needle>(haystack: Haystack, needle: Needle) -> Option<usize>
where
    Haystack: AsRef<[u8]>,
    Needle: AsRef<[u8]>,

This function is useful for finding the most recent or last occurrence of a pattern within a byte slice.

Returns

An Option<usize> representing the starting index of the last occurrence of needle within haystack if found, otherwise None.

Empty needle

The C core returns the end of haystack for an empty needle, the reverse mirror of strstr, so rfind(haystack, b"") is always Some(haystack.len()), matching "abc".rfind("") == Some(3). This holds even for an empty haystack.

Arguments

  • haystack: The byte slice to search.
  • needle: The byte slice to find within the haystack.
containsChecks whether needle occurs anywhere within haystack.
find.rs:133 ↗
pub fn contains<Haystack, Needle>(haystack: Haystack, needle: Needle) -> bool
where
    Haystack: AsRef<[u8]>,
    Needle: AsRef<[u8]>,

Returns

true if needle occurs within haystack, false otherwise.

Empty needle

Mirrors str::contains: an empty needle is always present, so contains(haystack, b"") is always true, matching "abc".contains("") == true, even for an empty haystack.

Arguments

  • haystack: The byte slice to search.
  • needle: The byte slice to look for within the haystack.
find_bytesetFinds the index of the first character in haystack that is also present in needles.
find.rs:155 ↗
pub fn find_byteset<Haystack>(haystack: Haystack, needles: Byteset) -> Option<usize>
where
    Haystack: AsRef<[u8]>,

This function is particularly useful for parsing and tokenization tasks where a set of delimiter characters is used.

Returns

An Option<usize> representing the index of the first occurrence of any byte from needles within haystack, if found, otherwise None.

Arguments

  • haystack: The byte slice to search.
  • needles: The set of bytes to search for within the haystack.
rfind_bytesetFinds the index of the last character in haystack that is also present in needles.
find.rs:195 ↗
pub fn rfind_byteset<Haystack>(haystack: Haystack, needles: Byteset) -> Option<usize>
where
    Haystack: AsRef<[u8]>,

This can be used to find the last occurrence of any character from a specified set, useful in parsing scenarios such as finding the last delimiter in a string.

Returns

An Option<usize> representing the index of the last occurrence of any byte from needles within haystack, if found, otherwise None.

Arguments

  • haystack: The byte slice to search.
  • needles: The set of bytes to search for within the haystack.
find_byte_fromFinds the index of the first character in haystack that is also present in needles.
find.rs:236 ↗
pub fn find_byte_from<Haystack, Needle>(haystack: Haystack, needles: Needle) -> Option<usize>
where
    Haystack: AsRef<[u8]>,
    Needle: AsRef<[u8]>,

This function is particularly useful for parsing and tokenization tasks where a set of delimiter characters is used.

Returns

An Option<usize> representing the index of the first occurrence of any byte from needles within haystack, if found, otherwise None.

Arguments

  • haystack: The byte slice to search.
  • needles: The set of bytes to search for within the haystack.
rfind_byte_fromFinds the index of the last character in haystack that is also present in needles.
find.rs:257 ↗
pub fn rfind_byte_from<Haystack, Needle>(haystack: Haystack, needles: Needle) -> Option<usize>
where
    Haystack: AsRef<[u8]>,
    Needle: AsRef<[u8]>,

This can be used to find the last occurrence of any character from a specified set, useful in parsing scenarios such as finding the last delimiter in a string.

Returns

An Option<usize> representing the index of the last occurrence of any byte from needles within haystack, if found, otherwise None.

Arguments

  • haystack: The byte slice to search.
  • needles: The set of bytes to search for within the haystack.
find_byte_not_fromFinds the index of the first character in haystack that is not present in needles.
find.rs:278 ↗
pub fn find_byte_not_from<Haystack, Needle>(haystack: Haystack, needles: Needle) -> Option<usize>
where
    Haystack: AsRef<[u8]>,
    Needle: AsRef<[u8]>,

This function is useful for skipping over a known set of characters and finding the first character that does not belong to that set.

Returns

An Option<usize> representing the index of the first occurrence of any byte not in needles within haystack, if found, otherwise None.

Arguments

  • haystack: The byte slice to search.
  • needles: The set of bytes that should not be matched within the haystack.
rfind_byte_not_fromFinds the index of the last character in haystack that is not present in needles.
find.rs:299 ↗
pub fn rfind_byte_not_from<Haystack, Needle>(haystack: Haystack, needles: Needle) -> Option<usize>
where
    Haystack: AsRef<[u8]>,
    Needle: AsRef<[u8]>,

Useful for text processing tasks such as trimming trailing characters that belong to a specified set.

Returns

An Option<usize> representing the index of the last occurrence of any byte not in needles within haystack, if found, otherwise None.

Arguments

  • haystack: The byte slice to search.
  • needles: The set of bytes that should not be matched within the haystack.
count_utf8Counts the number of UTF-8 characters in the text.
utf8_runes.rs:109 ↗
pub fn count_utf8<Text>(text: Text) -> usize
where
    Text: AsRef<[u8]>,

This function efficiently counts UTF-8 characters by identifying character start bytes, the non-continuation bytes. Uses SIMD acceleration when available.

Returns

The number of UTF-8 characters, or codepoints, in the text.

Arguments

  • text: The UTF-8 encoded byte slice to count characters in.

Examples

use stringzilla as sz;

let text = "Hello";
assert_eq!(sz::count_utf8(text), 5);

let text_unicode = "Hello🌍";
assert_eq!(sz::count_utf8(text_unicode), 6);

let text_cjk = "你好世界";
assert_eq!(sz::count_utf8(text_cjk), 4);
find_nth_utf8Finds the byte offset of the Nth UTF-8 character (0-indexed).
utf8_runes.rs:157 ↗
pub fn find_nth_utf8<Text>(text: Text, n: usize) -> Option<usize>
where
    Text: AsRef<[u8]>,

This function efficiently locates the Nth UTF-8 character without decoding the entire string. Uses SIMD acceleration when available.

Returns

An Option<usize> containing the byte offset of the Nth character. Returns None if the string has fewer than N+1 characters.

Arguments

  • text: The UTF-8 encoded byte slice to search.
  • n: The 0-based index of the character to find.

Examples

use stringzilla as sz;

let text = "Hello";
assert_eq!(sz::find_nth_utf8(text, 0), Some(0)); // 'H'
assert_eq!(sz::find_nth_utf8(text, 4), Some(4)); // 'o'
assert_eq!(sz::find_nth_utf8(text, 5), None);

let text_unicode = "Hello🌍";
assert_eq!(sz::find_nth_utf8(text_unicode, 5), Some(5)); // 🌍 starts at byte 5
assert_eq!(sz::find_nth_utf8(text_unicode, 6), None);

Constants

SUBSTRINGS_HOT_STATES_AUTOThe hot_states argument's "size the hot tier yourself" value, so zero stays a real all-cold request.
substrings.rs:14 ↗
pub const SUBSTRINGS_HOT_STATES_AUTO: usize = usize::MAX;
SUBSTRINGS_MATCHES_BUDGET_AUTOThe matches_budget argument's "let the tier choose" value; a host tier reads no budget at all.
substrings.rs:17 ↗
pub const SUBSTRINGS_MATCHES_BUDGET_AUTO: usize = 0;