Zig · Retrieval and Streaming

Retrieval and Streaming

const usearch = @import("usearch");const DenseVectors = @import("usearch").DenseVectors;const DenseCursorBatch = @import("usearch").DenseCursorBatch;const DenseDraws = @import("usearch").DenseDraws;const DenseCursor = @import("usearch").DenseCursor;const DenseSearchStream = @import("usearch").DenseSearchStream; 3.0.0-rc.1 · 4c3e5023

get lands every vector stored under a key in the arena, one packed row per entry in insertion order, as the scalar you name; an absent key yields no rows.

const vectors = try index.get(key, .f32, &arena);
for (0..vectors.rows) |row| consume(try vectors.row(f32, row));

DenseVectors.stride is the byte pitch of each row and columns its dimensions, as the call reported them, which is how a sub-byte kind like .u1x8 is read without arithmetic on its packing. row refuses an element type other than the kind the rows were written as with Error.InvalidArgument, rather than reinterpreting them.

A cursor walks the index contents, reading each shard’s publication watermark as it enters that shard, so an entry once passed is never revisited. Without start keys it yields every entry as stored; with them, the starts’ own entries first, then the entries those link to breadth-first, each once. Each pull lands up to wanted keys in the arena, and the keys each links to when links is true; the vectors themselves come from get.

var cursor = try usearch.DenseCursor.init(index, &.{});
defer cursor.deinit();

while (true) {
    const batch = try index.on(executor).cursorNext(cursor, 1024, true, &arena);
    if (batch.written == 0) break; // exhausted
    consume(try batch.keys(), try batch.links(0));
}

A search stream is a resumable best-first traversal, so callers that post-filter aggressively pull until satisfied instead of guessing the “k” upfront. It keeps walk state rather than a thread, so any number may stay open on one executor, and every pull is an ordinary verb naming the index again, so it runs on whichever receiver pulls.

var stream = try usearch.DenseSearchStream.init(index, try .ofRow(f32, &query), .{});
defer stream.deinit();

while (true) {
    const found = try (try index.on(executor).searchStreamNext(stream, 16, &arena)).query(0);
    if (found.keys.len == 0) break; // drained
    for (found.keys, found.scores) |key, distance| std.debug.print("{f} at {d}\n", .{ key, distance });
}

A drained walk answers an empty query, as the C interface does, so a loop ends on the first pull that lands nothing.

§ Sampling

sample draws keys around focus, or across the whole index, each hanging from a shown or earlier draw and placed in three dimensions beside the shown keys already on screen, so a view grows one connected sample as it zooms. Equal seeds, shown keys and focus draw equally, and a larger wanted draws a superset.

const draws = try index.on(executor).sample(256, &arena, .{ .shown = shown, .shown_coordinates = places, .focus = near, .seed = 42 });
for (try draws.keys(), try draws.entryOrdinals(), 0..) |key, entry_ordinal, position| {
    const parent = try draws.parents.query(position); // empty for a root
    const point = try draws.coordinates.row(f32, position);
    _ = .{ key, entry_ordinal, parent, point };
}

API

Types

DenseSampleConfigOptional settings of DenseIndex.sample.
pub const DenseSampleConfig = struct

Fields

shown[]const KeyThe keys already shown: never drawn again, and the paths reach toward them.
shown_coordinates[]const [3]f32Where each shown key sits, one point apiece.
focus[]const KeyThe keys to look around; empty looks around the whole index.
seedu64Equal seeds, shown keys and focus over an unchanged index draw equally.
expansion_factorusizeThe breadth of the candidates of the walks behind the draws; zero derives.
progressProgressCallback
progress_stepsusizeReports the call makes at most, the last at completion; zero derives to ten.
stats?*DenseTraversalStats
DenseSearchStreamInitConfigOptional settings of DenseSearchStream.init.
pub const DenseSearchStreamInitConfig = struct

Fields

expansion_factorusizeThe breadth of the candidates the walk explores; zero derives.
diagnostics?*OutcomeWhere the call's final outcome, its static message included, is copied; null copies nothing.
DenseGetOne DenseIndex.get, as a Deferred holds it.
pub const DenseGet = struct

Fields

keyKey
scalarScalar
stampArena.StampThe call the borrowed arena serves, which every view over the outputs carries.
vectorsusearch_rows_view_t
writtenusize
outcomeOutcome
DenseSampleOne DenseIndex.sample, as a Deferred holds it.
pub const DenseSample = struct

Fields

wantedusize
configDenseSampleConfig
stampArena.StampThe call the borrowed arena serves, which every view over the outputs carries.
keysusearch_keys_view_t
entry_ordinals?[*]const usize
parentsusearch_matches_view_t
coordinatesusearch_rows_view_t
writtenusize
outcomeOutcome
DenseCursorNextOne DenseIndex.cursorNext, as a Deferred holds it.
pub const DenseCursorNext = struct

Fields

cursorDenseCursor
wantedusize
linksbool
stampArena.StampThe call the borrowed arena serves, which every view over the outputs carries.
keysusearch_keys_view_t
linkedusearch_keys_tape_view_t
writtenusize
outcomeOutcome
DenseSearchStreamNextOne DenseIndex.searchStreamNext, as a Deferred holds it.
pub const DenseSearchStreamNext = struct

Fields

streamDenseSearchStream
wantedusize
stampArena.StampThe call the borrowed arena serves, which every view over the outputs carries.
matchesusearch_matches_view_t
outcomeOutcome
StructDenseVectorsRows a call wrote packed into an arena - get one per entry under the key, attend one per query, a cursor pull one per entry - stride bytes apiece, which sub-byte kinds round up to whole bytes.3
dense.zig:2110 ↗
pub const DenseVectors = struct

Mirrors usearch_rows_view_t.

It reads the arena, so every read is refused once that arena serves another call or is freed.

Fields

stampArena.Stamp
data?[*]const u8
rowsusizeRows written.
strideusizeBytes per row, as the call reported them.
columnsusizeDimensions per row, as the call reported them.
scalarScalarThe scalar type the rows were written as.
rowOne row as a run of Element, the type the rows were written as; any other type and a position past rows are refused with Error.InvalidArgument, a stale view with Error.Uninitialized, and device memory with Error.Device.
dense.zig:2129 ↗
pub fn row(
    self: DenseVectors,
    comptime Element: type,
    position: usize,
) Error![]const Element
bytesEvery row, back to back, refused as row refuses a stale view or device memory.
dense.zig:2137 ↗
pub fn bytes(self: DenseVectors) Error![]const u8
residencyWhere the rows live, which the host reads only as .ram or .nvme.
dense.zig:2143 ↗
pub fn residency(self: DenseVectors) Residency
StructDenseCursorBatchOne batch a DenseCursor pull wrote into an arena: the keys, and the keys each links to where asked for; the vectors themselves come from DenseIndex.get.2
dense.zig:2223 ↗
pub const DenseCursorBatch = struct

It reads the arena, so every read is refused once that arena serves another call or is freed.

Fields

stampArena.Stamp
data?[*]const Key
writtenusizeEntries the pull wrote; zero once the walk is exhausted.
linked?usearch_keys_tape_view_tThe keys each entry links to, or null where the pull exported keys alone.
keysThe keys, refusing a stale view with Error.Uninitialized and device memory with Error.Device.
dense.zig:2232 ↗
pub fn keys(self: DenseCursorBatch) Error![]const Key
StructDenseDrawsThe draws DenseIndex.sample wrote in an arena, one per draw: its key, which of that key's entries it is, the shown or earlier draw it hangs from, and where it is placed.2
dense.zig:2252 ↗
pub const DenseDraws = struct

It reads the arena, so every read is refused once that arena serves another call or is freed.

Fields

stampArena.Stamp
writtenusizeDraws written; fewer than wanted only once the region is exhausted.
keys_data?[*]const Key
entry_ordinals_data?[*]const usize
parentsMatchesEach draw's parent and its distance, as a one-column run per draw; a root holds none.
coordinatesDenseVectorsWhere each draw is placed, one row of three f32 apiece.
keysThe keys drawn, refusing a stale view with Error.Uninitialized and device memory with Error.Device.
dense.zig:2264 ↗
pub fn keys(self: DenseDraws) Error![]const Key
entryOrdinalsWhich of its key's entries each draw is, in the order get writes them, refused as keys is.
dense.zig:2270 ↗
pub fn entryOrdinals(self: DenseDraws) Error![]const usize
StructDenseCursorA sequential walk over the index contents - export, migration, and visualization feed off it.2
dense.zig:2279 ↗
pub const DenseCursor = struct

Every pull names the index again, through DenseIndex.cursorNext, so free the cursor before the index. Mirrors usearch_dense_cursor_t.

Fields

handle?*anyopaqueThe opaque usearch_dense_cursor_t the ABI hands back.
initOpens a walk over the current contents; each shard's publication watermark is read as the walk enters it, so an entry once passed is never revisited.
dense.zig:2287 ↗
pub fn init(
    index: DenseIndex,
    starts: []const Key,
) Error!DenseCursor

Without starts it yields every entry as stored; with them, the start keys' own entries first in order, then the entries those link to breadth-first, each once. Calls usearch_dense_cursor_init.

deinitReleases the cursor, waiting for every deferred pull naming it.
dense.zig:2304 ↗
pub fn deinit(self: *DenseCursor) void

Calls usearch_dense_cursor_free.

StructDenseSearchStreamA resumable nearest-neighbor stream over one query.2
dense.zig:2317 ↗
pub const DenseSearchStream = struct

Best-first traversal is naturally pausable, so the stream keeps a live frontier and visited set and yields matches lazily - callers that post-filter aggressively pull until satisfied instead of guessing "k" upfront. It keeps walk state rather than a thread, and every pull names the index again through DenseIndex.searchStreamNext, so free the stream before the index. Mirrors usearch_dense_search_stream_t.

Fields

handle?*anyopaqueThe opaque usearch_dense_search_stream_t the ABI hands back.
initOpens a stream for one query row, refusing a run of several with Error.ShapeMismatch.
dense.zig:2323 ↗
pub fn init(
    index: DenseIndex,
    query: RowsView,
    config: DenseSearchStreamInitConfig,
) Error!DenseSearchStream

Calls usearch_dense_search_stream_init.

deinitReleases the stream and the walk state it keeps, waiting for every deferred pull naming it.
dense.zig:2343 ↗
pub fn deinit(self: *DenseSearchStream) void

Calls usearch_dense_search_stream_free.