C · Retrieval and Streaming

Retrieval and Streaming

#include <usearch/dense_index.h>#include <usearch/geospatial_index.h>#include <usearch/sequence_index.h>#include <usearch/shapes_index.h> 3.0.0-rc.1 · 4c3e5023

usearch_dense_get writes every vector stored under one key into an arena, shard by shard and in insertion order within each, as the Arenas example shows; an absent key leaves vectors->data NULL. A cursor walks the graph, every entry in storage order or breadth-first from starts, its keys and, when asked, each entry’s links copied into an arena batch by batch:

usearch_dense_cursor_t cursor = NULL;
usearch_dense_cursor_init(&(usearch_dense_cursor_init_t) {.index = index, .outcome = &outcome, .cursor = &cursor});
usearch_keys_view_t batch = {0};
for (size_t written = 1; written != 0 && outcome.status == usearch_status_success_k;)
    usearch_dense_cursor_next(&(usearch_dense_cursor_next_t) {.index = index,
                                                              .executor = executor,
                                                              .outcome = &outcome,
                                                              .cursor = cursor,
                                                              .keys = &batch,
                                                              .written = &written,
                                                              .arena = arena,
                                                              .wanted = 1024});
usearch_dense_cursor_free(cursor);

Each shard’s publication watermark is read as the walk enters it, so an entry once passed is never revisited, while a later shard may carry entries published after the walk began. Opened at starts, a cursor yields their own entries first, in the order given, then breadth-first the entries they link to, each entry once, so the first entries pulled carry the starts’ immediate links. links is a usearch_keys_tape_view_t, entry i spanning offsets[i] up to offsets[i+1], and the vectors themselves come from usearch_dense_get.

When a predicate is selective enough that no wanted can be guessed upfront, a search stream keeps pulling matches in distance order:

usearch_dense_search_stream_t stream = NULL;
usearch_dense_search_stream_init(&(usearch_dense_search_stream_init_t) {
    .stream = &stream, .index = index, .outcome = &outcome, .query = {query, 0, 128, usearch_scalar_f32_k}});
do {
    usearch_dense_search_stream_next(&(usearch_dense_search_stream_next_t) {.index = index,
                                                                            .executor = executor,
                                                                            .outcome = &outcome,
                                                                            .stream = stream,
                                                                            .matches = &matches,
                                                                            .arena = arena,
                                                                            .wanted = 10});
    // ... keep the matches that pass, and stop once enough did ...
} while (outcome.status == usearch_status_success_k && matches.counts[0] != 0);
usearch_dense_search_stream_free(stream);

No residency implements streams yet, so usearch_dense_search_stream_init answers usearch_status_unsupported_k today. A stream keeps the walk’s state between pulls, never a thread, so any number of them may stay open on one executor. Cursors and streams keep no hold on the index either: every pull passes the index it was opened on, any other is refused with usearch_status_invalid_argument_k, and an overlapping pull reports usearch_status_contended_k.

API

Types

usearch_dense_cursor_tA walk over the graph yielding each entry's key with the keys it links to, every entry as stored or breadth-first from start keys, which export, migration, and visualization read.
dense_index.h:198 ↗
typedef struct usearch_dense_cursor_handle_t * usearch_dense_cursor_t;

Each shard's publication watermark is read as the walk enters that shard, so an entry once passed is never revisited, while a later shard may still carry entries published after the walk began. Each pull names the index again; an overlapping pull reports usearch_status_contended_k.

usearch_dense_search_stream_tA resumable search over one query: best-first traversal is naturally pausable, so the stream keeps the walk's frontier & visited set between pulls and yields matches lazily.
dense_index.h:208 ↗
typedef struct usearch_dense_search_stream_handle_t * usearch_dense_search_stream_t;

Callers that post-filter aggressively pull until satisfied instead of guessing wanted upfront. The stream holds state, never a thread, so any number of them may stay open on one executor. Each pull names the index again; an overlapping pull reports usearch_status_contended_k.

usearch_geospatial_cursor_tA consistent snapshot cursor over the graph.
geospatial_index.h:95 ↗
typedef struct usearch_geospatial_cursor_handle_t * usearch_geospatial_cursor_t;

Captures the publication watermark at creation, so concurrent inserts never appear mid-iteration, and yields each entry's key with the keys it links to - every entry as stored, or breadth-first from start keys - which export, migration, and visualization feed off. Each pull names the index again; an overlapping pull reports usearch_status_contended_k.

usearch_sequence_cursor_tA consistent snapshot cursor over the graph.
sequence_index.h:76 ↗
typedef struct usearch_sequence_cursor_handle_t * usearch_sequence_cursor_t;

Captures the publication watermark at creation, so concurrent inserts never appear mid-iteration, and yields each entry's key with the keys it links to - every entry as stored, or breadth-first from start keys - which export, migration, and visualization feed off. Each pull names the index again; an overlapping pull reports usearch_status_contended_k.

usearch_shapes_cursor_tA consistent snapshot cursor over the graph.
shapes_index.h:123 ↗
typedef struct usearch_shapes_cursor_handle_t * usearch_shapes_cursor_t;

Captures the publication watermark at creation, so concurrent inserts never appear mid-iteration, and yields each entry's key with the keys it links to - every entry as stored, or breadth-first from start keys - which export, migration, and visualization feed off. Each pull names the index again; an overlapping pull reports usearch_status_contended_k.

Functions

usearch_dense_getRetrieves every vector stored under one key into arena, one row per entry sharing the key, shard by shard and in insertion order within each - one call, however many there are.
dense_index.h:1122 ↗
void usearch_dense_get(usearch_dense_get_t *arguments);

The vectors span written entries; an absent key leaves vectors->data NULL.

Parameters

indexusearch_dense_index_tThe handle to be queried.
executorusearch_executor_tThe executor whose stream or thread copies the vectors; its kind must match the residency.
outcomeusearch_outcome_t *Output for the outcome, final once its status leaves usearch_status_deferred_k.
completionusearch_completion_tBlocking, the default, or deferred.
keyusearch_key_tThe key whose entries are exported.
vectorsusearch_rows_view_t *Output for every vector, packed at their natural width, inside arena.
writtensize_t *The optional output for the vectors written; zero when the key is absent.
arenausearch_arena_tMemory for vectors, rewound by the call.
outputs_scalarusearch_scalar_tThe scalar type to export as, converted from storage where it differs.

Statuses

usearch_dense_cursor_initOpens a walk over the current contents; see usearch_dense_cursor_t.
dense_index.h:1520 ↗
void usearch_dense_cursor_init(usearch_dense_cursor_init_t *arguments);

Without starts the cursor yields every entry as stored. With them it yields the start keys' own entries first, in the order given, then the entries those link to, breadth-first, each entry once, so a caller wanting only the starts' immediate links reads the links of the first pulled entries. A key the index lacks starts nothing, and a key several entries share starts from its first stored entry.

Parameters

indexusearch_dense_index_tThe handle to be scanned.
outcomeusearch_outcome_t *Output for the outcome, final once its status leaves usearch_status_deferred_k.
starts_countsize_tThe optional count of starts; zero walks every entry as stored.
startsusearch_key_t const *The keys to walk from, in the order the walk begins with them.
cursorusearch_dense_cursor_t *Output for the new cursor; NULL on failure.

Statuses

usearch_dense_cursor_nextCopies the next batch of entries out of the cursor into arena, each with the keys it links to, in the order usearch_dense_cursor_init describes; the vectors themselves come from usearch_dense_get.
dense_index.h:1570 ↗
void usearch_dense_cursor_next(usearch_dense_cursor_next_t *arguments);

Links name only entries published when the pull ran, as the graph stored them then. The keys are copied rather than borrowed from the index, so the batch stays valid until the next call given the same arena, however the cursor moves meanwhile.

Parameters

indexusearch_dense_index_tThe handle the cursor was opened on.
executorusearch_executor_tThe executor the pull runs on.
outcomeusearch_outcome_t *Output for the outcome, final once its status leaves usearch_status_deferred_k.
completionusearch_completion_tBlocking, the default, or deferred.
cursorusearch_dense_cursor_tThe cursor to advance.
keysusearch_keys_view_t *Output for the identities, inside arena.
linksusearch_keys_tape_view_t *The optional output for each entry's links, the keys of the entries it links to, entry i spanning offsets[i] up to offsets[i+1], inside arena; NULL skips them.
writtensize_t *Output for the entries written; zero when the walk is exhausted.
arenausearch_arena_tMemory for keys and links, rewound by the call.
wantedsize_tThe upper bound on entries this pull writes.

Statuses

usearch_*_cursor_free× 4Releases the cursor; NULL is safe.
dense_index.h:1574 ↗
void usearch_dense_cursor_free(usearch_dense_cursor_t cursor);
Symbolcursor
usearch_dense_cursor_freeusearch_dense_cursor_t
usearch_geospatial_cursor_freeusearch_geospatial_cursor_t
usearch_sequence_cursor_freeusearch_sequence_cursor_t
usearch_shapes_cursor_freeusearch_shapes_cursor_t

Waits until every deferred call naming the handle is final.

usearch_*_sample× 4Draws keys around focus, or across the whole index, each hanging from a shown or earlier draw and placed in three dimensions beside the shown ones - so a view grows one connected sample as it zooms, never scanning the index.
dense_index.h:1670 ↗
void usearch_dense_sample(usearch_dense_sample_t *arguments);
Symbolindex
usearch_dense_sampleusearch_dense_index_t
usearch_geospatial_sampleusearch_geospatial_index_t
usearch_sequence_sampleusearch_sequence_index_t
usearch_shapes_sampleusearch_shapes_index_t

The draws follow the graph: along paths joining the focus to each other and toward the nearest shown key, then around each focus key as far as its nearest neighbor on those paths, ranked by seed so a closer view draws a superset of a farther one. A key holding several entries draws each as its own, named by its entry ordinal. Each draw is placed by its distances to its nearest shown keys, the shown keys keeping their coordinates; with nothing shown, its row is projected through a seeded Gaussian matrix.

The draws follow the graph: along paths joining the focus to each other and toward the nearest shown key, then around each focus key as far as its nearest neighbor on those paths, ranked by seed so a closer view draws a superset of a farther one. A key holding several entries draws each as its own, named by its entry ordinal. Each draw is placed at its Earth-fixed position, so shown_coordinates place nothing.

The draws follow the graph: along paths joining the focus to each other and toward the nearest shown key, then around each focus key as far as its nearest neighbor on those paths, ranked by seed so a closer view draws a superset of a farther one. A key holding several entries draws each as its own, named by its entry ordinal. Each draw is placed by its distances to its nearest shown keys, the shown keys keeping their coordinates; with nothing shown, the draws are scaled against one another.

Parameters

indexusearch_dense_index_tThe handle to be sampled.
executorusearch_executor_tThe executor the draws run on; its kind must match the handle's residency.
outcomeusearch_outcome_t *Output for the outcome, final once its status leaves usearch_status_deferred_k.
completionusearch_completion_tBlocking, the default, or deferred.
shown_countsize_tThe optional count of shown; zero shows nothing yet.
shownusearch_keys_view_tThe keys already shown: never drawn again, and the paths reach toward them.
shown_coordinatesusearch_rows_view_tWhere the shown keys sit, shown_count rows of three usearch_scalar_f32_k values, the points later draws are placed against.
focus_countsize_tThe optional count of focus; zero looks around the whole index.
focususearch_keys_view_tThe keys to look around, the roots the draws grow from.
keysusearch_keys_view_t *Output for the keys drawn, inside arena.
entry_ordinalssize_t const **Output for which of its key's entries each draw is, in the order usearch_dense_get writes them, inside arena.
parentsusearch_matches_view_t *Output for the one shown or earlier draw each draw hangs from and its distance, one column, a draw with none counting zero, inside arena.
coordinatesusearch_rows_view_t *Output for where each draw is placed, written rows of three usearch_scalar_f32_k values, inside arena.
writtensize_t *Output for the draws written; fewer than wanted only once the region is exhausted.
arenausearch_arena_tMemory for every output, rewound by the call.
wantedsize_tThe upper bound on draws this call writes.
seeduint64_tThe draw: equal seeds, shown keys and focus over an unchanged index draw equally, and a larger wanted draws a superset.
expansion_factorsize_tThe optional breadth of the candidates of the walks behind the draws; zero derives.
progressusearch_progress_tThe optional handler, called with draws written and wanted.
statsusearch_traversal_stats_t *The optional traversal telemetry; NULL skips reporting.

Statuses

usearch_dense_search_stream_initOpens a resumable search for one query; see usearch_dense_search_stream_t.
dense_index.h:1698 ↗
void usearch_dense_search_stream_init(
    usearch_dense_search_stream_init_t *arguments);

Parameters

streamusearch_dense_search_stream_t *Output for the new stream; NULL on failure.
indexusearch_dense_index_tThe handle to be queried.
outcomeusearch_outcome_t *Output for the outcome, final once its status leaves usearch_status_deferred_k.
queryusearch_rows_view_tThe single query row.
expansion_factorsize_tThe optional breadth of the candidates the walk explores, as in usearch_dense_search_t; zero derives.

Statuses

usearch_dense_search_stream_nextPulls the next-nearest matches from the stream.
dense_index.h:1739 ↗
void usearch_dense_search_stream_next(
    usearch_dense_search_stream_next_t *arguments);

Ordering beyond the expansion horizon is best-effort nearest rather than globally exact, so a caller pulling far past the breadth it configured sees the ranking degrade rather than the stream end.

Parameters

indexusearch_dense_index_tThe handle the stream was opened on.
executorusearch_executor_tThe executor the pull runs on.
outcomeusearch_outcome_t *Output for the outcome, final once its status leaves usearch_status_deferred_k.
completionusearch_completion_tBlocking, the default, or deferred.
streamusearch_dense_search_stream_tThe stream to advance.
matchesusearch_matches_view_t *Output for the pull, as one row of a search inside arena: up to wanted more matches, their number in matches->counts[0], zero once the walk is drained.
arenausearch_arena_tMemory for matches, rewound by the call.
wantedsize_tThe upper bound on matches this pull writes.

Statuses

usearch_dense_search_stream_freeReleases the stream and the walk state it keeps; NULL is safe.
dense_index.h:1743 ↗
void usearch_dense_search_stream_free(
    usearch_dense_search_stream_t stream);

Waits until every deferred call naming the handle is final.

usearch_geospatial_getRetrieves every observation stored under one key, in insertion order, within the rounding the index stores them at.
geospatial_index.h:1102 ↗
void usearch_geospatial_get(usearch_geospatial_get_t *arguments);

An absent key leaves observations NULL.

Parameters

indexusearch_geospatial_index_tThe handle to be queried.
executorusearch_executor_tThe executor whose stream or thread copies the observations; its kind must match the residency.
outcomeusearch_outcome_t *Output for the outcome, final once its status leaves usearch_status_deferred_k.
completionusearch_completion_tBlocking, the default, or deferred.
keyusearch_key_tThe key whose entries are exported.
observationsusearch_geospatial_observation_t const **Output for the observations inside arena, written of them.
writtensize_t *The optional output for the entries written; zero when the key is absent.
arenausearch_arena_tMemory for observations, rewound by the call.

Statuses

usearch_geospatial_cursor_initOpens a snapshot cursor over the current contents; see usearch_geospatial_cursor_t.
geospatial_index.h:1423 ↗
void usearch_geospatial_cursor_init(
    usearch_geospatial_cursor_init_t *arguments);

Without starts the cursor yields every entry as stored. With them it yields the start keys' own entries first, in the order given, then the entries those link to, breadth-first, each entry once, so a caller wanting only the starts' immediate links reads the links of the first pulled entries. A key the index lacks starts nothing, and a key several entries share starts from the first stored.

Parameters

indexusearch_geospatial_index_tThe handle to be scanned.
outcomeusearch_outcome_t *Output for the outcome, final once its status leaves usearch_status_deferred_k.
starts_countsize_tThe optional count of starts; zero walks every entry as stored.
startsusearch_key_t const *The keys to walk from, in the order the walk begins with them.
cursorusearch_geospatial_cursor_t *Output for the new cursor; NULL on failure.

Statuses

usearch_geospatial_cursor_nextCopies the next batch of entries out of the cursor into arena, each with the keys it links to, in the order usearch_geospatial_cursor_init describes; the observations themselves come from usearch_geospatial_get.
geospatial_index.h:1474 ↗
void usearch_geospatial_cursor_next(
    usearch_geospatial_cursor_next_t *arguments);

Links name only entries the snapshot holds, as the graph stored them when the pull ran. The keys are copied rather than borrowed from the index, so the batch stays valid until the next call given the same arena, however the cursor moves meanwhile.

Parameters

indexusearch_geospatial_index_tThe handle the cursor was opened on.
executorusearch_executor_tThe executor the pull runs on.
outcomeusearch_outcome_t *Output for the outcome, final once its status leaves usearch_status_deferred_k.
completionusearch_completion_tBlocking, the default, or deferred.
cursorusearch_geospatial_cursor_tThe cursor to advance.
keysusearch_keys_view_t *Output for the identities, inside arena.
linksusearch_keys_tape_view_t *The optional output for each entry's links, the keys of the entries it links to in the graph, entry i spanning offsets[i] up to offsets[i+1], inside arena; NULL skips them.
writtensize_t *Output for the entries written; zero when the snapshot is exhausted.
arenausearch_arena_tMemory for keys and links, rewound by the call.
wantedsize_tThe upper bound on entries this pull writes.

Statuses

usearch_sequence_getRetrieves every sequence stored under one key as an Apache Arrow tape, in insertion order - one call, however many and however long they turn out to be.
sequence_index.h:994 ↗
void usearch_sequence_get(usearch_sequence_get_t *arguments);

An absent key leaves the tape's pointers NULL.

Parameters

indexusearch_sequence_index_tThe handle to be queried.
executorusearch_executor_tThe executor whose thread copies the entries.
outcomeusearch_outcome_t *Output for the outcome, final once its status leaves usearch_status_deferred_k.
completionusearch_completion_tBlocking, the default, or deferred.
keyusearch_key_tThe key whose entries are exported.
sequencesusearch_tape_view_t *Output for the tape inside arena: entries' bytes end to end, never terminated, around written + 1 offsets.
writtensize_t *The optional output for the entries written; zero when the key is absent.
arenausearch_arena_tMemory for sequences, rewound by the call.

Statuses

usearch_sequence_cursor_initOpens a snapshot cursor over the current contents; see usearch_sequence_cursor_t.
sequence_index.h:1312 ↗
void usearch_sequence_cursor_init(
    usearch_sequence_cursor_init_t *arguments);

Without starts the cursor yields every entry as stored. With them it yields the start keys' own entries first, in the order given, then the entries those link to, breadth-first, each entry once, so a caller wanting only the starts' immediate links reads the links of the first pulled entries. A key the index lacks starts nothing, and a key several entries share starts from the first stored.

Parameters

indexusearch_sequence_index_tThe handle to be scanned.
outcomeusearch_outcome_t *Output for the outcome, final once its status leaves usearch_status_deferred_k.
starts_countsize_tThe optional count of starts; zero walks every entry as stored.
startsusearch_key_t const *The keys to walk from, in the order the walk begins with them.
cursorusearch_sequence_cursor_t *Output for the new cursor; NULL on failure.

Statuses

usearch_sequence_cursor_nextCopies the next batch of entries out of the cursor into arena, each with the keys it links to, in the order usearch_sequence_cursor_init describes; the sequences themselves come from usearch_sequence_get.
sequence_index.h:1363 ↗
void usearch_sequence_cursor_next(
    usearch_sequence_cursor_next_t *arguments);

Links name only entries the snapshot holds, as the graph stored them when the pull ran. The keys are copied rather than borrowed from the index, so the batch stays valid until the next call given the same arena, however the cursor moves meanwhile.

Parameters

indexusearch_sequence_index_tThe handle the cursor was opened on.
executorusearch_executor_tThe executor the pull runs on.
outcomeusearch_outcome_t *Output for the outcome, final once its status leaves usearch_status_deferred_k.
completionusearch_completion_tBlocking, the default, or deferred.
cursorusearch_sequence_cursor_tThe cursor to advance.
keysusearch_keys_view_t *Output for the identities, inside arena.
linksusearch_keys_tape_view_t *The optional output for each entry's links, the keys of the entries it links to in the graph, entry i spanning offsets[i] up to offsets[i+1], inside arena; NULL skips them.
writtensize_t *Output for the entries written; zero when the snapshot is exhausted.
arenausearch_arena_tMemory for keys and links, rewound by the call.
wantedsize_tThe upper bound on entries this pull writes.

Statuses

usearch_shapes_getRetrieves every shape stored under one key as an Apache Arrow tape, in insertion order - one call, however many and however large they turn out to be.
shapes_index.h:997 ↗
void usearch_shapes_get(usearch_shapes_get_t *arguments);

Each shape comes back in the pose it was inserted in, to the precision of the stored scalar.

An absent key leaves the tape's pointers NULL.

Parameters

indexusearch_shapes_index_tThe handle to be queried.
executorusearch_executor_tThe executor whose stream or thread copies the points; its kind must match the residency.
outcomeusearch_outcome_t *Output for the outcome, final once its status leaves usearch_status_deferred_k.
completionusearch_completion_tBlocking, the default, or deferred.
keyusearch_key_tThe key whose entries are exported.
shapesusearch_tape_view_t *Output for the tape inside arena: every entry's points end to end, x, y and z interleaved in outputs_scalar, around written + 1 point offsets.
writtensize_t *The optional output for the entries written; zero when the key is absent.
arenausearch_arena_tMemory for shapes, rewound by the call.
outputs_scalarusearch_scalar_tThe scalar type to export as, converted from storage where it differs.

Statuses

usearch_shapes_cursor_initOpens a snapshot cursor over the current contents; see usearch_shapes_cursor_t.
shapes_index.h:1314 ↗
void usearch_shapes_cursor_init(
    usearch_shapes_cursor_init_t *arguments);

Without starts the cursor yields every entry as stored. With them it yields the start keys' own entries first, in the order given, then the entries those link to, breadth-first, each entry once, so a caller wanting only the starts' immediate links reads the links of the first pulled entries. A key the index lacks starts nothing, and a key several entries share starts from the first stored.

Parameters

indexusearch_shapes_index_tThe handle to be scanned.
outcomeusearch_outcome_t *Output for the outcome, final once its status leaves usearch_status_deferred_k.
starts_countsize_tThe optional count of starts; zero walks every entry as stored.
startsusearch_key_t const *The keys to walk from, in the order the walk begins with them.
cursorusearch_shapes_cursor_t *Output for the new cursor; NULL on failure.

Statuses

usearch_shapes_cursor_nextCopies the next batch of entries out of the cursor into arena, each with the keys it links to, in the order usearch_shapes_cursor_init describes; the shapes themselves come from usearch_shapes_get.
shapes_index.h:1365 ↗
void usearch_shapes_cursor_next(
    usearch_shapes_cursor_next_t *arguments);

Links name only entries the snapshot holds, as the graph stored them when the pull ran. The keys are copied rather than borrowed from the index, so the batch stays valid until the next call given the same arena, however the cursor moves meanwhile.

Parameters

indexusearch_shapes_index_tThe handle the cursor was opened on.
executorusearch_executor_tThe executor the pull runs on.
outcomeusearch_outcome_t *Output for the outcome, final once its status leaves usearch_status_deferred_k.
completionusearch_completion_tBlocking, the default, or deferred.
cursorusearch_shapes_cursor_tThe cursor to advance.
keysusearch_keys_view_t *Output for the identities, inside arena.
linksusearch_keys_tape_view_t *The optional output for each entry's links, the keys of the entries it links to in the graph, entry i spanning offsets[i] up to offsets[i+1], inside arena; NULL skips them.
writtensize_t *Output for the entries written; zero when the snapshot is exhausted.
arenausearch_arena_tMemory for keys and links, rewound by the call.
wantedsize_tThe upper bound on entries this pull writes.

Statuses