Linear Algebra

§ Sparse Multiplication

multiply is the sparse sibling of search: every left row emits one pair per column the walk scored rather than one ranked match, so the scores are unnormalized logits and the index must hold the Inner kind. How many pairs a walk scores is known only once it ends, so the call answers with a DenseProducts run viewing its arena, laid out with Apache Arrow offsets:

const index = usearch.DenseIndex.initRam({ metric: usearch.DenseMetric.Inner, dimensions: 128 });
const products = index.on(executor).multiply(left, arena, { wanted: 64 }); // Up to 64 pairs per left row
for (let row = 0; row < products.queries; row++) {
  const pairs = products.get(row); // A QueryMatches over `offsets[row]` up to `offsets[row + 1]`
  for (let pair = 0; pair < pairs.count; pair++) accumulate(row, usearch.keyAt(pairs.keys, pair), pairs.scores[pair]);
}
products.required; // Pairs scored across every left row, before `wanted` cut any

Pairs arrive in traversal order rather than sorted, and a wanted bound keeps the first ones each walk scored; leaving it out keeps every pair. The Arrow offsets, one longer than the left rows, land in the arena beside the pairs.

§ Late Interaction

An index whose valuesPerKey is above one holds several token rows per document, and maxsim scores documents rather than rows. It answers the DenseProducts run multiply does, one run of documents per query, largest first, and a zero wanted keeps every document a query scored:

const index = usearch.DenseIndex.initRam({ metric: usearch.DenseMetric.Inner, dimensions: 128, valuesPerKey: 64 });
index.on(executor).insert(documentKeys, tokenRows);

// 32 padded token rows per query, up to 10 documents each, query `i` spanning `offsets[i]` up to `offsets[i + 1]`
const documents = index.on(executor).maxsim(queryTokens, 32, 10, arena);
const best = documents.get(0); // A QueryMatches over the first query's documents

§ Attention

attend is softmax attention over every key a query’s graph walk scores, folded as Flash Attention folds key tiles, with no top-k cut: the long tail the walk scores is what brings it toward exact dense attention. It lands one key-wide row per query in the arena, in the queries’ own kind unless scalar names another. The index stores no values: every key a walk scores, 16 bytes apiece, resolves through rowOf to a row of values, borrowed for the call rather than copied, as wide as the keys and stored in the index’s own scalar kind. Answering with no row of values, or lending values of another width or kind, refuses the call with ShapeMismatch, since a missing row is a wrong softmax rather than a smaller one.

// One value row per indexed key, stored like the keys
const values = usearch.RowsView.of(valueMatrix, dimensions);
// The row of `values` a 16-byte key resolves to
const rowOf = (key) => tokenRows.get(usearch.integerFromKey(key));
// A largest logit and a partition per query row, lent and updated in place
const carry = new Float32Array(queries.rows * 2);
// Row `t * 8 + h` is head `h` of token `t`
const outputs = index.attend(queries, values, rowOf, arena, { headCount: 8, carry, scoreScale });
// One key-wide row per query, `outputs.stride` bytes apart
outputs.data;

One index holds one layer’s one key-value head, so the headCount query heads of grouped-query attention share its keys and one walk per token. The first exception rowOf throws stops every later lookup and surfaces as itself. rowOf runs on the host, so attend serves RAM handles alone: a VRAM walk resolves rows through a device function, which this binding does not build, and a VRAM arena refuses with Status.Device. The carry is what folds one branch into another over the same queries, by the usual partial-softmax rescaling: attention sinks, a dense recent window, or a second index of one context. Its two floats travel in one row because a maximum without its partition is a number nothing can rescale, and omitting it altogether starts the softmax fresh. scoreScale multiplies every logit and derives from the geometry when left out, and a wider expansionFactor walks further, folding more of the tail.