Every verb keeps the pooled embeddings by default.
Ask for the final-norm states too, one row per token kept, with the outputs flags:
const { embeddings, states, stats } = model.textEncode(texts, arena, {
outputs: Outputs.Embeddings | Outputs.States,
outputsScalar: Scalar.BF16,
});
const tokensOfFirst = states.item(0); // rows from states.offsets[0] up to states.offsets[1]
console.log(stats.truncated, stats.tokens);A checkpoint with a token head also lands unit token vectors for late-interaction reranking, tokenWidth wide and sharing the states’ offsets:
const { tokenWidth } = model.describe(); // zero without a token head, which answers `unsupported`
const { tokenVectors } = model.textEncode(documents, arena, { outputs: Outputs.TokenVectors });Images, clips and recordings arrive encoded, as JPEG, PNG, WebP, MP4, WebM, WAV, FLAC, MP3 or Opus files:
import fs from "node:fs/promises";
const images = model.imageEncode([await fs.readFile("panda.jpg")], arena);
const clips = model.videoEncode([await fs.readFile("panda.mp4")], arena);
const recordings = model.audioEncode([await fs.readFile("panda.wav")], arena);Or decoded, as RGB or RGBA pixels with their shape, and as mono samples with their rate:
const pixels = model.imageEncodePixels([{ data: rgb, width: 640, height: 480 }], arena);
const samples = model.audioEncodeSamples([new Float32Array(16000)], 16000, arena);OutputsThe surfaces an encode call keeps, combined with |.enum OutputsThe C verb fills whichever of embeddings, states and token_vectors is non-NULL, and these flags decide which it is handed.
Embeddings | 1 | One pooled embedding per item. |
|---|---|---|
States | 2 | The final-norm states, one row per token kept after truncation. |
TokenVectors | 4 | The token head's unit vectors, one row per token kept, tokenWidth wide; unsupported without a token head. |
EncodeConfigOptional per-call settings of every encode verb.interface EncodeConfigoutputs? | number | The surfaces to keep; absent keeps Outputs.Embeddings alone. |
|---|---|---|
outputsScalar? | Scalar | The scalar every output lands as; absent lands f32, which a Float32Array reads. |
progress? | ProgressCallback | Hears the items encoded so far and may stop the call by answering false; only a model's own verbs, which run on the calling thread, take one. |
PixelsOne decoded image: ImageData as a canvas hands it out, or interleaved RGB or RGBA bytes with their shape.interface Pixelsdata | Uint8Array | Uint8ClampedArray | Rows end to end, three or four bytes per pixel. |
|---|---|---|
width | number | Pixels per row. |
height | number | Rows. |
EncodeStatsWhat an encode call reports beside its outputs.interface EncodeStatstruncated | number | Items cut at the model's context. |
|---|---|---|
tokens | number | Tokens the call encoded, after truncation. |
EncodedThe surfaces one encode call kept, present as they were asked for.interface Encodedembeddings? | Rows | One pooled embedding per item. |
|---|---|---|
states? | States | The final-norm states of every item. |
tokenVectors? | States | The token head's unit vectors of every item, sharing the states' offsets. |
stats | EncodeStats | Truncation and token counts. |
TapeA ragged batch packed as the C interface reads it: elements end to end, and count + 1 offsets into them.interface Tape<Elements extends Uint8Array | Float32Array>Mirrors uform_tape_view_t.
Every verb taking an array of entries also takes a packed tape, which the Node addon lends to
the call as it is when its offsets are a BigUint64Array; an array of entries is first gathered
into one tape, the one copy that path makes. A Probabilities answer is itself a tape, so it
trains a step's choiceTargets as it lands.
data | Elements | |
|---|---|---|
offsets | Offsets |
OffsetsThe count + 1 offsets of a tape: a BigUint64Array, which the Node addon lends as size_t without a copy, or a Uint32Array, which it widens and the WebAssembly module, whose size_t is 32 bits wide, lends as it is.type Offsets = BigUint64Array | Uint32ArrayScalarThe scalar kinds a model computes in and its outputs land as, numbered as USearch numbers them.enum ScalarF64 | "f64" | |
|---|---|---|
F32 | "f32" | |
BF16 | "bf16" | |
F16 | "f16" | |
E5M2 | "e5m2" | |
E4M3 | "e4m3" | |
E3M2 | "e3m2" | |
E2M3 | "e2m3" | |
I8 | "i8" | |
I4x2 | "i4x2" | |
I32 | "i32" | |
U8 | "u8" | |
U4x2 | "u4x2" | |
U1x8 | "u1x8" |
RowsRows a call landed in its arena.7class Rows extends ArenaViewMirrors a uform_rows_view_t output.
data is a Float32Array for f32, a Float16Array for f16 where the runtime has one, a
Uint16Array for bf16, and raw bytes for the kinds JavaScript spells no element for.
RowsMade by a verb, never directly.new Rows(landed: CompiledRows)rowBorrows row row without copying.rowsRows written.readonly rows: numbercolumnsValues in each row, the model's dimensions.readonly columns: numberstrideBytes between consecutive rows.readonly stride: numberscalarThe scalar kind the rows landed as.readonly scalar: ScalardataEvery row, laid stride bytes apart.readonly data: ArrayBufferViewStatesThe final-norm states or the token vectors a call landed in its arena: rows end to end, item i spanning rows offsets[i] up to offsets[i + 1].4StatesMade by a verb, never directly.new States(landed: CompiledStates)itemBorrows the rows of item item without copying.rowsEvery item's rows, end to end.readonly rows: RowsoffsetsThe count + 1 row offsets, the last of them the total.readonly offsets: BigUint64ArrayEncodeVerbsThe encode verbs every receiver of a model shares, written once.8abstract class EncodeVerbs<Receiving extends "blocking" | "deferred">The receiver chooses where a verb runs and how it completes, and the verb stays the same: a
Model runs it on the calling thread, model.on(executor) blocks on an executor, and
model.deferred(executor) submits it there and answers a Deferred. Every verb lands its
outputs in arena and answers views into it.
textEncodeEncodes texts through one forward; every UTF-8 byte is a token.textEncode(
texts: readonly (string | Uint8Array)[] | Tape<Uint8Array>,
arena: Arena,
config: EncodeConfig = {},
): Answer<Receiving, Encoded>Calls uform_text_encode.
textsreadonly (string | Uint8Array)[] | Tape<Uint8Array> | Strings, UTF-8 bytes already encoded, or a tape of them. | |
|---|---|---|
arenaArena | The memory the outputs land in, rewound by the call. | |
configEncodeConfig | The surfaces, their scalar, and a progress handler. |
arena.textChooseScores each item's legal choices through one forward, a softmax over them alone at the item's group's calibrated inverse temperature; a model without the byte decoder reports unsupported.textChoose(
texts: readonly (string | Uint8Array)[] | Tape<Uint8Array>,
choices: readonly (string | Uint8Array)[] | Tape<Uint8Array>,
arena: Arena,
config: ChooseConfig = {},
): Answer<Receiving, Chosen>Calls uform_text_choose.
textsreadonly (string | Uint8Array)[] | Tape<Uint8Array> | The states, each ending where its choice goes. | |
|---|---|---|
choicesreadonly (string | Uint8Array)[] | Tape<Uint8Array> | Each item's legal choices, one byte each, like "yn". | |
arenaArena | The memory the probabilities land in, rewound by the call. | |
configChooseConfig | The question types and a progress handler. |
arena.imageEncodeDecodes and encodes JPEG, PNG or WebP files; the browser build reports unsupported.imageEncode(
images: readonly Uint8Array[] | Tape<Uint8Array>,
arena: Arena,
config: EncodeConfig = {},
): Answer<Receiving, Encoded>Calls uform_image_encode.
imagesreadonly Uint8Array[] | Tape<Uint8Array> | One encoded file per item, or a tape of them. | |
|---|---|---|
arenaArena | The memory the outputs land in, rewound by the call. | |
configEncodeConfig | The surfaces, their scalar, and a progress handler. |
imageEncodePixelsEncodes decoded images, resized on the executor's side.imageEncodePixels(
images: readonly Pixels[],
arena: Arena,
config: EncodeConfig = {},
): Answer<Receiving, Encoded>Calls uform_image_encode_pixels.
imagesreadonly Pixels[] | ImageData, or RGB or RGBA bytes with their width and height. | |
|---|---|---|
arenaArena | The memory the outputs land in, rewound by the call. | |
configEncodeConfig | The surfaces, their scalar, and a progress handler. |
videoEncodeDecodes and encodes MP4 or WebM clips; the browser build reports unsupported.videoEncode(
videos: readonly Uint8Array[] | Tape<Uint8Array>,
arena: Arena,
config: EncodeConfig = {},
): Answer<Receiving, Encoded>Calls uform_video_encode.
videosreadonly Uint8Array[] | Tape<Uint8Array> | One encoded file per item, or a tape of them. | |
|---|---|---|
arenaArena | The memory the outputs land in, rewound by the call. | |
configEncodeConfig | The surfaces, their scalar, and a progress handler. |
audioEncodeDecodes and encodes WAV, FLAC, MP3 or Opus files; the browser build reports unsupported.audioEncode(
recordings: readonly Uint8Array[] | Tape<Uint8Array>,
arena: Arena,
config: EncodeConfig = {},
): Answer<Receiving, Encoded>Calls uform_audio_encode.
recordingsreadonly Uint8Array[] | Tape<Uint8Array> | One encoded file per item, or a tape of them. | |
|---|---|---|
arenaArena | The memory the outputs land in, rewound by the call. | |
configEncodeConfig | The surfaces, their scalar, and a progress handler. |
audioEncodeSamplesEncodes decoded mono recordings, resampled to the model's rate where sampleRate differs.audioEncodeSamples(
recordings: readonly Float32Array[] | Tape<Float32Array>,
sampleRate: number,
arena: Arena,
config: EncodeConfig = {},
): Answer<Receiving, Encoded>Calls uform_audio_encode_samples.
recordingsreadonly Float32Array[] | Tape<Float32Array> | One run of samples per item, or a tape of them. | |
|---|---|---|
sampleRatenumber | Samples per second shared by every recording. | |
arenaArena | The memory the outputs land in, rewound by the call. | |
configEncodeConfig | The surfaces, their scalar, and a progress handler. |
executorThe executor the verbs run on; absent runs them on the calling thread.readonly executor: Executor | undefined