C · Encoding

Encoding

#include <uform/types.h>#include <uform/encode.h> 4.0.0-rc.1 · 3ee5e534

Every encode verb runs the same tower, the modality’s entry, the transformer blocks and the final norm, and keeps whichever surfaces the caller names from one forward:

  • embeddings land one pooled row per item, past the attention pool and the projection head.
  • states land one row per token each item kept, the surface rerankers and masked heads read.
  • token_vectors land the token head’s unit vectors, token_width wide, on the same offsets as the states, for late-interaction reranking.

A NULL output skips its surface, naming none is refused with uform_status_invalid_argument_k, and a model without a token head refuses token_vectors with uform_status_unsupported_k. Every surface lands in the arena as outputs_scalar, and an item longer than the model’s context is truncated and counted in the stats. One verb per input form takes only the fields that form needs:

  • uform_text_encode takes UTF-8 texts, a byte per token, since models are tokenizer-free.
  • uform_image_encode takes JPEG, PNG or WebP files, and uform_image_encode_pixels interleaved 8-bit RGB.
  • uform_video_encode takes MP4 or WebM clips.
  • uform_audio_encode takes WAV, FLAC, MP3 or Opus files, and uform_audio_encode_samples mono samples at a rate.
uform_rows_view_t embeddings = {0};
uform_rows_tape_view_t states = {0};
uform_text_encode(&(uform_text_encode_t){
    .model = model, .executor = executor, .outcome = &outcome, .count = n,
    .texts = {bytes, offsets, uform_scalar_u8_k}, .embeddings = &embeddings, .states = &states,
    .arena = arena, .outputs_scalar = uform_scalar_bf16_k});

API

Types

uform_bytes_view_tA borrowed run of bytes and its length, never null-terminated by contract - the shape every path, checkpoint buffer and name arrives in.
types.h:158 ↗
struct uform_bytes_view_t

Fields

datauint8_t const *First byte of the run.
lengthsize_tBytes the run spans, with no terminator assumed or written.
uform_scalar_tScalar types a model computes in and its outputs land as, numbered as USearch's usearch_scalar_t, so a run of embeddings is a run USearch indexes without conversion.
types.h:255 ↗
enum uform_scalar_t

Sub-byte types pack several dimensions per byte, as their xN suffixes spell.

Values

uform_scalar_unknown_k0No scalar named - an unset field, refused rather than written at the wrong width.
uform_scalar_f64_k1IEEE-754 binary64.
uform_scalar_f32_k2IEEE-754 binary32.
uform_scalar_bf16_k3Truncated f32 with an 8-bit exponent and a 7-bit mantissa.
uform_scalar_f16_k4IEEE-754 binary16.
uform_scalar_e5m2_k5FP8 with a 5-bit exponent: ±57344, with infinities and NaNs.
uform_scalar_e4m3_k6FP8 with a 4-bit exponent: ±448, no infinities, and two NaN codes.
uform_scalar_e3m2_k7FP6 with a 3-bit exponent, byte-aligned with two padding bits.
uform_scalar_e2m3_k8FP6 with a 2-bit exponent, byte-aligned with two padding bits.
uform_scalar_i8_k108-bit signed integer.
uform_scalar_i4x2_k11Two signed nibbles per byte, even dimensions in the low nibble.
uform_scalar_i32_k1232-bit signed integer.
uform_scalar_u8_k208-bit unsigned integer - the element of UTF-8 text and of encoded files.
uform_scalar_u4x2_k21Two unsigned nibbles per byte, even dimensions in the low nibble.
uform_scalar_u1x8_k22Eight single-bit dimensions per byte, the sign of each embedding dimension - the form USearch's Hamming and Jaccard metrics score.

Spellings

char const * uform_scalar_name(uform_scalar_t scalar);
uform_scalar_t uform_scalar_from_name(uform_bytes_view_t name);
uform_rows_view_tA run of rows along a call's batch axis - written packed into an arena as an output, one row per item, field for field USearch's usearch_rows_view_t, so pooled embeddings feed an index as they are.
types.h:311 ↗
struct uform_rows_view_t

The call writes one row's bytes as stride, and columns and scalar as it wrote them.

Fields

datavoid const *Base pointer of the first row.
stridesize_tBytes between the starts of consecutive rows.
columnssize_tValues per row, the model's embedding dimensions.
scalaruform_scalar_tThe scalar type the rows hold.
uform_tape_view_tA borrowed ragged batch as Apache Arrow lays one out: elements end to end, and count + 1 offsets around them.
types.h:335 ↗
struct uform_tape_view_t

Entry i spans elements offsets[i] up to offsets[i+1]. What an element is belongs to the verb: a byte of an encoded file or of UTF-8 text, a byte of an RGB pixel row, a sample of a recording. The data may live on the executor's device or in host memory; the offsets always live in host memory, since they drive host-side dispatch.

Fields

datavoid const *First element of the tape, which every offset counts from.
offsetssize_t const *The call's count + 1 non-decreasing offsets, in elements.
scalaruform_scalar_tThe scalar type the elements hold: u8 for text and encoded files, f32 for decoded samples.
uform_rows_tape_view_tA ragged run of rows a call wrote into its arena, for outputs with as many rows per item as the item had tokens left after truncation: rows end to end, and one more offset than items.
types.h:355 ↗
struct uform_rows_tape_view_t

Item i spans rows offsets[i] up to offsets[i+1], so the last offset is the total.

Fields

rowsuform_rows_view_tEvery item's rows, end to end, packed.
offsetssize_t const *The call's count + 1 non-decreasing offsets into rows, in rows.
uform_text_encode_stats_tWhat a text call reports beside its outputs.
encode.h:46 ↗
struct uform_text_encode_stats_t

Fields

truncatedsize_tItems cut at the model's context.
tokenssize_tTokens the call encoded, after truncation.
uform_image_encode_stats_tWhat an image call reports beside its outputs.
encode.h:156 ↗
struct uform_image_encode_stats_t

Fields

truncatedsize_tItems whose patch grid was cut at the model's context.
tokenssize_tPatches the call encoded, after truncation.
uform_video_encode_stats_tWhat a video call reports beside its outputs.
encode.h:269 ↗
struct uform_video_encode_stats_t

Fields

truncatedsize_tItems cut at the model's context in tubes.
tokenssize_tTubes the call encoded, after truncation.
uform_audio_encode_stats_tWhat an audio call reports beside its outputs.
encode.h:327 ↗
struct uform_audio_encode_stats_t

Fields

truncatedsize_tItems cut at the model's context in frames.
tokenssize_tFrames the call encoded, after truncation.

Functions

uform_text_encodeEncodes count texts through one forward, keeping the surfaces the caller names.
encode.h:103 ↗
void uform_text_encode(uform_text_encode_t *arguments);

Parameters

modeluform_model_tThe model to encode with.
executoruform_executor_tThe executor the call runs on; it must share the model's residency.
outcomeuform_outcome_t *Output for the outcome, final once its status leaves uform_status_deferred_k.
completionuform_completion_tWhether the call blocks or returns once queued; zero blocks.
countsize_tItems in texts, and the batch axis of every output.
textsuform_tape_view_tUTF-8 texts, one item per entry, in bytes.
embeddingsuform_rows_view_t *The optional output for one pooled embedding of the model's dimensions per item, inside arena; NULL skips the pool and the head.
statesuform_rows_tape_view_t *The optional output for the final-norm states, one row of the model's dimensions per token left after truncation, inside arena; NULL skips them.
token_vectorsuform_rows_tape_view_t *The optional output for the token head's unit vectors, one row of the model's token_width per token left after truncation, the prefix included, inside arena; a model without a token head reports uform_status_unsupported_k.
arenauform_arena_tMemory for every output, rewound by the call.
outputs_scalaruform_scalar_tThe scalar type every output lands as.
progressuform_progress_tThe optional handler, called with items encoded.
statsuform_text_encode_stats_t *The optional report; NULL skips it.
uform_image_encodeDecodes and encodes count image files; an undecodable file fails the call with uform_status_malformed_k.
encode.h:211 ↗
void uform_image_encode(uform_image_encode_t *arguments);

Parameters

modeluform_model_tThe model to encode with.
executoruform_executor_tThe executor the call runs on; it must share the model's residency.
outcomeuform_outcome_t *Output for the outcome, final once its status leaves uform_status_deferred_k.
completionuform_completion_tWhether the call blocks or returns once queued; zero blocks.
countsize_tItems in images, and the batch axis of every output.
imagesuform_tape_view_tJPEG, PNG or WebP files, one item per entry, in bytes.
embeddingsuform_rows_view_t *The optional output for one pooled embedding per item, inside arena.
statesuform_rows_tape_view_t *The optional output for the final-norm states, one row per patch, inside arena.
token_vectorsuform_rows_tape_view_t *The optional output for the token head's unit vectors, one row per patch, inside arena.
arenauform_arena_tMemory for every output, rewound by the call.
outputs_scalaruform_scalar_tThe scalar type every output lands as.
progressuform_progress_tThe optional handler, called with items encoded.
statsuform_image_encode_stats_t *The optional report; NULL skips it.
uform_image_encode_pixelsEncodes count decoded images, resizing on the executor's side; a tape entry whose length isn't heights[i] × widths[i] × 3 fails with uform_status_shape_mismatch_k.
encode.h:266 ↗
void uform_image_encode_pixels(uform_image_encode_pixels_t *arguments);

Parameters

modeluform_model_tThe model to encode with.
executoruform_executor_tThe executor the call runs on; it must share the model's residency.
outcomeuform_outcome_t *Output for the outcome, final once its status leaves uform_status_deferred_k.
completionuform_completion_tWhether the call blocks or returns once queued; zero blocks.
countsize_tItems in pixels, and the batch axis of every output.
pixelsuform_tape_view_tInterleaved 8-bit RGB, one item per entry, in bytes: item i holds heights[i] rows of widths[i] pixels, three bytes each, rows end to end.
heightsuint32_t const *Rows of each item, count of them in host memory.
widthsuint32_t const *Pixels per row of each item, count of them in host memory.
embeddingsuform_rows_view_t *The optional output for one pooled embedding per item, inside arena.
statesuform_rows_tape_view_t *The optional output for the final-norm states, one row per patch, inside arena.
token_vectorsuform_rows_tape_view_t *The optional output for the token head's unit vectors, one row per patch, inside arena.
arenauform_arena_tMemory for every output, rewound by the call.
outputs_scalaruform_scalar_tThe scalar type every output lands as.
progressuform_progress_tThe optional handler, called with items encoded.
statsuform_image_encode_stats_t *The optional report; NULL skips it.
uform_video_encodeDecodes and encodes count clips.
encode.h:324 ↗
void uform_video_encode(uform_video_encode_t *arguments);

Parameters

modeluform_model_tThe model to encode with.
executoruform_executor_tThe executor the call runs on; it must share the model's residency.
outcomeuform_outcome_t *Output for the outcome, final once its status leaves uform_status_deferred_k.
completionuform_completion_tWhether the call blocks or returns once queued; zero blocks.
countsize_tItems in videos, and the batch axis of every output.
videosuform_tape_view_tMP4 or WebM files, one item per entry, in bytes, demuxed on the host and decoded by NVDEC.
embeddingsuform_rows_view_t *The optional output for one pooled embedding per item, inside arena.
statesuform_rows_tape_view_t *The optional output for the final-norm states, one row per tube, inside arena.
token_vectorsuform_rows_tape_view_t *The optional output for the token head's unit vectors, one row per tube, inside arena.
arenauform_arena_tMemory for every output, rewound by the call.
outputs_scalaruform_scalar_tThe scalar type every output lands as.
progressuform_progress_tThe optional handler, called with items encoded.
statsuform_video_encode_stats_t *The optional report; NULL skips it.
uform_audio_encodeDecodes and encodes count recordings.
encode.h:382 ↗
void uform_audio_encode(uform_audio_encode_t *arguments);

Parameters

modeluform_model_tThe model to encode with.
executoruform_executor_tThe executor the call runs on; it must share the model's residency.
outcomeuform_outcome_t *Output for the outcome, final once its status leaves uform_status_deferred_k.
completionuform_completion_tWhether the call blocks or returns once queued; zero blocks.
countsize_tItems in recordings, and the batch axis of every output.
recordingsuform_tape_view_tWAV, FLAC, MP3 or Opus files, one item per entry, in bytes, decoded and resampled to mono on the host.
embeddingsuform_rows_view_t *The optional output for one pooled embedding per item, inside arena.
statesuform_rows_tape_view_t *The optional output for the final-norm states, one row per frame, inside arena.
token_vectorsuform_rows_tape_view_t *The optional output for the token head's unit vectors, one row per frame, inside arena.
arenauform_arena_tMemory for every output, rewound by the call.
outputs_scalaruform_scalar_tThe scalar type every output lands as.
progressuform_progress_tThe optional handler, called with items encoded.
statsuform_audio_encode_stats_t *The optional report; NULL skips it.
uform_audio_encode_samplesEncodes count decoded recordings.
encode.h:432 ↗
void uform_audio_encode_samples(
    uform_audio_encode_samples_t *arguments);

Parameters

modeluform_model_tThe model to encode with.
executoruform_executor_tThe executor the call runs on; it must share the model's residency.
outcomeuform_outcome_t *Output for the outcome, final once its status leaves uform_status_deferred_k.
completionuform_completion_tWhether the call blocks or returns once queued; zero blocks.
countsize_tItems in samples, and the batch axis of every output.
samplesuform_tape_view_tMono samples, one item per entry, in samples, of uform_scalar_f32_k.
sample_rateuint32_tSamples per second shared by every item; resampled to the model's rate where it differs.
embeddingsuform_rows_view_t *The optional output for one pooled embedding per item, inside arena.
statesuform_rows_tape_view_t *The optional output for the final-norm states, one row per frame, inside arena.
token_vectorsuform_rows_tape_view_t *The optional output for the token head's unit vectors, one row per frame, inside arena.
arenauform_arena_tMemory for every output, rewound by the call.
outputs_scalaruform_scalar_tThe scalar type every output lands as.
progressuform_progress_tThe optional handler, called with items encoded.
statsuform_audio_encode_stats_t *The optional report; NULL skips it.