Mojo · Encoding

Encoding

from uform import Outputs, EncodeStats, Encoded, … 4.0.0-rc.1 · 3ee5e534

Every encode verb takes its inputs as tapes, elements end to end with one more offset than items, then the Outputs to keep, the arena, and an optional EncodeConfig naming the scalar outputs land as and a report slot:

var stats = EncodeStats()
var config = EncodeConfig(outputs_scalar=ScalarKind.BF16, stats=Pointer(to=stats))
var images = model.image_encode(files, Outputs.EMBEDDINGS | Outputs.TOKEN_VECTORS, arena, config)
var pixels = model.image_encode_pixels(rgb, heights, widths, Outputs.EMBEDDINGS, arena)
var clips = model.video_encode(videos, Outputs.EMBEDDINGS, arena)
var sounds = model.audio_encode_samples(samples, 16000, Outputs.EMBEDDINGS, arena)
var probabilities = model.text_choose(states, choices, arena)  # one F32 per legal byte

API

StructOutputsThe surfaces an encode verb keeps from its one forward, named after the fields they fill. Binding-only; it decides which output pointers a verb passes.2 
struct Outputs

Fields

codeUInt32The surfaces kept, one flag each.

Values

EMBEDDINGSOutputs(UInt32(1))One pooled embedding per item, past the attention pool and the projection head.
STATESOutputs(UInt32(2))One final-norm state per token left after truncation.
TOKEN_VECTORSOutputs(UInt32(4))One unit vector of the token head per token left after truncation; a model without a token head fails the call with UNSUPPORTED.

Conforms to

Copyable, Deinitable, Equatable, ImplicitlyCopyable, Movable, RegisterPassable, TrivialRegisterPassable
__or__Both sets of surfaces. 
def __or__(self, other: Self) -> Self
containsWhether every surface other names is kept. 
def contains(self, other: Self) -> Bool
StructEncodeStatsWhat an encode call reports beside its outputs: uform_text_encode_stats_t, and the image, video and audio reports that share its two fields.1 
struct EncodeStats

Fields

truncatedc_size_tItems cut at the model's context.
tokensc_size_tTokens the call encoded after truncation: bytes, patches, tubes or frames.

Conforms to

Copyable, Defaultable, Deinitable, ImplicitlyCopyable, Movable
__init__Zeroes both fields, which is what a failed call leaves. 
def __init__(out self)
StructEncodedWhat an encode verb wrote into the arena it borrows: the surfaces its Outputs named. Binding-only.3 
struct Encoded[origin: MutOrigin]

Parameters

originMutOrigincomptimeThe arena the surfaces live in, which this result borrows.

Fields

arenaPointer[Arena, origin]The arena holding the surfaces.
generationIntThe arena's call these surfaces came from, so a read after a later call is refused.
keptOutputsThe surfaces the call kept.
writtenEncodedArgumentsThe views the engine wrote, inside the arena.
countIntItems the call encoded.

Conforms to

Deinitable, Movable
embeddingsOne pooled embedding of the model's dimensions per item, without normalization; absent unless kept. 
def embeddings(self) -> Optional[Rows[origin]]
statesOne final-norm state per token left after truncation, item by item; absent unless kept. 
def states(self) -> Optional[RowsTape[origin]]
token_vectorsOne unit vector of the model's token width per token left after truncation, item by item; absent unless kept. 
def token_vectors(self) -> Optional[RowsTape[origin]]
StructEncodeConfigThe knobs every encode verb shares, the verbs over texts, images, clips and recordings alike.1 
struct EncodeConfig[stats_origin: MutOrigin = MutUntrackedOrigin]

Parameters

stats_originMutOrigincomptimeThe lent report slot's origin, inferred where one is lent.

Fields

outputs_scalarScalarKindThe scalar type every output lands as.
progressProgressReports the items encoded, and stops the call where it says to.
statsOptional[Pointer[EncodeStats, stats_origin]]Receives the truncation report, overwritten by the call; absent skips it.

Conforms to

Copyable, Deinitable, ImplicitlyCopyable, Movable
__init__Names only the knobs that differ from the defaults. 
def __init__(
    out self,
    *,
    outputs_scalar: ScalarKind = ScalarKind.F32,
    progress: Progress = Progress(),
    stats: Optional[Pointer[EncodeStats, stats_origin]] = Optional()
)
StructScalarKindThe scalar types a model computes in and its outputs land as, uform_scalar_t named by width.3 
struct ScalarKind

Named ScalarKind because Mojo's own Scalar is the element type it describes. The xN suffixes spell how many logical dimensions share one byte. Library.scalar_name spells each one as the engine does.

Fields

codec_intThe raw uform_scalar_t value the ABI exchanges.

Values

UNKNOWNScalarKind(Int32(0))No scalar named, an unset field refused rather than written at the wrong width.
F64ScalarKind(Int32(1))IEEE-754 binary64.
F32ScalarKind(Int32(2))IEEE-754 binary32.
BF16ScalarKind(Int32(3))Truncated f32 with an 8-bit exponent and a 7-bit mantissa.
F16ScalarKind(Int32(4))IEEE-754 binary16.
E5M2ScalarKind(Int32(5))FP8 with a 5-bit exponent, with infinities and NaNs.
E4M3ScalarKind(Int32(6))FP8 with a 4-bit exponent, no infinities.
E3M2ScalarKind(Int32(7))FP6 with a 3-bit exponent, byte-aligned with two padding bits.
E2M3ScalarKind(Int32(8))FP6 with a 2-bit exponent, byte-aligned with two padding bits.
I8ScalarKind(Int32(10))8-bit signed integer.
I4X2ScalarKind(Int32(11))Two signed nibbles per byte, even dimensions in the low nibble.
I32ScalarKind(Int32(12))32-bit signed integer.
U8ScalarKind(Int32(20))8-bit unsigned integer, the element of UTF-8 text and of encoded files.
U4X2ScalarKind(Int32(21))Two unsigned nibbles per byte, even dimensions in the low nibble.
U1X8ScalarKind(Int32(22))Eight single-bit dimensions per byte, the sign of each embedding dimension.

Conforms to

Copyable, Deinitable, Equatable, ImplicitlyCopyable, Movable, RegisterPassable, TrivialRegisterPassable
ofThe kind a Mojo scalar type stores one-to-one; the packed and FP6 kinds have none. Binding-only. 
def of[dtype: DType]() -> Self
bits_per_scalarHow many bits one logical dimension occupies, which sub-byte kinds share; zero for UNKNOWN. 
def bits_per_scalar(self) -> Int
storesWhether values of this kind read as whole dtype elements: equal widths, or bytes for packed kinds. Binding-only. 
def stores[dtype: DType](self) -> Bool
StructBytesViewA borrowed run of bytes and its length, never null-terminated: every path, buffer and name.2 
struct BytesView[origin: ImmOrigin = ImmUntrackedOrigin]

Parameters

originImmOrigincomptimeThe borrowed storage, which the view keeps alive for as long as it is used.

Fields

dataPointer[UInt8, origin]The first byte.
lengthIntHow many bytes the run covers, since nothing terminates it.

Conforms to

Copyable, Deinitable, ImplicitlyCopyable, Movable
ofBorrows a string's bytes, with no terminator manufactured. 
def of[text_origin: ImmOrigin, //](text: StringSpan[text_origin]) -> BytesView[text_origin]
def of[bytes_origin: ImmOrigin, //](bytes: Span[UInt8, bytes_origin]) -> BytesView[bytes_origin]

Borrows a run of raw bytes, such as a checkpoint held in memory.

*× 2The C layout, erased only inside a verb so the borrow lives through the call. 
def arguments(self) -> BytesViewArguments
StructTapeViewA borrowed ragged batch laid out as Apache Arrow lays one out: elements end to end, and one more offset than entries around them.2 
struct TapeView[origin: ImmOrigin = ImmUntrackedOrigin]

Entry i spans elements offsets[i] up to offsets[i + 1]. The elements may live in host or device memory; the offsets always live in host memory, since they drive host-side dispatch.

Parameters

originImmOrigincomptimeThe borrowed elements and offsets, which the view keeps alive for as long as it is used.

Fields

dataPointer[NoneType, origin]The first element, which every offset counts from.
offsetsPointer[UInt, origin]The entries' boundaries, one more of them than entries.
countIntEntries in the tape, which the verb takes as its count.
scalarScalarKindThe scalar type the elements hold: U8 for text and encoded files, F32 for decoded samples.

Conforms to

Copyable, Deinitable, ImplicitlyCopyable, Movable, Sized
__len__The entries in the tape. 
def __len__(self) -> Int
ofReads host elements and their offsets as a tape, refusing offsets that end past the elements. 
def of[dtype: DType, data_origin: ImmOrigin, offsets_origin: ImmOrigin, //](
    data: Span[Scalar[dtype], data_origin],
    offsets: Span[UInt, offsets_origin],
    scalar: ScalarKind = ScalarKind.of[dtype]()
) -> TapeView[origin_of(data_origin, offsets_origin)]
def of[dtype: DType, data_origin: ImmOrigin, offsets_origin: ImmOrigin, //](
    data: Pointer[Scalar[dtype], data_origin],
    offsets: Span[UInt, offsets_origin],
    scalar: ScalarKind = ScalarKind.of[dtype]()
) -> TapeView[origin_of(data_origin, offsets_origin)]

Reads elements behind a host or device pointer, such as buffer.unsafe_ptr() of a MAX DeviceBuffer, with their offsets in host memory.

Raises

Error with SHAPE_MISMATCH when no offset is given or the last one exceeds len(data).
StructRowsRows a call wrote into the arena it borrows, one per item, packed, uform_rows_view_t read in place.6 
struct Rows[origin: MutOrigin]

Parameters

originMutOrigincomptimeThe arena the rows live in, which this result borrows.

Fields

arenaPointer[Arena, origin]The arena holding the rows.
generationIntThe arena's call these rows came from, so a read after a later call is refused.
viewRowsViewArgumentsThe first row, the stride, the width and the kind, as the engine wrote them.
countIntRows written.

Conforms to

Copyable, Deinitable, ImplicitlyCopyable, Movable, Sized
__len__The rows written. 
def __len__(self) -> Int
columnsValues per row, the model's embedding dimensions or token width. 
def columns(self) -> Int
strideBytes between the starts of consecutive rows. 
def stride(self) -> Int
scalarThe scalar type the rows hold. 
def scalar(self) -> ScalarKind
rowOne row's values, or its packed bytes for a sub-byte kind, borrowed from the arena. 
def row[dtype: DType](ref self, row: Int) -> Span[Scalar[dtype], origin_of(self)]

Raises

Error with DEVICE over device memory, INVALID_ARGUMENT once a later call rewound the arena, or when the rows hold a kind dtype does not store.
valuesEvery row's values end to end, refused for padded rows. 
def values[dtype: DType](ref self) -> Span[Scalar[dtype], origin_of(self)]

Raises

Error with DEVICE over device memory, INVALID_ARGUMENT once a later call rewound the arena or for another kind, and with UNSUPPORTED_LAYOUT when the rows are padded.
StructRowsTapeA ragged run of rows a call wrote into the arena it borrows, as many per item as the item had tokens left after truncation, uform_rows_tape_view_t read in place.4 
struct RowsTape[origin: MutOrigin]

Parameters

originMutOrigincomptimeThe arena the rows live in, which this result borrows.

Fields

arenaPointer[Arena, origin]The arena holding the rows.
generationIntThe arena's call these rows came from, so a read after a later call is refused.
viewRowsTapeViewArgumentsThe rows and their offsets, as the engine wrote them.
countIntItems the call wrote rows for.

Conforms to

Copyable, Deinitable, ImplicitlyCopyable, Movable, Sized
__len__The items the call wrote rows for. 
def __len__(self) -> Int
offsetsThe items' boundaries in rows, one more of them than items. 
def offsets(ref self) -> Span[UInt, origin_of(self)]

Raises

Error with INVALID_ARGUMENT once a later call rewound the arena, or DEVICE over device memory.
rowsEvery item's rows end to end, as one run. 
def rows(self) -> Rows[origin]

Raises

Error with INVALID_ARGUMENT once a later call rewound the arena, or DEVICE over device memory.
itemOne item's rows. 
def item(self, item: Int) -> Rows[origin]

Raises

Error with INVALID_ARGUMENT once a later call rewound the arena, or DEVICE over device memory.
StructTapeA ragged run of values a call wrote into the arena it borrows, uform_tape_view_t read in place.4 
struct Tape[origin: MutOrigin]

Parameters

originMutOrigincomptimeThe arena the values live in, which this result borrows.

Fields

arenaPointer[Arena, origin]The arena holding the values.
generationIntThe arena's call these values came from, so a read after a later call is refused.
viewTapeViewArgumentsThe values and their offsets, as the engine wrote them.
countIntItems the call wrote values for.

Conforms to

Copyable, Deinitable, ImplicitlyCopyable, Movable, Sized
__len__The items the call wrote values for. 
def __len__(self) -> Int
scalarThe scalar type the values hold. 
def scalar(self) -> ScalarKind
offsetsThe items' boundaries in values, one more of them than items. 
def offsets(ref self) -> Span[UInt, origin_of(self)]

Raises

Error with INVALID_ARGUMENT once a later call rewound the arena, or DEVICE over device memory.
itemOne item's values. 
def item[dtype: DType](ref self, item: Int) -> Span[Scalar[dtype], origin_of(self)]

Raises

Error with DEVICE over device memory, INVALID_ARGUMENT once a later call rewound the arena, or when the values hold a kind dtype does not store.