Models | UForm for Mojo

API

StructModelDescriptionWhat a loaded model reports about itself, uform_model_description_t.1

struct ModelDescription

Fields

`dimensions`	`c_size_t`	Scalars per embedding row; Matryoshka prefixes of it are embeddings too.
`context`	`c_size_t`	Tokens one item may span before it is truncated: bytes of text, patches, tubes or frames.
`quantization`	`ScalarKind`	The scalar the transformer blocks run in.
`memory_usage`	`c_size_t`	Bytes the model holds; activations are the executor's.
`cpu_class`	`CPUClass`	The class a RAM model's tower was packed for; `UNKNOWN` for a VRAM model.
`gpu_class`	`GPUClass`	The class every replica of a VRAM model runs as; `UNKNOWN` for a RAM model.
`token_width`	`c_size_t`	Scalars per token vector; zero when the checkpoint carries no token head.
`choice_temperature_groups`	`c_size_t`	Question types the choice temperatures were calibrated over; zero without the byte decoder.

Conforms to

Copyable, Defaultable, Deinitable, ImplicitlyCopyable, Movable

__init__Zeroes every field, which is what a failed report leaves.

def __init__(out self)

StructModelA loaded encoder, released when it goes out of scope.8

struct Model

Verbs called on the model run on the calling thread; model.on(executor) runs the same verbs blocking on an executor, and model.deferred(executor) submits them there. One model serves any number of executors of its residency, whose calls take turns on its session.

Fields

`library`	`Library`	The loaded engine, held so its symbol table outlives every call made through the handle.
`address`	`Address`	The opaque `uform_model_t`.

Conforms to

Deinitable, ModelVerbs, Movable

__init__Adopts a handle the engine just created.

def __init__(out self, *, library: Library, address: Int)

__deinit__Frees the model; every call using it must be final. Calls uform_model_free.

def __deinit__(deinit self)

The loader reference is surrendered only after the call, so Mojo cannot dlclose the engine between the load of the entry point and the branch to it.

load_ramCopies a checkpoint file into host memory and packs its products for the CPU. Calls uform_model_load_ram.

def load_ram[path_origin: ImmOrigin, ram_origin: ImmOrigin, //](
    executor: Executor,
    path: StringSpan[path_origin],
    ram: RamOptions[ram_origin] = RamOptions(cpu_class=CPUClass.UNKNOWN),
    quantization: ScalarKind = ScalarKind.BF16,
    progress: Progress = Progress()
) -> Self

Parameters

`executorExecutor`		The executor the load runs on; it also lends its engine.
`pathStringSpan[path_origin]`		A safetensors checkpoint, read in full and closed before the load completes.
`ramRamOptions[ram_origin]`		The memory domains to place the weights in.
`quantizationScalarKind`		`BF16` as the checkpoint stores it, or `E4M3` quantized at load.
`progressProgress`		Reports the tensors placed, and stops the load where it says to.

Raises

Error if the checkpoint cannot be read or decoded.

load_ram_bufferCopies a checkpoint held in memory into host memory and packs its products for the CPU. Calls uform_model_load_ram_buffer.

def load_ram_buffer[buffer_origin: ImmOrigin, ram_origin: ImmOrigin, //](
    executor: Executor,
    buffer: Span[UInt8, buffer_origin],
    ram: RamOptions[ram_origin] = RamOptions(cpu_class=CPUClass.UNKNOWN),
    quantization: ScalarKind = ScalarKind.BF16,
    progress: Progress = Progress()
) -> Self

Raises

Error if the bytes cannot be decoded.

load_vramCopies a checkpoint file onto the named GPUs, replicated on each, and packs its products there. Calls uform_model_load_vram.

def load_vram[path_origin: ImmOrigin, vram_origin: ImmOrigin, //](
    executor: Executor,
    path: StringSpan[path_origin],
    vram: VramOptions[vram_origin] = VramOptions(gpu_class=GPUClass.UNKNOWN),
    quantization: ScalarKind = ScalarKind.BF16,
    progress: Progress = Progress()
) -> Self

Raises

Error if the checkpoint cannot be read, or with UNSUPPORTED in a build without a GPU.

load_vram_bufferCopies a checkpoint held in host or device memory onto the named GPUs and packs its products there. Calls uform_model_load_vram_buffer.

def load_vram_buffer[buffer_origin: ImmOrigin, vram_origin: ImmOrigin, //](
    executor: Executor,
    buffer: Span[UInt8, buffer_origin],
    vram: VramOptions[vram_origin] = VramOptions(gpu_class=GPUClass.UNKNOWN),
    quantization: ScalarKind = ScalarKind.BF16,
    progress: Progress = Progress()
) -> Self

Raises

Error if the bytes cannot be decoded, or with UNSUPPORTED in a build without a GPU.

onThe same verbs, blocking on executor after every call submitted there before them. Binding-only.

def on(ref self, ref executor: Executor) -> ModelOn[origin_of(executor, self)]

Returns

A receiver borrowing both, which neither may outlive.

deferredThe same verbs, each submitted to executor and answering a Deferred at once. Binding-only.

def deferred(ref self, ref executor: Executor) -> ModelDeferredOn[origin_of(executor, self)]

Returns

A receiver borrowing both, which neither it nor any Deferred it answers may outlive.

TraitModelVerbsThe verbs every blocking receiver offers, the model on the calling thread and model.on(executor) on an executor, each written once over the receiver's _route.8

trait ModelVerbs

describeReports the model's geometry and placement in one call. Calls uform_model_describe.

def describe(self) -> ModelDescription

text_encodeEncodes UTF-8 texts through one forward, a token per byte, keeping the surfaces outputs names. Calls uform_text_encode.

def text_encode[texts_origin: ImmOrigin, stats_origin: MutOrigin, //](
    self,
    texts: TapeView[texts_origin],
    outputs: Outputs,
    mut arena: Arena,
    config: EncodeConfig[stats_origin] = EncodeConfig(outputs_scalar=ScalarKind.F32, progress=Progress(), stats=Optional())
) -> Encoded[origin_of(arena)]

Parameters

`textsTapeView[texts_origin]`		UTF-8 texts, one item per entry, in bytes.
`outputsOutputs`		The surfaces to keep from the forward.
`arenaArena`	mut	The arena to rewind and refill, which the result borrows.
`configEncodeConfig[stats_origin]`		The scalar every output lands as, and the report slot.

Raises

Error if the shapes disagree or the model lacks a kept surface.

text_chooseScores each item's legal choices through one forward, a softmax over them alone at the item's group's calibrated inverse temperature, answering one F32 probability per choice. Calls uform_text_choose.

def text_choose[texts_origin: ImmOrigin, choices_origin: ImmOrigin, groups_origin: ImmOrigin, stats_origin: MutOrigin, //](
    self,
    texts: TapeView[texts_origin],
    choices: TapeView[choices_origin],
    mut arena: Arena,
    config: TextChooseConfig[groups_origin, stats_origin] = TextChooseConfig(temperature_groups=Span(), progress=Progress(), stats=Optional())
) -> Tape[origin_of(arena)]

Parameters

`textsTapeView[texts_origin]`		UTF-8 states, one per item, each ending where its choice goes.
`choicesTapeView[choices_origin]`		Each item's legal choices, one byte each.
`arenaArena`	mut	The arena to rewind and refill, which the result borrows.
`configTextChooseConfig[groups_origin, stats_origin]`		The question type of each item, and the report slot.

Raises

Error with UNSUPPORTED for a model without the byte decoder.

image_encodeDecodes and encodes JPEG, PNG or WebP files, one item per entry. Calls uform_image_encode.

def image_encode[images_origin: ImmOrigin, stats_origin: MutOrigin, //](
    self,
    images: TapeView[images_origin],
    outputs: Outputs,
    mut arena: Arena,
    config: EncodeConfig[stats_origin] = EncodeConfig(outputs_scalar=ScalarKind.F32, progress=Progress(), stats=Optional())
) -> Encoded[origin_of(arena)]

Raises

Error with MALFORMED for an undecodable file.

image_encode_pixelsEncodes decoded interleaved 8-bit RGB images, resizing on the executor's side. Calls uform_image_encode_pixels.

def image_encode_pixels[pixels_origin: ImmOrigin, heights_origin: ImmOrigin, widths_origin: ImmOrigin, stats_origin: MutOrigin, //](
    self,
    pixels: TapeView[pixels_origin],
    heights: Span[UInt32, heights_origin],
    widths: Span[UInt32, widths_origin],
    outputs: Outputs,
    mut arena: Arena,
    config: EncodeConfig[stats_origin] = EncodeConfig(outputs_scalar=ScalarKind.F32, progress=Progress(), stats=Optional())
) -> Encoded[origin_of(arena)]

Raises

Error with SHAPE_MISMATCH when an entry is not heights[i] × widths[i] × 3 bytes.

video_encodeDecodes and encodes MP4 or WebM clips, one item per entry. Calls uform_video_encode.

def video_encode[videos_origin: ImmOrigin, stats_origin: MutOrigin, //](
    self,
    videos: TapeView[videos_origin],
    outputs: Outputs,
    mut arena: Arena,
    config: EncodeConfig[stats_origin] = EncodeConfig(outputs_scalar=ScalarKind.F32, progress=Progress(), stats=Optional())
) -> Encoded[origin_of(arena)]

Raises

Error with MALFORMED for an undecodable clip.

audio_encodeDecodes and encodes WAV, FLAC, MP3 or Opus recordings, one item per entry. Calls uform_audio_encode.

def audio_encode[recordings_origin: ImmOrigin, stats_origin: MutOrigin, //](
    self,
    recordings: TapeView[recordings_origin],
    outputs: Outputs,
    mut arena: Arena,
    config: EncodeConfig[stats_origin] = EncodeConfig(outputs_scalar=ScalarKind.F32, progress=Progress(), stats=Optional())
) -> Encoded[origin_of(arena)]

Raises

Error with MALFORMED for an undecodable recording.

audio_encode_samplesEncodes decoded mono F32 recordings sharing one sample_rate, resampled to the model's. Calls uform_audio_encode_samples.

def audio_encode_samples[samples_origin: ImmOrigin, stats_origin: MutOrigin, //](
    self,
    samples: TapeView[samples_origin],
    sample_rate: Int,
    outputs: Outputs,
    mut arena: Arena,
    config: EncodeConfig[stats_origin] = EncodeConfig(outputs_scalar=ScalarKind.F32, progress=Progress(), stats=Optional())
) -> Encoded[origin_of(arena)]

Raises

Error if the shapes disagree.

StructResidencyWhere an arena's outputs live, as Arena.describe reports it, uform_residency_t.

struct Residency

Fields

`code`	`c_int`	The raw `uform_residency_t` value the ABI exchanges.

Values

`RAM`	`Residency(Int32(1))`	Host pages, which the host reads directly.
`VRAM`	`Residency(Int32(2))`	One GPU's memory, which only device code reads.

Conforms to

Copyable, Deinitable, Equatable, ImplicitlyCopyable, Movable, RegisterPassable, TrivialRegisterPassable