# Models

> UForm for Mojo

`from uform import ModelDescription, Model, ModelVerbs, …`

## ModelDescription

What a loaded model reports about itself, `uform_model_description_t`.

```mojo
struct ModelDescription
```

- `dimensions` `c_size_t`: Scalars per embedding row; Matryoshka prefixes of it are embeddings too.
- `context` `c_size_t`: Tokens one item may span before it is truncated: bytes of text, patches, tubes or frames.
- `quantization` `ScalarKind`: The scalar the transformer blocks run in.
- `memory_usage` `c_size_t`: Bytes the model holds; activations are the executor's.
- `cpu_class` `CPUClass`: The class a RAM model's tower was packed for; [`UNKNOWN`](https://www.unum.cloud/docs/uform/mojo/encoding.md#UNKNOWN) for a VRAM model.
- `gpu_class` `GPUClass`: The class every replica of a VRAM model runs as; [`UNKNOWN`](https://www.unum.cloud/docs/uform/mojo/encoding.md#UNKNOWN) for a RAM model.
- `token_width` `c_size_t`: Scalars per token vector; zero when the checkpoint carries no token head.
- `choice_temperature_groups` `c_size_t`: Question types the choice temperatures were calibrated over; zero without the byte decoder.

Conforms to: `Copyable`, `Defaultable`, `Deinitable`, `ImplicitlyCopyable`, `Movable`

## __init__

Zeroes every field, which is what a failed report leaves.

```mojo
def __init__(out self)
```

## Model

A loaded encoder, released when it goes out of scope.

```mojo
struct Model
```

Verbs called on the model run on the calling thread; `model.on(executor)` runs the same verbs
blocking on an executor, and `model.deferred(executor)` submits them there. One model serves any
number of executors of its residency, whose calls take turns on its session.

- `library` `Library`: The loaded engine, held so its symbol table outlives every call made through the handle.
- `address` `Address`: The opaque `uform_model_t`.

Conforms to: `Deinitable`, `[ModelVerbs](#ModelVerbs)`, `Movable`

## __init__

Adopts a handle the engine just created.

```mojo
def __init__(out self, *, library: Library, address: Int)
```

## __deinit__

Frees the model; every call using it must be final. Calls `uform_model_free`.

```mojo
def __deinit__(deinit self)
```

The loader reference is surrendered only after the call, so Mojo cannot `dlclose` the engine
between the load of the entry point and the branch to it.

## load_ram

Copies a checkpoint file into host memory and packs its products for the CPU. Calls `uform_model_load_ram`.

```mojo
def load_ram[path_origin: ImmOrigin, ram_origin: ImmOrigin, //](
    executor: Executor,
    path: StringSpan[path_origin],
    ram: RamOptions[ram_origin] = RamOptions(cpu_class=CPUClass.UNKNOWN),
    quantization: ScalarKind = ScalarKind.BF16,
    progress: Progress = Progress()
) -> Self
```

- `executor`: The executor the load runs on; it also lends its engine.
- `path`: A safetensors checkpoint, read in full and closed before the load completes.
- `ram`: The memory domains to place the weights in.
- `quantization`: [`BF16`](https://www.unum.cloud/docs/uform/mojo/encoding.md#BF16) as the checkpoint stores it, or [`E4M3`](https://www.unum.cloud/docs/uform/mojo/encoding.md#E4M3) quantized at load.
- `progress`: Reports the tensors placed, and stops the load where it says to.

Raises: [`Error`](https://www.unum.cloud/docs/uform/mojo/errors.md#Error) if the checkpoint cannot be read or decoded.

## load_ram_buffer

Copies a checkpoint held in memory into host memory and packs its products for the CPU. Calls `uform_model_load_ram_buffer`.

```mojo
def load_ram_buffer[buffer_origin: ImmOrigin, ram_origin: ImmOrigin, //](
    executor: Executor,
    buffer: Span[UInt8, buffer_origin],
    ram: RamOptions[ram_origin] = RamOptions(cpu_class=CPUClass.UNKNOWN),
    quantization: ScalarKind = ScalarKind.BF16,
    progress: Progress = Progress()
) -> Self
```

Raises: [`Error`](https://www.unum.cloud/docs/uform/mojo/errors.md#Error) if the bytes cannot be decoded.

## load_vram

Copies a checkpoint file onto the named GPUs, replicated on each, and packs its products there. Calls `uform_model_load_vram`.

```mojo
def load_vram[path_origin: ImmOrigin, vram_origin: ImmOrigin, //](
    executor: Executor,
    path: StringSpan[path_origin],
    vram: VramOptions[vram_origin] = VramOptions(gpu_class=GPUClass.UNKNOWN),
    quantization: ScalarKind = ScalarKind.BF16,
    progress: Progress = Progress()
) -> Self
```

Raises: [`Error`](https://www.unum.cloud/docs/uform/mojo/errors.md#Error) if the checkpoint cannot be read, or with [`UNSUPPORTED`](https://www.unum.cloud/docs/uform/mojo/errors.md#UNSUPPORTED) in a build without a GPU.

## load_vram_buffer

Copies a checkpoint held in host or device memory onto the named GPUs and packs its products there. Calls `uform_model_load_vram_buffer`.

```mojo
def load_vram_buffer[buffer_origin: ImmOrigin, vram_origin: ImmOrigin, //](
    executor: Executor,
    buffer: Span[UInt8, buffer_origin],
    vram: VramOptions[vram_origin] = VramOptions(gpu_class=GPUClass.UNKNOWN),
    quantization: ScalarKind = ScalarKind.BF16,
    progress: Progress = Progress()
) -> Self
```

Raises: [`Error`](https://www.unum.cloud/docs/uform/mojo/errors.md#Error) if the bytes cannot be decoded, or with [`UNSUPPORTED`](https://www.unum.cloud/docs/uform/mojo/errors.md#UNSUPPORTED) in a build without a GPU.

## on

The same verbs, blocking on `executor` after every call submitted there before them. Binding-only.

```mojo
def on(ref self, ref executor: Executor) -> ModelOn[origin_of(executor, self)]
```

## deferred

The same verbs, each submitted to `executor` and answering a [`Deferred`](https://www.unum.cloud/docs/uform/mojo/receivers-and-deferred-calls.md#Deferred) at once. Binding-only.

```mojo
def deferred(ref self, ref executor: Executor) -> ModelDeferredOn[origin_of(executor, self)]
```

## ModelVerbs

The verbs every blocking receiver offers, the model on the calling thread and `model.on(executor)` on an executor, each written once over the receiver's `_route`.

```mojo
trait ModelVerbs
```

## describe

Reports the model's geometry and placement in one call. Calls `uform_model_describe`.

```mojo
def describe(self) -> ModelDescription
```

## text_encode

Encodes UTF-8 texts through one forward, a token per byte, keeping the surfaces `outputs` names. Calls `uform_text_encode`.

```mojo
def text_encode[texts_origin: ImmOrigin, stats_origin: MutOrigin, //](
    self,
    texts: TapeView[texts_origin],
    outputs: Outputs,
    mut arena: Arena,
    config: EncodeConfig[stats_origin] = EncodeConfig(outputs_scalar=ScalarKind.F32, progress=Progress(), stats=Optional())
) -> Encoded[origin_of(arena)]
```

- `texts`: UTF-8 texts, one item per entry, in bytes.
- `outputs`: The surfaces to keep from the forward.
- `arena` (mut): The arena to rewind and refill, which the result borrows.
- `config`: The scalar every output lands as, and the report slot.

Raises: [`Error`](https://www.unum.cloud/docs/uform/mojo/errors.md#Error) if the shapes disagree or the model lacks a kept surface.

## text_choose

Scores each item's legal choices through one forward, a softmax over them alone at the item's group's calibrated inverse temperature, answering one [`F32`](https://www.unum.cloud/docs/uform/mojo/encoding.md#F32) probability per choice. Calls `uform_text_choose`.

```mojo
def text_choose[texts_origin: ImmOrigin, choices_origin: ImmOrigin, groups_origin: ImmOrigin, stats_origin: MutOrigin, //](
    self,
    texts: TapeView[texts_origin],
    choices: TapeView[choices_origin],
    mut arena: Arena,
    config: TextChooseConfig[groups_origin, stats_origin] = TextChooseConfig(temperature_groups=Span(), progress=Progress(), stats=Optional())
) -> Tape[origin_of(arena)]
```

- `texts`: UTF-8 states, one per item, each ending where its choice goes.
- `choices`: Each item's legal choices, one byte each.
- `arena` (mut): The arena to rewind and refill, which the result borrows.
- `config`: The question type of each item, and the report slot.

Raises: [`Error`](https://www.unum.cloud/docs/uform/mojo/errors.md#Error) with [`UNSUPPORTED`](https://www.unum.cloud/docs/uform/mojo/errors.md#UNSUPPORTED) for a model without the byte decoder.

## image_encode

Decodes and encodes JPEG, PNG or WebP files, one item per entry. Calls `uform_image_encode`.

```mojo
def image_encode[images_origin: ImmOrigin, stats_origin: MutOrigin, //](
    self,
    images: TapeView[images_origin],
    outputs: Outputs,
    mut arena: Arena,
    config: EncodeConfig[stats_origin] = EncodeConfig(outputs_scalar=ScalarKind.F32, progress=Progress(), stats=Optional())
) -> Encoded[origin_of(arena)]
```

Raises: [`Error`](https://www.unum.cloud/docs/uform/mojo/errors.md#Error) with [`MALFORMED`](https://www.unum.cloud/docs/uform/mojo/errors.md#MALFORMED) for an undecodable file.

## image_encode_pixels

Encodes decoded interleaved 8-bit RGB images, resizing on the executor's side. Calls `uform_image_encode_pixels`.

```mojo
def image_encode_pixels[pixels_origin: ImmOrigin, heights_origin: ImmOrigin, widths_origin: ImmOrigin, stats_origin: MutOrigin, //](
    self,
    pixels: TapeView[pixels_origin],
    heights: Span[UInt32, heights_origin],
    widths: Span[UInt32, widths_origin],
    outputs: Outputs,
    mut arena: Arena,
    config: EncodeConfig[stats_origin] = EncodeConfig(outputs_scalar=ScalarKind.F32, progress=Progress(), stats=Optional())
) -> Encoded[origin_of(arena)]
```

Raises: [`Error`](https://www.unum.cloud/docs/uform/mojo/errors.md#Error) with [`SHAPE_MISMATCH`](https://www.unum.cloud/docs/uform/mojo/errors.md#SHAPE_MISMATCH) when an entry is not `heights[i] × widths[i] × 3` bytes.

## video_encode

Decodes and encodes MP4 or WebM clips, one item per entry. Calls `uform_video_encode`.

```mojo
def video_encode[videos_origin: ImmOrigin, stats_origin: MutOrigin, //](
    self,
    videos: TapeView[videos_origin],
    outputs: Outputs,
    mut arena: Arena,
    config: EncodeConfig[stats_origin] = EncodeConfig(outputs_scalar=ScalarKind.F32, progress=Progress(), stats=Optional())
) -> Encoded[origin_of(arena)]
```

Raises: [`Error`](https://www.unum.cloud/docs/uform/mojo/errors.md#Error) with [`MALFORMED`](https://www.unum.cloud/docs/uform/mojo/errors.md#MALFORMED) for an undecodable clip.

## audio_encode

Decodes and encodes WAV, FLAC, MP3 or Opus recordings, one item per entry. Calls `uform_audio_encode`.

```mojo
def audio_encode[recordings_origin: ImmOrigin, stats_origin: MutOrigin, //](
    self,
    recordings: TapeView[recordings_origin],
    outputs: Outputs,
    mut arena: Arena,
    config: EncodeConfig[stats_origin] = EncodeConfig(outputs_scalar=ScalarKind.F32, progress=Progress(), stats=Optional())
) -> Encoded[origin_of(arena)]
```

Raises: [`Error`](https://www.unum.cloud/docs/uform/mojo/errors.md#Error) with [`MALFORMED`](https://www.unum.cloud/docs/uform/mojo/errors.md#MALFORMED) for an undecodable recording.

## audio_encode_samples

Encodes decoded mono [`F32`](https://www.unum.cloud/docs/uform/mojo/encoding.md#F32) recordings sharing one `sample_rate`, resampled to the model's. Calls `uform_audio_encode_samples`.

```mojo
def audio_encode_samples[samples_origin: ImmOrigin, stats_origin: MutOrigin, //](
    self,
    samples: TapeView[samples_origin],
    sample_rate: Int,
    outputs: Outputs,
    mut arena: Arena,
    config: EncodeConfig[stats_origin] = EncodeConfig(outputs_scalar=ScalarKind.F32, progress=Progress(), stats=Optional())
) -> Encoded[origin_of(arena)]
```

Raises: [`Error`](https://www.unum.cloud/docs/uform/mojo/errors.md#Error) if the shapes disagree.

## Residency

Where an arena's outputs live, as [`Arena.describe`](#describe) reports it, `uform_residency_t`.

```mojo
struct Residency
```

- `RAM` = `Residency(Int32(1))`: Host pages, which the host reads directly.
- `VRAM` = `Residency(Int32(2))`: One GPU's memory, which only device code reads.
- `code` `c_int`: The raw `uform_residency_t` value the ABI exchanges.

Conforms to: `Copyable`, `Deinitable`, `Equatable`, `ImplicitlyCopyable`, `Movable`, `RegisterPassable`, `TrivialRegisterPassable`