struct ModelDescriptionMirrors uform_model_description_t.
Conforms to
SendableResidency is the constructor’s name, as in the C lifecycle verbs, so a residency a build lacks has no constructor to call.
let inRam = try Model.loadRam(path: "model.safetensors", memoryDomains: [0], quantization: .bf16)
let inVram = try Model.loadVram(path: "model.safetensors", executor: deviceStreams, devices: [0, 1])
let fromBytes = try Model.loadRamBuffer(checkpoint.span.bytes, quantization: .e4m3)
let description = try inRam.describe()
print(description.dimensions, description.context, description.quantization, description.memoryUsage)
print(description.cpuClass, description.tokenWidth, description.choiceTemperatureGroups)quantization is the scalar the transformer blocks run in: bf16 as the checkpoint stores it, or e4m3 quantized at load.
Every RAM load and executor takes a cpuClass pin, and every VRAM one a gpuClass, dispatching as if the hardware were of that class; nil derives its own. describe() reports the class the model runs as, nil on the other residency, the tokenWidth of its token head, and its choiceTemperatureGroups, each zero when the checkpoint carries no such head. gpuDevices() lists every GPU the process can drive, and a device’s position in it is the ordinal devices names it by.
let pinned = try Model.loadRam(path: "model.safetensors", cpuClass: CPUClass(name: "intel-haswell"))
for (ordinal, device) in try gpuDevices().enumerated() { print(ordinal, device.name, device.gpuClass!) }API
StructModelDescriptionWhat a loaded model reports about itself.8
dimensionsScalars per embedding row; Matryoshka prefixes of it are embeddings too.
var dimensions: IntcontextTokens one item may span before it is truncated.
var context: IntquantizationThe scalar the transformer blocks run in.
var quantization: ScalarmemoryUsageBytes the model holds; activations are the executor's.
var memoryUsage: IntcpuClassThe class a RAM model's tower was packed for, or nil for a VRAM model.
var cpuClass: CPUClass?gpuClassThe class every replica of a VRAM model runs as, or nil for a RAM model.
var gpuClass: GPUClass?tokenWidthScalars per token vector; zero when the checkpoint carries no token head.
var tokenWidth: IntchoiceTemperatureGroupsQuestion types the choice temperatures were calibrated over: zero without the byte decoder, one when uncalibrated.
var choiceTemperatureGroups: IntClassModelA loaded encoder: its weights, their packed copies, and on a GPU the session its attention runs on.14
final class ModelOne model serves every modality it was trained on and is freely shareable across threads. Every
verb called on the model runs on the calling thread and blocks until final; on(_:) runs the
same verbs on an executor, and deferred(_:) submits them there.
Conforms to
SendabletextEncodeEncodes UTF-8 texts, a byte per token.
func textEncode(
_ texts: [String],
outputs: Outputs = .embeddings,
arena: Arena,
config: EncodeConfig = EncodeConfig()
) throws(UFormError) -> EncodedCalls uform_text_encode.
imageEncodeDecodes and encodes JPEG, PNG or WebP files.
func imageEncode(
_ images: [[UInt8]],
outputs: Outputs = .embeddings,
arena: Arena,
config: EncodeConfig = EncodeConfig()
) throws(UFormError) -> EncodedCalls uform_image_encode.
imageEncodePixelsEncodes decoded images of interleaved 8-bit RGB, item i holding heights[i] rows of widths[i] pixels.
func imageEncodePixels(
_ pixels: [[UInt8]],
heights: [UInt32],
widths: [UInt32],
outputs: Outputs = .embeddings,
arena: Arena,
config: EncodeConfig = EncodeConfig()
) throws(UFormError) -> EncodedCalls uform_image_encode_pixels.
videoEncodeDecodes and encodes MP4 or WebM clips.
func videoEncode(
_ videos: [[UInt8]],
outputs: Outputs = .embeddings,
arena: Arena,
config: EncodeConfig = EncodeConfig()
) throws(UFormError) -> EncodedCalls uform_video_encode.
audioEncodeDecodes and encodes WAV, FLAC, MP3 or Opus recordings.
func audioEncode(
_ recordings: [[UInt8]],
outputs: Outputs = .embeddings,
arena: Arena,
config: EncodeConfig = EncodeConfig()
) throws(UFormError) -> EncodedCalls uform_audio_encode.
audioEncodeSamplesEncodes decoded mono recordings sharing one sampleRate, resampled to the model's where it differs.
func audioEncodeSamples(
_ samples: [[Float]],
sampleRate: UInt32,
outputs: Outputs = .embeddings,
arena: Arena,
config: EncodeConfig = EncodeConfig()
) throws(UFormError) -> EncodedCalls uform_audio_encode_samples.
textChooseScores each text's legal choices, one byte each, through one forward, a softmax over them alone at the calibrated inverse temperature of the item's temperatureGroups entry; nil puts every item in group zero.
func textChoose(
_ texts: [String],
choices: [[UInt8]],
temperatureGroups: [UInt32]? = nil,
arena: Arena,
progress: Progress? = nil,
progressSteps: Int = 0
) throws(UFormError) -> ChosenThe call appends the mask byte, and a text longer than the
context loses its head. A model without the byte decoder reports unsupported. progress
hears items scored. Calls uform_text_choose.
loadRamCopies a safetensors checkpoint file into host memory on memoryDomains, an empty list deriving every node the process may allocate on, and packs it for the CPU.
static func loadRam(
path: String,
executor: Executor? = nil,
memoryDomains: [Int32] = [],
cpuClass: CPUClass? = nil,
quantization: Scalar = .bf16,
progress: Progress? = nil,
progressSteps: Int = 0
) throws(UFormError) -> Modelquantization is
the scalar the blocks run in: bf16 as stored, or e4m3 quantized at load. A cpuClass
packs as if the CPU were of that class; nil derives its own. progress hears tensors
placed. Calls uform_model_load_ram.
loadRamBufferCopies a checkpoint held in memory into host memory, as loadRam does a path; nothing of buffer is kept.
static func loadRamBuffer(
_ buffer: RawSpan,
executor: Executor? = nil,
memoryDomains: [Int32] = [],
cpuClass: CPUClass? = nil,
quantization: Scalar = .bf16,
progress: Progress? = nil,
progressSteps: Int = 0
) throws(UFormError) -> ModelCalls uform_model_load_ram_buffer.
loadVramCopies a checkpoint file onto the named GPUs, replicated on each, an empty list deriving one free device, and packs it there.
static func loadVram(
path: String,
executor: Executor? = nil,
devices: [Int32] = [],
gpuClass: GPUClass? = nil,
quantization: Scalar = .bf16,
progress: Progress? = nil,
progressSteps: Int = 0
) throws(UFormError) -> ModelA gpuClass dispatches every device as that class; nil
derives theirs. Calls uform_model_load_vram.
loadVramBufferCopies a checkpoint held in memory onto the named GPUs, as loadVram does a path.
static func loadVramBuffer(
_ buffer: RawSpan,
executor: Executor? = nil,
devices: [Int32] = [],
gpuClass: GPUClass? = nil,
quantization: Scalar = .bf16,
progress: Progress? = nil,
progressSteps: Int = 0
) throws(UFormError) -> ModelCalls
uform_model_load_vram_buffer.
onThe same verbs, blocking on executor after every call submitted there before them.
func on(_ executor: Executor) -> ModelOnBinding-only: names the executor every call passes.
deferredThe same verbs, submitted to executor and answered at once as a Deferred.
func deferred(_ executor: Executor) -> ModelDeferredOnBinding-only: names the executor and the deferred completion every call passes.
describeReports the model's geometry and placement.
func describe() throws(UFormError) -> ModelDescriptionCalls uform_model_describe.