The receiver decides where a verb runs and how it completes, while the verb and its arguments stay the same:
model.text_encode(..) runs on the calling thread, through &Model.model.on(&executor).text_encode(..) blocks on the executor, failing Contended while another call holds it.model.deferred(&executor).text_encode(..) submits the call and answers a Deferred.ModelVerbs and EncodeVerbs write every verb once for all three, and TrainVerbs does so for a trainer’s two receivers.
A blocking verb returns its Result directly, so nothing about asynchrony reaches a caller who never asks for it.
An On borrows the model and the executor, so neither it nor a Deferred it submits can outlive them.
let texts = vec![String::from("a cat on a mat"), String::from("a dog in the fog")];
let call = model.deferred(&executor).text_encode(texts, Outputs::EMBEDDINGS, Scalar::F32, &mut arena)?;
if call.status() == Err(Status::Deferred) {
// other work
}
let encoded = call.await?; // or: call.wait()?A Deferred owns the inputs the library reads until the call is final, and borrows the model and the arena, as a blocking call does. status answers Ok(()) once the call succeeded, Err(Status::Deferred) while it is not final, and its failure otherwise.
Inside an async block, .await polls the call’s status until it is final, so other tasks run meanwhile, while wait blocks the thread without spinning.
The crate starts no thread, so waiting off-thread goes through a scoped thread, as std::thread::scope(|scope| scope.spawn(move || call.wait()).join()) does.
Dropping an unfinished Deferred waits for the call, and a panic a progress handler raised resurfaces from wait, .await, or a drop outside unwinding.
The library reads a deferred call’s inputs after the verb returns, so a deferred receiver takes owned inputs: items in a Vec or an array of owned or 'static values, a tape as (Vec<_>, Vec<usize>), a PathBuf, String or 'static path, and 'static configs and handlers.
Lending &Vec<String> fails to compile with “a deferred receiver needs owned inputs”, and a config borrowing a local fails as “does not live long enough”.
A deferred call allocates one box holding its outcome, argument struct and inputs until it is final.
An executor without threads of its own may hold a deferred call until its wait, its next drain or blocking call, or the drop of what it names, so drain it before an .await there.
A call finding another one holding its executor finishes at once with Status::Contended.
ProgressCallbackA periodic report of finished and total units; Break stops the call at the next slice.pub type ProgressCallback<'a> = dyn FnMut(usize, usize) -> ControlFlow<()> + Send + 'a;It runs on whichever thread advanced the work, never on two at once, so it owes Send and
may be FnMut. A panic inside it stops the call like a Break and is re-raised once the
call is final.
Mirrors uform_progress_callback_t.
BlockingThe completion of a blocking receiver, whose verbs return once their call is final and answer what it produced.DeferredA submitted call, which owns its outcome, argument struct and inputs in one box the library reads until the call is final, and borrows its arena as a blocking call does.2pub struct Deferred<'handle, Verb: Calling>Deferred::status polls without blocking, and Deferred::wait blocks until the call is
final and hands back what it produced, while .await polls it from async code. Dropping an
unfinished one waits for it, then frees the box.
A panic a handler raised resurfaces from wait, .await, or a drop outside unwinding.
'handle is the borrow of the On that submitted it.
Rust-only; waiting and dropping call uform_outcome_wait.
Drop, Debug, Send, IntoFuturestatusErr(Status::Deferred) until the call is final, then its final status, read without blocking.waitBlocks until the call is final, then hands back what it produced.DeferringThe completion of a deferring receiver, whose verbs return once their call is submitted and answer a Deferred.MoodWhat an executor does with its threads or streams once it goes idle.pub enum MoodMirrors uform_mood_t.
Parked | 0 | Sleep on futexes or events, freeing the core; the next call pays a wake-up. |
|---|---|---|
Spinning | 1 | Poll them, trading a core for the lowest wake-up latency. |
Clone, Copy, StructuralPartialEq, PartialEq, Eq, Debug, Default, Hash, Display, FromStrOnA model or a trainer borrowed with the executor its calls run on and how they complete.11pub struct On<'handle, Handle, Completion = Blocking>Neither the receiver nor a Deferred it submits outlives the handle or the executor.
Rust-only.
text_encodeEncodes UTF-8 texts through one forward, a token per byte.Inherited from EncodeVerbs
fn text_encode<'config, 'arena, Texts: TapeView<u8>>(
self,
texts: Texts,
outputs: Outputs,
config: impl Into<TextEncodeConfig<'config>>,
arena: &'arena mut Arena,
) -> <<Self as ModelVerbs<'model>>::Completion as Completion>::Answer<'model, TextEncode<'config, 'arena, Texts>>
where
Self::Completion: Accepts<Texts> + Accepts<TextEncodeConfig<'config>>,Calls uform_text_encode.
text_chooseScores each item's legal choices through one forward: texts are UTF-8 states, each ending where its choice goes, and choices holds each item's legal bytes, one choice per byte.Inherited from EncodeVerbs
fn text_choose<'config, 'arena, Texts: TapeView<u8>, Choices: TapeView<u8>>(
self,
texts: Texts,
choices: Choices,
config: impl Into<TextChooseConfig<'config>>,
arena: &'arena mut Arena,
) -> <<Self as ModelVerbs<'model>>::Completion as Completion>::Answer<'model, TextChoose<'config, 'arena, Texts, Choices>>
where
Self::Completion: Accepts<Texts> + Accepts<Choices> + Accepts<TextChooseConfig<'config>>,Answers one probability per choice in choices order, a softmax over the item's choices
alone at its question type's calibrated inverse temperature.
A state longer than the model's context loses its head, never its end, and a model without
the byte decoder fails the call with Status::Unsupported.
Calls uform_text_choose.
image_encodeDecodes and encodes JPEG, PNG or WebP files; an undecodable file fails the call with Status::Malformed.Inherited from EncodeVerbs
fn image_encode<'config, 'arena, Images: TapeView<u8>>(
self,
images: Images,
outputs: Outputs,
config: impl Into<ImageEncodeConfig<'config>>,
arena: &'arena mut Arena,
) -> <<Self as ModelVerbs<'model>>::Completion as Completion>::Answer<'model, ImageEncode<'config, 'arena, Images>>
where
Self::Completion: Accepts<Images> + Accepts<ImageEncodeConfig<'config>>,Calls uform_image_encode.
image_encode_pixelsEncodes decoded images of interleaved 8-bit RGB, item i holding heights[i] rows of widths[i] pixels; an item of another length fails with Status::ShapeMismatch.Inherited from EncodeVerbs
fn image_encode_pixels<'config, 'arena, Pixels: TapeView<u8>, Heights: AsRef<[u32]>, Widths: AsRef<[u32]>>(
self,
pixels: Pixels,
heights: Heights,
widths: Widths,
outputs: Outputs,
config: impl Into<ImageEncodePixelsConfig<'config>>,
arena: &'arena mut Arena,
) -> <<Self as ModelVerbs<'model>>::Completion as Completion>::Answer<'model, ImageEncodePixels<'config, 'arena, Pixels, Heights, Widths>>
where
Self::Completion: Accepts<Pixels> + Accepts<Heights> + Accepts<Widths> + Accepts<ImageEncodePixelsConfig<'config>>,Calls uform_image_encode_pixels.
video_encodeDecodes and encodes MP4 or WebM clips.Inherited from EncodeVerbs
fn video_encode<'config, 'arena, Videos: TapeView<u8>>(
self,
videos: Videos,
outputs: Outputs,
config: impl Into<VideoEncodeConfig<'config>>,
arena: &'arena mut Arena,
) -> <<Self as ModelVerbs<'model>>::Completion as Completion>::Answer<'model, VideoEncode<'config, 'arena, Videos>>
where
Self::Completion: Accepts<Videos> + Accepts<VideoEncodeConfig<'config>>,Calls uform_video_encode.
audio_encodeDecodes and encodes WAV, FLAC, MP3 or Opus recordings, resampled to mono.Inherited from EncodeVerbs
fn audio_encode<'config, 'arena, Recordings: TapeView<u8>>(
self,
recordings: Recordings,
outputs: Outputs,
config: impl Into<AudioEncodeConfig<'config>>,
arena: &'arena mut Arena,
) -> <<Self as ModelVerbs<'model>>::Completion as Completion>::Answer<'model, AudioEncode<'config, 'arena, Recordings>>
where
Self::Completion: Accepts<Recordings> + Accepts<AudioEncodeConfig<'config>>,Calls uform_audio_encode.
audio_encode_samplesEncodes decoded mono recordings sharing one sample_rate, resampled to the model's rate where it differs.Inherited from EncodeVerbs
fn audio_encode_samples<'config, 'arena, Samples: TapeView<f32>>(
self,
samples: Samples,
sample_rate: u32,
outputs: Outputs,
config: impl Into<AudioEncodeSamplesConfig<'config>>,
arena: &'arena mut Arena,
) -> <<Self as ModelVerbs<'model>>::Completion as Completion>::Answer<'model, AudioEncodeSamples<'config, 'arena, Samples>>
where
Self::Completion: Accepts<Samples> + Accepts<AudioEncodeSamplesConfig<'config>>,Calls uform_audio_encode_samples.
handlesThe model this receiver reaches and the executor its calls run on.describeReports the model's geometry and placement in one call.Inherited from ModelVerbs
fn describe(
self,
) -> <<Self as ModelVerbs<'model>>::Completion as Completion>::Answer<'model, ModelDescribe>Calls uform_model_describe.
stepApplies one optimizer step over this process's items and every peer's: forward, losses, backward, the world gradient reduction, and the update.Inherited from TrainVerbs
fn step<'config, 'arena, Modalities: AsRef<[Modality]>, Contents: TapeView<u8>>(
self,
modalities: Modalities,
contents: Contents,
config: impl Into<TrainerStepConfig<'config, 'arena>>,
) -> <<Self as TrainVerbs<'trainer>>::Completion as Completion>::Answer<'trainer, TrainerStep<'config, 'arena, Modalities, Contents>>
where
Self::Completion: Accepts<Modalities> + Accepts<Contents> + Accepts<TrainerStepConfig<'config, 'arena>>,Every process of the run calls it
once per step, with one modality per item and each item's bytes in contents.
Answers the pooled embeddings when the config names an arena.
Calls uform_trainer_step.
saveWrites the weights and the step, which the model loads serve and Trainer::init_vram resumes; only the process of index zero writes, the rest return at once.Inherited from TrainVerbs
fn save<'config, Location: AsRef<Path>>(
self,
config: impl Into<TrainerSaveConfig<'config, Location>>,
) -> <<Self as TrainVerbs<'trainer>>::Completion as Completion>::Answer<'trainer, TrainerSave<'config, Location>>
where
Self::Completion: Accepts<TrainerSaveConfig<'config, Location>>,Calls uform_trainer_save.
WaitA Deferred being awaited: every poll reads its status, and one not yet final asks to be polled again.pub struct Wait<'handle, Verb: Calling>(_);The C library has no wake-up, so an executor re-polls it while other tasks run, spinning only
when nothing else is ready; Deferred::wait blocks without spinning.
Rust-only; the final poll calls uform_outcome_wait, which returns at once.
Future