typealias Progress = @Sendable (_ completed: Int, _ total: Int) -> ContinuationNever invoked concurrently, so it needs no synchronization of its own, but it runs on whichever
thread advanced the work, hence @Sendable.
The receiver chooses where a verb runs and how it completes, and the verb stays the same.
Called on the model, a verb runs on the calling thread; on model.on(executor) it blocks on that executor, throwing contended while another call holds it; on model.deferred(executor) it submits there and answers a Deferred at once.
let here = try model.textEncode(texts, arena: arena) // the calling thread
let blocking = try model.on(executor).textEncode(texts, arena: arena) // blocking, on the executor
let call = try model.deferred(executor).imageEncode(files, arena: arena) // Deferred<Encoded>
let images = try await call.value // or: try call.wait()A Deferred owns its outcome, its outputs and copies of its inputs until the call is final, so the caller’s arrays may go at once.
It borrows the model, the executor and the arena as a blocking call does, since releasing any of them waits for the calls naming it. status reads deferred until then, through uform_outcome_status, and wait() blocks through uform_outcome_wait and answers the result or throws the failure, the same every time it is asked. value waits on a Dispatch thread and then reads the result in the caller’s isolation, so an async caller suspends rather than blocks.
Releasing an unfinished one waits for it first.
let call = try model.deferred(executor).audioEncode(recordings, arena: arena)
if call.status == .deferred {
// other work
}
let encoded = try await call.value // or: try call.wait()A failure this binding catches before the call crosses, like heights and widths that do not name one per image, throws at submission, and one the library catches arrives through status and wait.
A call finding another one holding its executor finishes at once with contended.
API
Types
ProgressA periodic report of a long call's advance - units finished and units in total, items for an encoding and tensors for a load - and the caller's chance to stop it.
StructMoodWhat an executor does with its threads or streams once it goes idle, carrying the C enumerator.4
parkedIdle threads and streams sleep, freeing the core.
static let parked: MoodspinningIdle threads and streams poll, trading a core for the lowest wake-up latency.
static let spinning: MooddescriptionThe mood as the library spells it.
initReads a spelling back, and nil for one no mood carries.
EnumContinuationWhether the call a progress handler is watching carries on or unwinds.
enum ContinuationValues
continue | Carry on to the next unit of work. | |
|---|---|---|
cancel | Stop, and report cancelled. |
Conforms to
Equatable, Hashable, SendableClassDeferredA call submitted to an executor, which finishes there while the caller does other work.3
final class Deferred<Value>It owns the call's outcome, outputs and copied inputs until the call is final. The handles it names, the arena among them, it borrows as a blocking call does. Releasing an unfinished one waits for it first.
Conforms to
SendablewaitBlocks until the call is final, then answers its result or throws its failure, the same every time it is asked.
valueWaits for the call on a Dispatch thread, so an async caller suspends rather than blocks and the cooperative pool keeps every thread; the result is read back in the caller's isolation.
nonisolated(nonsending) var value: Value { get async throws(UFormError) }Binding-only: runs wait.
StructModelOnA model's verbs on an executor, each blocking until final after every call submitted there before it; see Model.on(_:) and the model's own verbs for what each one does.8
text*× 2Calls uform_text_encode.
func textEncode(
_ texts: [String],
outputs: Outputs = .embeddings,
arena: Arena,
config: EncodeConfig = EncodeConfig()
) throws(UFormError) -> Encodedimage*× 2Calls uform_image_encode.
func imageEncode(
_ images: [[UInt8]],
outputs: Outputs = .embeddings,
arena: Arena,
config: EncodeConfig = EncodeConfig()
) throws(UFormError) -> EncodedimageEncode*× 2Calls uform_image_encode_pixels.
func imageEncodePixels(
_ pixels: [[UInt8]],
heights: [UInt32],
widths: [UInt32],
outputs: Outputs = .embeddings,
arena: Arena,
config: EncodeConfig = EncodeConfig()
) throws(UFormError) -> Encodedvideo*× 2Calls uform_video_encode.
func videoEncode(
_ videos: [[UInt8]],
outputs: Outputs = .embeddings,
arena: Arena,
config: EncodeConfig = EncodeConfig()
) throws(UFormError) -> Encodedaudio*× 2Calls uform_audio_encode.
func audioEncode(
_ recordings: [[UInt8]],
outputs: Outputs = .embeddings,
arena: Arena,
config: EncodeConfig = EncodeConfig()
) throws(UFormError) -> EncodedaudioEncode*× 2Calls uform_audio_encode_samples.
func audioEncodeSamples(
_ samples: [[Float]],
sampleRate: UInt32,
outputs: Outputs = .embeddings,
arena: Arena,
config: EncodeConfig = EncodeConfig()
) throws(UFormError) -> Encodedtext*× 2Calls uform_text_choose.
func textChoose(
_ texts: [String],
choices: [[UInt8]],
temperatureGroups: [UInt32]? = nil,
arena: Arena,
progress: Progress? = nil,
progressSteps: Int = 0
) throws(UFormError) -> Chosen*× 2Calls uform_model_describe.
func describe() throws(UFormError) -> ModelDescriptionStructModelDeferredOnA model's verbs submitted to an executor, each answered at once as a Deferred that finishes there.
struct ModelDeferredOnA deferred call copies its inputs, so the caller's arrays may go at once. The model, the executor and the arena are borrowed as a blocking call borrows them, since releasing any of them waits for the calls naming it.
Conforms to
Sendable