# Training > UForm for JavaScript Training runs on GPUs through the Node addon. A run's processes join a mesh once, and a trainer borrows it until freed. ```javascript import { Executor, Mesh, Trainer, Objective, Modality, PairLossForm, LateInteractionSide } from "@unum-cloud/uform"; const gpus = Executor.forVram(); const mesh = Mesh.bootstrap(gpus, { hostDomainIndex: 0, hostDomainCount: 2, directory: "/shared", job: "run-1" }); const trainer = Trainer.initVram( gpus, { depth: 12, hidden: 512, batch: { maxRowsPerDevice: 64 }, objective: Objective.Joint, schedule: { steps: 100000 } }, mesh, ); const { stats } = trainer.step({ modalities: [Modality.Text, Modality.Image], contents: ["a small red panda", jpeg], relations: { attract: { offsets: [0, 1, 2], ids: [1, 0] } }, }); await trainer.deferred().save("checkpoint.safetensors"); // or: call.wait() ``` A `lateInteraction` term trains the token head over items whose `sides` name them queries or documents, and a `choices` term trains the byte decoder on each item's legal bytes and their target probabilities: ```javascript const reranker = Trainer.initVram(gpus, { depth: 12, hidden: 512, objective: Objective.Contrastive, schedule: { steps: 100000 }, batch: { maxRowsPerDevice: 64, maxChoicesPerItem: 16 }, lateInteraction: { weight: 1, pairLoss: PairLossForm.Ranked }, choices: { weight: 1 }, }); reranker.step({ modalities: [Modality.Text, Modality.Text], contents: ["red panda diet", "Red pandas eat bamboo."], sides: [LateInteractionSide.Query, LateInteractionSide.Document], choices: ["yn", ""], choiceTargets: [Float32Array.of(0.9, 0.1), new Float32Array()], relations: { attract: { offsets: [0, 1, 2], ids: [1, 0] } }, }); ``` The choices take the shape `textChoose` takes, and their targets the shape of the `Probabilities` it answers, so one model's probabilities can train another as they land. Every batch, `contents` and `choices` alike, also takes a packed `{ data, offsets }` tape, which the Node addon lends to the library without a copy when its offsets are a `BigUint64Array`; an array of entries is first gathered into one tape, the one copy that path makes. `import * as uform from 'uform';` `import { Trainer } from 'uform';` ## BatchShape What one device's step carries at most. ```typescript interface BatchShape ``` Mirrors `uform_batch_shape_t`; every absent field reads zero. - `maxRowsPerDevice?` `number` - `maxSegmentsPerDevice?` `number` - `maxColumnsPerRow?` `number` - `maxBagsPerDevice?` `number` - `maxChoicesPerItem?` `number`: Legal bytes one item offers to choose from at most; absent makes no choices. - `image?` `{ patchSide?: number; channels?: number }` - `video?` `{ patchSide?: number; tubeFrames?: number; frameStride?: number; channels?: number }` - `audio?` `{ chunkSamples?: number; sampleRate?: number }` ## Masking How the masked objective corrupts a batch and weighs its heads. ```typescript interface Masking ``` Mirrors `uform_masking_t`; absent fields derive. - `tokenMaskProbability?` `number` - `tokenMeanSpan?` `number` - `replaceFraction?` `number` - `randomFraction?` `number` - `imageMaskRatio?` `number` - `videoMaskRatio?` `number` - `audioMaskRatio?` `number` - `textWeight?` `number` - `imageWeight?` `number` - `videoWeight?` `number` - `audioWeight?` `number` ## Contrastive How the contrastive loss scores the pooled embeddings. ```typescript interface Contrastive ``` Mirrors `uform_contrastive_t`; absent fields derive. - `matryoshkaDimensions?` `number[]` - `temperatureGroups?` `number` - `inverseTemperature?` `number` - `bias?` `number` - `negatives?` `Negatives` - `ambientRepulsionWeight?` `number` - `maxEdgesPerDevice?` `number` - `pairLoss?` `PairLossForm`: The pair loss over each anchor's row; absent keeps sigmoid. ## LateInteraction How the late-interaction loss scores token vectors. ```typescript interface LateInteraction ``` Mirrors `uform_late_interaction_t`; absent fields derive. - `weight?` `number`: The loss weight; absent trains no token head. - `width?` `number`: The token vectors' width, a multiple of 8; absent derives 128. - `inverseTemperature?` `number` - `bias?` `number` - `maxQuerySpan?` `number` - `maxQueryTokensPerDevice?` `number` - `maxDocumentsPerDevice?` `number` - `maxDocumentTokensPerDevice?` `number` - `pairLoss?` `PairLossForm`: The pair loss over each query's row of hard MaxSim scores; absent keeps ranked. ## Schedule The learning-rate schedule. ```typescript interface Schedule ``` Mirrors `uform_schedule_t`. - `learningRate?` `number` - `minLearningRate?` `number` - `warmupSteps?` `number` - `steps` `number`: Steps the run takes, which the cosine spans; required. ## TrainerConfig A training run's geometry and objectives. ```typescript interface TrainerConfig ``` Mirrors `uform_trainer_init_vram_t`. - `resume?` `string`: The checkpoint to continue from; absent starts from `seed`. - `depth` `number`: Transformer blocks stacked. - `hidden` `number`: Embedding width, a multiple of 128. - `batch` `BatchShape` - `objective` `Objective` - `masking?` `Masking` - `contrastive?` `Contrastive` - `lateInteraction?` `LateInteraction` - `choices?` `Choices`: Read when its weight is positive, which needs the batch's `maxChoicesPerItem`. - `schedule` `Schedule` - `seed?` `number | bigint`: The base of every keyed draw. ## Edges One edge set over a step's items, as compressed sparse rows. ```typescript interface Edges ``` Mirrors `uform_edges_view_t`. - `offsets` `Uint32Array | number[]`: The step's `count + 1` offsets into `ids`. - `ids` `Uint32Array | number[]`: Item indices each item links to, ascending within each run. - `weights?` `Float32Array | number[]`: The weight of each edge; absent weighs every edge one. ## TrainerBatch This process's share of one step. ```typescript interface TrainerBatch ``` Mirrors the inputs of `uform_trainer_step_t`. - `modalities` `readonly Modality[]`: The modality of each item. - `contents` `readonly (string | Uint8Array)[] | Tape`: Each item's bytes, UTF-8 text or an encoded file, or a tape of them. - `bags?` `Uint32Array | number[]`: The bag of each item; items sharing a bag land on one device. - `temperatureGroups?` `Uint32Array | number[]`: The temperature group of each item. - `sides?` `readonly LateInteractionSide[]`: The late-interaction side of each item; absent makes every item [`LateInteractionSide.None`](#None). - `choices?` `readonly (string | Uint8Array)[] | Tape`: Each item's legal choices, one byte each, as [`textChoose`](https://www.unum.cloud/docs/uform/javascript/models.md#textChoose) takes them; an item offering none makes no choice. - `choiceTargets?` `readonly Float32Array[] | Tape`: The target probability of each choice, an entry per item as long as its choices and summing to one, as [`textChoose`](https://www.unum.cloud/docs/uform/javascript/models.md#textChoose) answers its [`Probabilities`](https://www.unum.cloud/docs/uform/javascript/choices.md#Probabilities); required with `choices`. - `relations?` `{ attract?: Edges; repel?: Edges; ignore?: Edges }`: The contrastive targets. ## TrainerStepConfig Where a step lands the embeddings it scored, when a caller wants them. ```typescript interface TrainerStepConfig ``` - `arena?` `Arena`: The memory the embeddings land in; absent skips the copy. - `outputsScalar?` `Scalar`: The scalar they land as; absent lands `f32`. ## Stepped What one step reports, and the embeddings it scored when an arena was given. ```typescript interface Stepped ``` - `stats` `CompiledStepped["stats"]` - `embeddings?` `Rows` ## Modality What the bytes of one training item hold. ```typescript enum Modality ``` - `Text` = `"text"` - `Image` = `"image"` - `Video` = `"video"` - `Audio` = `"audio"` ## Objective Which losses a training step relaxes. ```typescript enum Objective ``` - `Masked` = `"masked"`: Masked reconstruction alone. - `Contrastive` = `"contrastive"`: The pairwise contrastive loss alone. - `Joint` = `"joint"`: Both, joined at the final-norm states. ## Negatives Which pairs the contrastive loss pushes apart. ```typescript enum Negatives ``` - `Ambient` = `"ambient"`: Repel edges, and every pair no edge names. - `Named` = `"named"`: Repel edges alone. ## PairLossForm How a contrastive term turns a row of scores into forces, numbered as `uform_pair_loss_form_t` numbers them. ```typescript enum PairLossForm ``` - `Derived` = `0`: The term's own form: sigmoid for pooled embeddings, ranked for token vectors. - `Balanced` = `1`: The sigmoid cross-entropy of every cell, its attract and repel halves normalized apart. - `Ranked` = `2`: The sigmoid cross-entropy of every positive's score over every negative's, where the bias cancels. - `Sigmoid` = `3`: SigLIP's sigmoid cross-entropy of every cell, each repel weighed like an attract. ## LateInteractionSide Which side of a late-interaction score an item takes, numbered as `uform_late_interaction_side_t` numbers them. ```typescript enum LateInteractionSide ``` - `None` = `0`: The item takes no part in late interaction. - `Query` = `1`: The item's tokens score the documents it attracts and repels. - `Document` = `2`: The item's tokens are scored by the queries naming it. - `QueryAndDocument` = `3`: The item plays both sides. ## Trainer A training run on a VRAM executor's devices, every verb blocking on that executor; `deferred()` submits them there instead. ```typescript class Trainer extends TrainerVerbs<"blocking"> implements Disposable ``` ## initVram Builds one replica per device and starts or resumes the run. ```typescript static initVram( executor: Executor, config: TrainerConfig, mesh?: Mesh, ): Trainer ``` Calls `uform_trainer_init_vram`. - `executor`: A VRAM executor, every device of which holds one replica. - `config`: The geometry, objectives and schedule. - `mesh`: The mesh to borrow until the trainer is freed; absent runs this process alone. ## deferred The same verbs, each submitted to the trainer's executor and answered as a [`Deferred`](https://www.unum.cloud/docs/uform/javascript/errors.md#Deferred). ```typescript deferred(): TrainerDeferredOn ``` Binding-only. ## free Frees the trainer and lets go of its mesh. ```typescript free(): void ``` Calls `uform_trainer_free`. ## step Applies one optimizer step over this process's items and every peer's. ```typescript step( batch: TrainerBatch, config: TrainerStepConfig = {}, ): Answer ``` Calls `uform_trainer_step`. - `batch`: This process's items, their modalities, bags and relations. - `config`: Where the scored embeddings land. ## save Writes the weights and the step as a safetensors checkpoint; only process zero writes. ```typescript save(path: string): Answer ``` Calls `uform_trainer_save`. - `path`: The destination, written whole and renamed into place.