Batches | UCall for Mojo

def score(batch: Batch) raises:
    for index in range(len(batch)):
        var request = batch.take(index)
        var length = String(len(request.body()))
        request^.reply(HTTPStatus.OK, length.as_bytes())

var scores = server.batch["/score", score](Verb.POST, Batching(2, 32, 10_000))
scores.take_batches()  # on its own thread, until server.stop()

Requests from every event loop share one queue per endpoint, drained by whichever thread polls it; requests the handler does not take answer 500 once it returns.

API

StructBatchingWhen a batch endpoint hands its queue to the handler; zero fields take 1, 32 and 10 milliseconds.1

struct Batching

Fields

`min_size`	`Int`	Dispatch as soon as this many requests wait.
`max_size`	`Int`	Never hand the handler more than this many requests at once.
`max_latency_microseconds`	`Int`	Dispatch a smaller batch once its oldest request waited this long.

Conforms to

Copyable, Defaultable, Deinitable, ImplicitlyCopyable, Movable, RegisterPassable, TrivialRegisterPassable

__init__Every default.

def __init__() -> Self

StructBatchThe requests of one due batch, from every event loop, lent to the handler in arrival order.3

struct Batch

Each request is taken at most once; whatever the handler leaves answers 500 once it returns.

Conforms to

Deinitable, Movable, Sized

__init__

def __init__(out self, state: Int, requests: Pointer[Int, ImmUntrackedOrigin], count: Int)

__len__Requests in the batch, taken or not.

def __len__(self) -> Int

takeTakes request index out of the batch, to reply to it here or on another thread. Binding-only.

def take(self, index: Int) -> DetachedRequest

Parameters

`indexInt`		The request's position in arrival order.

Returns

The request, owned from here on.

Raises

Error with INVALID_ARGUMENT past the last request, or ALREADY_EXISTS once taken.

StructBodyDeliveryWhen an endpoint's handler runs, ucall_body_delivery_t in C.

struct BodyDelivery

Fields

`code`	`c_int`	The raw `ucall_body_delivery_t` value the ABI exchanges.

Values

`BUFFERED`	`BodyDelivery(Int32(0))`	Once the whole body is buffered.
`STREAMED`	`BodyDelivery(Int32(1))`	At headers time, the body pulled through `DetachedRequest.body_chunk`.

Conforms to

Copyable, Deinitable, Equatable, ImplicitlyCopyable, Movable, RegisterPassable, TrivialRegisterPassable