# Serving

> UCall for Python

`ucall.Server` takes the interfaces to answer on, and `backend` and `cpu_class` keywords naming a `ucall.Backend` and a `ucall.CPUClass`, by member or by name such as "linux-epollaio".
Its `log=` callable receives each line the server writes as a `ucall.LogLevel` and a `str`, on whichever thread wrote it and holding the interpreter lock, filtered by `log_level=` and shaped by `log_format=`, and `metrics=ucall.Metrics(address, sample_rate, prefix, tags, interval_microseconds)` sends StatsD datagrams from every event loop.
An `Interface` also takes `max_request_bytes=`, past which a request is answered 413, and `reply_directories=`, the directories a `ucall.Path` part names by position.
`descriptor=` takes a listening socket, or its descriptor, that systemd or a supervisor opened, TCP or AF_UNIX, in place of an address; the server accepts on duplicates and never closes it.
Without interfaces it answers on `ucall.DEFAULT_ADDRESS`, "127.0.0.1:8545".
An `Interface` is an address, optionally pinned to a Linux device, with its own security and admission limits, so one server answers plaintext on loopback and Noise on the wire at once.
It admits 1024 connections by default, reads one request per connection at a time unless `pipelined_requests` allows more, and replies still leave in request order.
At its connection budget a new peer evicts the idle keep-alive connection that progressed least recently.
Port zero binds a free port, which `server.addresses` reports.

```python
import ucall

public = ucall.Interface("0.0.0.0:0", event_loops=4, connections=16_384, pipelined_requests=16)
health = ucall.Interface("127.0.0.1:0", connections=4)
server = ucall.Server(public, health, backend="automatic", cpu_class=ucall.CPUClass.SERIAL)
print(f"{server.event_loops} event loops on {server.backend.name} with {server.cpu_class.name} kernels")

@server.get("/users/{user}/files/{*path}")
def listing(user: str, path: str, limit: int = 10, hidden: bool = False):
    return {"user": user, "path": path, "limit": limit, "hidden": hidden}

@server.post("/scale")
def scale(value: float, factor: float = 2.0):
    return value * factor

@server.route("/anything", ucall.Verb.ANY)
def anything(request: ucall.Request):
    request.reply_error(501, f"{request.verb.name} is not served\n")
```

Endpoints register before any loop is polled, with `route` taking any `Verb` and `get`, `post`, `put`, `delete` and `patch` naming one.
Each decorator registers the function through `add_endpoint` and returns it unchanged, so it stays callable from Python.
Templates capture `{name}` segments and a greedy `{*name}` tail, literal segments match before captures, and registering one template and verb twice raises `ucall.Error` with `Status.ALREADY_EXISTS`.
Routing is exact: an empty segment is a segment, so `/users` and `/users/` are different paths.
A path no endpoint matches answers 404, one whose endpoints take other verbs answers 405 with an `allow` header, a GET endpoint also serves HEAD, and `OPTIONS` answers 204 with the same `allow` list.
A request that cannot be parsed answers 400, 413 or 501 before reaching any handler.

Registration reads a handler's signature once.
A handler taking one parameter annotated `ucall.Request` receives the request itself.
Any other handler takes parameters annotated `str`, `int`, `float`, `bool` or `bytes`, or left bare, and each binds by name from the first place that has it:

- a placeholder of the template, as sent;
- the query, percent-decoded with "+" as a space, a "%" without two hexadecimal digits answering 400;
- a JSON object body, parsed once in C and only when an earlier place missed a parameter, with integers of any width kept exact;
- the parameter's default.

Text converts as `int(text)` and `float(text)` do, `bool` reads "1", "true", "yes" and "on" or "0", "false", "no" and "off" in any case, and `bytes` takes the raw bytes.
A JSON value must already have the annotated type, except that an integer widens to `float` and a string encodes to `bytes`.
A missing or malformed value answers 400 with the reason as plain text, such as "missing parameter 'limit'", without calling the handler.
Registration raises `TypeError` for `*args`, `**kwargs`, more than 16 parameters, or an annotation outside that list.
It refuses an `async def` handler with `TypeError` too, since handlers run on the polling thread; a coroutine application registers a plain function, calls `request.detach()`, and replies from its own event loop.

What a handler returns becomes the reply: `bytes`, `bytearray` and `memoryview` as application/octet-stream, `str` as UTF-8 text/plain, and anything else as application/json, written in C, or through `json.dumps` for a value C does not write, such as a dict keyed by numbers.
None from a typed handler answers 204, and None from a handler taking the request means it replies itself.
A handler's exception reaches `sys.unraisablehook` and answers 500, and the loop keeps serving.

`server.event_loop_statistics(loop)` copies one loop's counters as a dict, from any thread while the loop runs: polls, system calls, submissions and completions, accepted connections, requests, and the bytes received, sent and copied.
Divided by `requests`, they price one call.

`import ucall`

`from ucall import Interface, Metrics, BatchEndpoint, …`

## Backend

An I/O implementation for [`Server`](#Server) and [`Client`](https://www.unum.cloud/docs/ucall/python/calling.md#Client); [`AUTOMATIC`](#AUTOMATIC) picks the best one available.

```python
class Backend(enum.IntEnum)
```

- `AUTOMATIC` = `0`
- `POSIX` = `1`
- `LINUX_EPOLLAIO` = `2`
- `LINUX_URING612` = `3`
- `LINUX_URING618` = `4`
- `BSD_KQUEUE` = `5`
- `WINDOWS_IOCP` = `6`

Examples: ```
>>> ucall.Server(ucall.Interface("127.0.0.1:0"), backend=ucall.Backend.POSIX).backend
<Backend.POSIX: 1>
```

## Verb

An HTTP method; [`ANY`](#ANY) registers an endpoint for every method and never arrives itself.

```python
class Verb(enum.IntEnum)
```

- `UNKNOWN` = `0`
- `GET` = `1`
- `POST` = `2`
- `PUT` = `3`
- `DELETE` = `4`
- `PATCH` = `5`
- `HEAD` = `6`
- `OPTIONS` = `7`
- `ANY` = `8`

Examples: ```
>>> ucall.Verb.POST == 2
True
```

## LogLevel

The severity of a line the [`Server`](#Server) log callback receives, from the most verbose up.

```python
class LogLevel(enum.IntEnum)
```

- `DEBUG` = `0`
- `INFO` = `1`
- `WARNING` = `2`
- `ERROR` = `3`
- `FATAL` = `4`

Examples: ```
>>> ucall.LogLevel.WARNING > ucall.LogLevel.INFO
True
```

## LogFormat

How the library spells a log line: as human-readable fields, or as one JSON object.

```python
class LogFormat(enum.IntEnum)
```

- `HUMAN` = `0`
- `JSON` = `1`

Examples: ```
>>> ucall.LogFormat.JSON
<LogFormat.JSON: 1>
```

## Interface

One address a server answers on, with the security and admission of what arrives there.

```python
class Interface
```

Raises: - `TypeError`: An argument of the wrong type.

- `ValueError`: A key other than 32 bytes long, or an unknown [`security`](#security).

Examples: ```
>>> interface = ucall.Interface("127.0.0.1:0", connections=64, pipelined_requests=16)
>>> interface.connections, interface.security
(64, <Security.NONE: 0>)
```

## Interface

```python
Interface(
    address: str = '',
    /,
    *,
    descriptor: int | socket.socket | None = None,
    device_name: str | None = None,
    security: Security | str = Security.NONE,
    noise_private_key: bytes | None = None,
    peer_authorizer: Callable[[bytes], bool] | None = None,
    noise_peer_key: bytes | None = None,
    event_loops: int = 0,
    connections: int = 1024,
    pipelined_requests: int = 0,
    max_request_bytes: int = 0,
    reply_directories: Sequence[str | bytes | os.PathLike[str] | os.PathLike[bytes]] = (),
)
```

## address

The "host:port" to listen on.

```python
address: str
```

## device_name

The pinned device, or None.

```python
device_name: str | None
```

## security

Encryption of accepted connections.

```python
security: Security
```

## noise_private_key

The 32-byte Noise identity, or None.

```python
noise_private_key: bytes | None
```

## peer_authorizer

The Noise peer vetting, or None.

```python
peer_authorizer: Callable[[bytes], bool] | None
```

## event_loops

Requested loops; 0 chooses.

```python
event_loops: int
```

## connections

Connections open at once.

```python
connections: int
```

## pipelined_requests

Requests one connection carries at once; 0 means one.

```python
pipelined_requests: int
```

## noise_peer_key

The 32-byte static key KK expects, or None.

```python
noise_peer_key: bytes | None
```

## max_request_bytes

The largest request head and buffered body; 0 takes the kernel's receive buffer.

```python
max_request_bytes: int
```

## reply_directories

The directories [`Path`](https://www.unum.cloud/docs/ucall/python/files-and-streams.md#Path) parts name by position.

```python
reply_directories: tuple[str | bytes, ...]
```

## Metrics

Where a [`Server`](#Server) sends StatsD datagrams, each event loop on its own schedule.

```python
class Metrics
```

Examples: ```
>>> metrics = ucall.Metrics("127.0.0.1:8125", tags=("service:inference",))
>>> server = ucall.Server(ucall.Interface("127.0.0.1:0"), metrics=metrics)
```

## Metrics

```python
Metrics(
    address: str | bytes,
    sample_rate: float = 0.0,
    prefix: str | bytes | None = None,
    tags: Sequence[str | bytes] = (),
    interval_microseconds: int = 0,
)
```

## address

The StatsD "host:port".

```python
address: str | bytes
```

## sample_rate

Latencies reported; 0 takes 1.

```python
sample_rate: float
```

## prefix

The metric name prefix, or None for "ucall".

```python
prefix: str | bytes | None
```

## tags

Global "key:value" tags.

```python
tags: tuple[str | bytes, ...]
```

## interval_microseconds

Microseconds between datagrams.

```python
interval_microseconds: int
```

## BatchEndpoint

One batching route of a server, polled by one application thread.

```python
class BatchEndpoint
```

Requests matching it from every event loop share one queue, which [`take_batch`](#take_batch) and [`take_batches`](#take_batches)
hand to the handler as a list once enough arrived or the oldest waited long enough. The handler
runs on its polling owner's thread and replies to each request, in any order.

Examples: ```
>>> @server.batch("/infer", min_size=8, max_size=64, max_latency_microseconds=500)
... def infer(requests: list[ucall.Request]):
...     for request in requests:
...         request.reply(bytes(request.body))
>>> infer in server.batch_endpoints
True
```

## take_batch

Runs the handler on this thread if a batch is due, waiting for one at most `timeout_microseconds`.

```python
def take_batch(timeout_microseconds: int = 0) -> None
```

Each batch endpoint has one polling owner, and the wait polls no event loop.

- `timeout_microseconds`: The longest wait in microseconds; 0 checks once.

Raises: - [`Error`](https://www.unum.cloud/docs/ucall/python/errors.md#Error): The server failed, or was released.

Examples: ```
>>> endpoint.take_batch(timeout_microseconds=1_000)
```

## take_batches

Runs every due batch on this thread, sleeping in between, until `Server.stop()`.

```python
def take_batches() -> None
```

Requests still queued at the stop are released without a call to the handler.

Raises: - [`Error`](https://www.unum.cloud/docs/ucall/python/errors.md#Error): The server failed, or was released.

Examples: ```
>>> poller = threading.Thread(target=endpoint.take_batches)
>>> poller.start()
```

## statistics

Counts the batches the endpoint handed out since registration, and why each fired.

```python
def statistics() -> dict[str, int]
```

Each counter is read on its own, so a snapshot of an endpoint being polled is not one
instant, and batches a stop released count nowhere.

Examples: ```
>>> counters = endpoint.statistics()
>>> counters["requests"] / max(counters["batches"], 1)
0.0
```

## handler

The function handed each batch.

```python
handler: Callable[[list[Request]], object]
```

## Server

An HTTP server over one or more interfaces, polled by threads the application starts.

```python
class Server
```

It starts no threads: each event loop needs one owner calling [`take_calls`](#take_calls), and each batch
endpoint one calling [`BatchEndpoint.take_batches`](#take_batches), while [`run`](#run) serves the simplest server on
the calling thread. Register every endpoint before the first poll.

Raises: - [`Error`](https://www.unum.cloud/docs/ucall/python/errors.md#Error): An interface that cannot be bound, such as an address in use, or a reply directory
that cannot be opened.

Examples: ```
>>> server = ucall.Server(ucall.Interface("127.0.0.1:0"), backend="posix")
>>> server.event_loops, server.backend
(1, <Backend.POSIX: 1>)
```

## Server

```python
Server(
    *interfaces: Interface = (),
    backend: Backend | str = Backend.AUTOMATIC,
    cpu_class: CPUClass | str = CPUClass.UNKNOWN,
    log: Callable[[LogLevel, str], object] | None = None,
    log_level: LogLevel | str = LogLevel.DEBUG,
    log_format: LogFormat | str = LogFormat.HUMAN,
    metrics: Metrics | None = None,
)
```

## add_endpoint

Routes [`verb`](https://www.unum.cloud/docs/ucall/python/requests.md#verb) requests matching the URI template [`path`](https://www.unum.cloud/docs/ucall/python/files-and-streams.md#path) to [`handler`](#handler), on the polling thread.

```python
def add_endpoint(
    path: str | bytes,
    verb: Verb | str,
    handler: F,
    /,
    body: BodyDelivery | str = BodyDelivery.BUFFERED,
) -> F
```

A handler taking one parameter annotated [`Request`](https://www.unum.cloud/docs/ucall/python/requests.md#Request) receives the request, which it may answer
later from any thread, and what it returns other than None is the reply. Any other handler's
parameters, annotated `str`, `int`, `float`, `bool` or `bytes` or left bare, bind by name from
a path placeholder, then the query, then a JSON object body, then their defaults; a missing
or malformed value answers 400, and None returned answers 204. Returned bytes-like objects
reply as binary, `str` as UTF-8 text, and anything else as JSON. An exception answers 500.

- `path`: A URI template such as `/users/{name}` or `/files/{*path}`.
- `verb`: The method to answer; [`Verb.ANY`](#ANY) answers every one.
- `handler`: The function to route to.
- `body`: [`BodyDelivery.STREAMED`](https://www.unum.cloud/docs/ucall/python/batches.md#STREAMED) runs the handler once the headers
arrived, its body pulled through [`DetachedRequest.body_chunks`](https://www.unum.cloud/docs/ucall/python/requests.md#body_chunks). Defaults to buffered.

Raises: - `TypeError`: A parameter that cannot bind, such as `*args` or one annotated `list`.

- [`Error`](https://www.unum.cloud/docs/ucall/python/errors.md#Error): A duplicate route or an invalid template.

Examples: ```
>>> def scale(value: float, factor: float = 2.0):
...     return value * factor
>>> server.add_endpoint("/scale/{value}", ucall.Verb.GET, scale) is scale
True
```

## add_batch_endpoint

Queues matching requests from every event loop for `handler(requests)`, called in batches.

```python
def add_batch_endpoint(
    path: str | bytes,
    verb: Verb | str,
    handler: Callable[[list[Request]], object],
    /,
    min_size: int = 0,
    max_size: int = 0,
    max_latency_microseconds: int = 0,
) -> BatchEndpoint
```

A batch is due once `min_size` requests arrived, `max_size` are queued, or the oldest waited
`max_latency_microseconds`. The handler runs on the thread polling the returned endpoint and
replies to each request.

- `path`: A URI template such as "/infer".
- `verb`: The method to answer; [`Verb.ANY`](#ANY) answers every one.
- `handler`: The function handed each batch.
- `min_size`: Requests that make a batch due at once; 0 means 1.
- `max_size`: Requests one batch holds at most; 0 means 32.
- `max_latency_microseconds`: The longest the oldest request waits for a batch to
fill, in microseconds; 0 means 10 ms.

Raises: - [`Error`](https://www.unum.cloud/docs/ucall/python/errors.md#Error): A duplicate route or an invalid template.

Examples: ```
>>> endpoint = server.add_batch_endpoint("/sizes", "POST", lambda requests: None, 4)
>>> endpoint in server.batch_endpoints
True
```

## route

Registers the decorated function for [`verb`](https://www.unum.cloud/docs/ucall/python/requests.md#verb) requests matching the URI template [`path`](https://www.unum.cloud/docs/ucall/python/files-and-streams.md#path).

```python
def route(
    path: str | bytes,
    verb: Verb | str = Verb.GET,
    *,
    body: BodyDelivery | str = BodyDelivery.BUFFERED,
) -> Callable[[F], F]
```

- `path`: A URI template such as "/users/{name}".
- `verb`: The method to answer. Defaults to [`Verb.GET`](#GET).
- `body`: When the handler runs, as [`add_endpoint`](#add_endpoint) takes it.

Examples: ```
>>> @server.route("/anything", ucall.Verb.ANY)
... def anything(request: ucall.Request):
...     return request.verb.name
```

## get

Registers the decorated function for GET requests matching the URI template [`path`](https://www.unum.cloud/docs/ucall/python/files-and-streams.md#path).

```python
def get(
    path: str | bytes,
    /,
    *,
    body: BodyDelivery | str = BodyDelivery.BUFFERED,
) -> Callable[[F], F]
```

- `path`: A URI template such as "/users/{name}".
- `body`: When the handler runs, as [`add_endpoint`](#add_endpoint) takes it.

Examples: ```
>>> @server.get("/users/{name}")
... def user(name: str, verbose: bool = False):
...     return {"name": name, "verbose": verbose}
```

## post

Registers the decorated function for POST requests matching the URI template [`path`](https://www.unum.cloud/docs/ucall/python/files-and-streams.md#path).

```python
def post(
    path: str | bytes,
    /,
    *,
    body: BodyDelivery | str = BodyDelivery.BUFFERED,
) -> Callable[[F], F]
```

- `path`: A URI template such as "/sum".
- `body`: When the handler runs, as [`add_endpoint`](#add_endpoint) takes it.

Examples: ```
>>> @server.post("/sum")
... def sum(a: int, b: float = 0.5):
...     return a + b
```

## put

Registers the decorated function for PUT requests matching the URI template [`path`](https://www.unum.cloud/docs/ucall/python/files-and-streams.md#path).

```python
def put(
    path: str | bytes,
    /,
    *,
    body: BodyDelivery | str = BodyDelivery.BUFFERED,
) -> Callable[[F], F]
```

- `path`: A URI template such as `/files/{*path}`.
- `body`: When the handler runs, as [`add_endpoint`](#add_endpoint) takes it.

Examples: ```
>>> @server.put("/files/{*path}")
... def store(request: ucall.Request):
...     return len(request.body)
```

## delete

Registers the decorated function for DELETE requests matching the URI template [`path`](https://www.unum.cloud/docs/ucall/python/files-and-streams.md#path).

```python
def delete(
    path: str | bytes,
    /,
    *,
    body: BodyDelivery | str = BodyDelivery.BUFFERED,
) -> Callable[[F], F]
```

- `path`: A URI template such as `/files/{*path}`.
- `body`: When the handler runs, as [`add_endpoint`](#add_endpoint) takes it.

Examples: ```
>>> @server.delete("/files/{*path}")
... def remove(path: str):
...     return None
```

## patch

Registers the decorated function for PATCH requests matching the URI template [`path`](https://www.unum.cloud/docs/ucall/python/files-and-streams.md#path).

```python
def patch(
    path: str | bytes,
    /,
    *,
    body: BodyDelivery | str = BodyDelivery.BUFFERED,
) -> Callable[[F], F]
```

- `path`: A URI template such as "/users/{name}".
- `body`: When the handler runs, as [`add_endpoint`](#add_endpoint) takes it.

Examples: ```
>>> @server.patch("/users/{name}")
... def rename(name: str, alias: str):
...     return f"{name} is {alias}"
```

## batch

Registers the decorated `handler(requests)` as a batch endpoint, replacing it with that.

```python
def batch(
    path: str | bytes,
    verb: Verb | str = Verb.POST,
    *,
    min_size: int = 0,
    max_size: int = 0,
    max_latency_microseconds: int = 0,
) -> Callable[[Callable[[list[Request]], object]], BatchEndpoint]
```

- `path`: A URI template such as "/infer".
- `verb`: The method to answer. Defaults to [`Verb.POST`](#POST).
- `min_size`: Requests that make a batch due at once; 0 means 1.
- `max_size`: Requests one batch holds at most; 0 means 32.
- `max_latency_microseconds`: The longest the oldest request waits, in microseconds;
0 means 10 ms.

Examples: ```
>>> @server.batch("/reverse", min_size=4, max_size=4)
... def reverse(requests: list[ucall.Request]):
...     for request in reversed(requests):
...         request.reply(request.body)
>>> type(reverse).__name__
'BatchEndpoint'
```

## take_call

Advances one event loop and runs its ready handlers on this thread.

```python
def take_call(
    event_loop: int = 0,
    timeout_microseconds: int = 0,
) -> bool
```

Each loop has one polling owner, and handlers run inside the call, so a slow one delays
its return past `timeout_microseconds`.

- `event_loop`: The loop to advance, below [`event_loops`](#event_loops). Defaults to 0.
- `timeout_microseconds`: The longest wait for I/O in microseconds; 0 polls once.

Raises: - [`Error`](https://www.unum.cloud/docs/ucall/python/errors.md#Error): An `event_loop` past the last one, or an I/O failure.

Examples: ```
>>> while server.take_call(0, timeout_microseconds=1_000): pass
```

## take_calls

Drives one event loop on this thread until `stop()` or an I/O failure.

```python
def take_calls(event_loop: int = 0) -> None
```

The interpreter lock is released meanwhile and taken back only to run handlers, so one
thread per loop serves in parallel with the others. Signals reach Python only once it
returns, so call it from threads other than the main one.

- `event_loop`: The loop to drive, below [`event_loops`](#event_loops). Defaults to 0.

Raises: - [`Error`](https://www.unum.cloud/docs/ucall/python/errors.md#Error): An `event_loop` past the last one, or an I/O failure.

Examples: ```
>>> pollers = [threading.Thread(target=server.take_calls, args=(loop,))
...            for loop in range(server.event_loops)]
```

## run

Serves a server of one event loop and no batch endpoints on the calling thread.

```python
def run() -> None
```

It returns after `stop()` or `KeyboardInterrupt`, polling in slices of 100 ms and checking
for signals between them, and once stopped it lets the loop finish the requests in flight;
a second `KeyboardInterrupt` cuts them.

Raises: - `RuntimeError`: The server has more than one event loop or any batch endpoint.

- `KeyboardInterrupt`: Ctrl+C arrived while serving.

- [`Error`](https://www.unum.cloud/docs/ucall/python/errors.md#Error): An I/O failure.

Examples: ```
>>> runner = threading.Thread(target=server.run)
>>> runner.start()
>>> server.stop()
>>> runner.join()
```

## stop

Asks the server to stop, waking every polling caller, of loops and batch endpoints, to return.

```python
def stop() -> None
```

The first call drains: no new connections, idle ones close, and the requests
already held are answered before their connections close. A second call cuts the
replies still in flight.

Safe from any thread and from a handler. Join the polling threads before releasing
the server.

Examples: ```
>>> server.stop()
```

## event_loop_statistics

Counts what one event loop did since initialization.

```python
def event_loop_statistics(event_loop: int, /) -> dict[str, int]
```

Divided by `requests`, the counters price one call.

- `event_loop`: The loop to read, below [`event_loops`](#event_loops).

Raises: - [`Error`](https://www.unum.cloud/docs/ucall/python/errors.md#Error): An `event_loop` past the last one.

Examples: ```
>>> server.event_loop_statistics(0)["requests"]
0
```

## event_loops

Resolved loops, each needing one polling thread.

```python
event_loops: int
```

## backend

The I/O implementation initialization chose.

```python
backend: Backend
```

## cpu_class

The CPU kernels initialization chose.

```python
cpu_class: CPUClass
```

## addresses

The resolved "host:port" of each interface.

```python
addresses: tuple[str, ...]
```

## batch_endpoints

Batch endpoints, each needing a polling owner.

```python
batch_endpoints: tuple[BatchEndpoint, ...]
```

## F

```python
F = TypeVar('F', )
```

## DEFAULT_ADDRESS

The address a [`Server`](#Server) without interfaces answers on, "127.0.0.1:8545".

```python
DEFAULT_ADDRESS: Final[str]
```