Rust · Serving

Serving

use ucall::BatchEndpoint;use ucall::BatchEndpointStatistics;use ucall::EventLoopStatistics;use ucall::Interface;use ucall::Metrics;use ucall::Server;use ucall::ServerOptions;use ucall::Backend;use ucall::Log;use ucall::LogFormat;use ucall::LogLevel;use ucall::Verb;use ucall::*; 1.0.0-rc.1 · 60202332

Server::new takes the interfaces to answer on, and Server::with_options also takes ServerOptions: the Backend, an optional CPUClass ceiling, a Log sink and a StatsD endpoint in Metrics. It fails rather than substituting when the backend is unavailable or the CPU does not run the class. An Interface is an address, optionally pinned to a Linux device, with its own security and admission limits, so one server answers plaintext on loopback and Noise on the wire at once. It admits 1024 connections by default, reads one request per connection at a time unless pipelined_requests allows more, and replies still leave in request order. max_request_bytes caps a request’s head and buffered body, answering 413 past it, and reply_directories names the directories path replies resolve beneath. Interface::listening accepts on a listener another owner opened, a TcpListener, a UnixListener or one systemd passed in, instead of binding an address, and never closes it. At its connection budget an interface evicts the idle keep-alive connection that progressed least recently for a new peer. Port zero binds a free port, which Server::address reports.

use std::sync::Arc;
use ucall::*;

let public = Interface::new("0.0.0.0:8545").event_loops(4).connections(16_384).pipelined_requests(16);
let health = Interface::new("127.0.0.1:8546").connections(4);
let log = Log { callback: Arc::new(|level: LogLevel, line: &[u8]| eprintln!("{level:?} {}", String::from_utf8_lossy(line))), format: LogFormat::Human, level: LogLevel::Warning };
let metrics = Metrics { address: "127.0.0.1:8125".into(), prefix: "rpc".into(), ..Default::default() };
let options = ServerOptions { cpu_class: Some(CPUClass::Serial), log: Some(log), metrics: Some(metrics), ..Default::default() };
let mut server = Server::with_options(&[public, health], options)?;
println!("{} event loops on {} with {} kernels", server.event_loops(), server.backend(), server.cpu_class());

server.get("/users/{user}/files/{*path}", |request: Request| {
    let user = request.path_parameter("user").unwrap_or_default();
    let path = request.path_parameter("path").unwrap_or_default();
    let listing = [user, b":".as_slice(), path].concat();
    let _ = request.reply(HTTPStatus::OK, &[], &listing);
})?;
server.route("/anything", Verb::Any, |request: Request| {
    let message = format!("{:?} is not served", request.verb());
    let _ = request.reply_error(HTTPStatus::NOT_IMPLEMENTED, &message);
})?;

Endpoints register through &mut Server before any loop is polled, with route taking any Verb and get, post, put, delete and patch naming one. Templates capture {name} segments and a greedy {*name} tail, literal segments match before captures, and registering one template and verb twice fails with Status::AlreadyExists. Routing is exact: an empty segment is a segment, so /users and /users/ are different paths. A path no endpoint matches answers 404, one whose endpoints take other verbs answers 405 with an allow header, a GET endpoint also serves HEAD, and OPTIONS answers 204 with the same allow list. A request that cannot be parsed answers 400, 413 or 501 before reaching any handler.

Log lines reach the callback on whichever thread produced them, and StatsD datagrams carry each loop’s counters and sampled request latencies, every second unless Metrics::interval says otherwise. Server::event_loop_statistics copies one loop’s counters, from any thread while the loop runs: polls, system calls, submissions and completions, accepted connections, requests, and the bytes received, sent and copied. Divided by requests, they price one call.

API

Types

LogCallbackReceives one formatted log line, without a trailing newline, on whichever thread produced it, polling threads included; it should return quickly, and a panic drops the line.
lib.rs:922 ↗
pub type LogCallback = dyn Fn(LogLevel, &[u8]) + Send + Sync;
StructBatchEndpointOne batch endpoint, polled by one thread at a time and observable from any.3
server.rs:1232 ↗
pub struct BatchEndpoint

Requests from every event loop share its queue, so one thread drains what all of them received. Dropping it leaves its queued requests unanswered until the server is freed, and the queue, once full, refuses new ones with 503.

use std::time::Duration;
use ucall::*;

let mut server = Server::new(&[Interface::new("127.0.0.1:0")])?;
let batching = Batching { min_size: 2, max_size: 2, max_latency: Duration::from_secs(1) };
let scores = server.batch("/score", Verb::Post, batching, |batch: Batch| {
    for request in batch {
        let score = request.body().len().to_string();
        let _ = request.reply(HTTPStatus::OK, &[], score.as_bytes());
    }
})?;
let address = server.address(0).unwrap().to_owned();

let mut client = Client::new(ClientOptions { connections: 2, ..Default::default() })?;
client.submit(ClientCall::post(format!("http://{address}/score"), b"four".as_slice()), 0)?;
client.submit(ClientCall::post(format!("http://{address}/score"), b"seven..".as_slice()), 1)?;
std::thread::scope(|scope| {
    scope.spawn(|| server.take_calls(0));
    scope.spawn(|| scores.take_batches());
    let mut completed: [Completed; 2] = Default::default();
    let mut answered = 0;
    while answered < 2 {
        match client.take_completed(&mut completed, Duration::from_secs(1)) {
            Ok(done) => answered += done.len(),
            Err(_) => break,
        }
    }
    server.stop();
});
let statistics = scores.statistics()?;
assert_eq!((statistics.batches, statistics.requests, statistics.filled_batches), (1, 2, 1));

Implements

Send, Sync, Drop
take_batchRuns the handler on this thread if a batch is due, waiting up to timeout for one; zero performs one nonblocking round.
server.rs:1249 ↗
pub fn take_batch(&self, timeout: Duration) -> Result<()>

Fails with Status::Contended while another thread drains the same endpoint.

take_batchesRuns the handler on this thread for every due batch until Server::stop; requests still queued then are released without a reply.
server.rs:1269 ↗
pub fn take_batches(&self) -> Result<()>

Fails with Status::Contended while another thread drains the same endpoint.

statisticsA snapshot of the endpoint's counters, readable from any thread while it is polled.
server.rs:1285 ↗
pub fn statistics(&self) -> Result<BatchEndpointStatistics>
StructBatchEndpointStatisticsCounters of one batch endpoint since registration: how large its batches were, and why each was dispatched.
server.rs:1147 ↗
pub struct BatchEndpointStatistics

Each counter is read on its own, so a snapshot of an endpoint being polled is not one instant. Requests released by Server::stop count nowhere.

Fields

batchesu64Batches handed to the handler.
requestsu64Requests across those batches.
filled_batchesu64Batches dispatched on reaching Batching::max_size.
ready_batchesu64Batches dispatched on reaching Batching::min_size, below the maximum.
expired_batchesu64Batches dispatched below Batching::min_size once Batching::max_latency passed.

Implements

Clone, Copy, StructuralPartialEq, PartialEq, Eq, Debug, Default
StructEventLoopStatisticsCounters of one event loop since the server started; divided by requests, they price one call.
server.rs:1073 ↗
pub struct EventLoopStatistics

Each counter is read on its own, so a snapshot of a running loop is not one instant. Byte counters cover the bytes UCall itself moves, never the kernel's own copies.

Fields

pollsu64Engine polls the loop issued, each taking zero or more completions.
empty_pollsu64Engine polls that took no completion.
kernel_entersu64System calls: io_uring enters, or other backends' waits, registrations and socket calls.
submissionsu64Operations queued: io_uring submission entries, or requests another backend accepted.
completionsu64Results taken: io_uring completion entries, or operations another backend finished.
cancellationsu64Pending operations, or io_uring multishot receives, the engine asked to cancel.
acceptsu64Connections accepted and handed to this loop.
requestsu64HTTP requests parsed and dispatched to an endpoint or an error reply.
receive_rearmsu64Multishot receives armed again on a socket that already streamed, only on io_uring 7.0.
bytes_receivedu64Bytes the kernel reported received on this loop's connections, ciphertext under Noise.
bytes_sentu64Bytes the kernel reported sent from this loop's connections, ciphertext under Noise.
bytes_copiedu64Bytes UCall copied: receive appends, reply bodies, Noise records and parked receives.
zero_copy_sendsu64Sends that left without copying: reply segments and file windows.
zero_copy_fallbacksu64Zero-copy sends the kernel copied anyway or refused to pin, as loopback always does.
copying_sendsu64Sends that copied instead of pinning: the socket was demoted, refused zero-copy, or had no room to track one.
file_reads_asynchronousu64File reads handed to the kernel to finish later: io_uring, Linux or FreeBSD AIO, or overlapped reads.
file_reads_blockingu64File reads the loop issued itself and blocked on, as the posix backend and macOS do.
bytes_readu64Bytes the kernel reported read from files into the loop's file windows.
file_window_overlapsu64File reads submitted while an earlier window of the same stream waited to send or sent.
linger_resetsu64Lingering closes reset at their deadline because the kernel still held what they sent.
ready_passesu64Passes the loop made over a ready connection, each advancing it as far as it goes.
replied_dequeuesu64Replied exchanges the loop took from its queue to adopt their reply segments.
refused_requestsu64Requests the loop answered itself with 400, 404, 405, 413, 501, 503 or 505.
handshake_failuresu64Connections whose Noise handshake failed.
authorizer_refusalsu64Connections whose authenticated peer key the interface's authorizer refused.

Implements

Clone, Copy, StructuralPartialEq, PartialEq, Eq, Debug, Default
StructInterfaceOne address a server answers on, with the security and admission limits of the connections arriving there.11
server.rs:437 ↗
pub struct Interface

One server may answer plaintext on loopback and Noise on the wire at once, through two interfaces:

use ucall::*;

let (private_key, _) = generate_noise_keypair()?;
let plain = Interface::new("127.0.0.1:0");
let sealed = Interface::new("127.0.0.1:0")
    .security(Security::NoiseXX, private_key)
    .peer_authorizer(|noise_private_key: &NoiseKey| noise_private_key != &[0; 32]);
let server = Server::new(&[plain, sealed])?;
assert!(server.address(1).is_some());

Implements

Clone
newAn interface listening on address in "host:port" form, admitting 1024 connections at once; port zero picks a free one, which Server::address reports.
server.rs:455 ↗
pub fn new(address: impl Into<String>) -> Self
listeningAn interface accepting on a listening stream socket another owner opened, such as a TcpListener, a UnixListener or one systemd passed in; the server accepts on duplicates and never closes listener, which must outlive Server::new.
server.rs:476 ↗
pub fn listening(listener: &impl AsRawFd) -> Self
device_namePins the interface to a physical device, like "eth0"; only Linux supports it.
server.rs:494 ↗
pub fn device_name(self, name: impl Into<String>) -> Self
securityRequires a Noise handshake under this server identity; NoiseXX also needs an authorizer, and NoiseKK the initiator's key.
server.rs:501 ↗
pub fn security(self, security: Security, private_key: NoiseKey) -> Self
noise_peer_keyThe one initiator's static public key a NoiseKK interface admits, known in advance.
server.rs:508 ↗
pub fn noise_peer_key(self, public_key: NoiseKey) -> Self
peer_authorizerVets each client's authenticated static key before any of its requests is read.
server.rs:514 ↗
pub fn peer_authorizer(self, authorizer: impl Fn(&NoiseKey) -> bool + Send + Sync + 'static) -> Self
event_loopsEvent loops accepting on this interface; zero means one, or the receive-queue count of a named Linux device.
server.rs:524 ↗
pub fn event_loops(self, count: usize) -> Self
connectionsConnections open at once across this interface's loops; further ones wait in the listen queue until one closes.
server.rs:531 ↗
pub fn connections(self, count: usize) -> Self
pipelined_requestsRequests one connection may have in flight before the server stops reading from it; zero means one, and replies still leave in request order.
server.rs:538 ↗
pub fn pipelined_requests(self, count: usize) -> Self
max_request_bytesThe largest request head and buffered body; past it the reply is 413 and the connection closes.
server.rs:546 ↗
pub fn max_request_bytes(self, bytes: usize) -> Self

Zero takes the largest receive buffer the kernel grants a socket, and streamed bodies are bounded by the handler's pulls instead.

reply_directoriesDirectories the ReplyPart::Path parts of replies to this interface's requests resolve beneath, by position; each is opened when the server starts, failing it if one cannot be.
server.rs:553 ↗
pub fn reply_directories(self, directories: impl IntoIterator<Item = impl Into<String>>) -> Self
StructMetricsThe StatsD endpoint a server sends each event loop's counters and sampled request latencies to, as UDP datagrams.
server.rs:579 ↗
pub struct Metrics

Fields

addressStringThe endpoint in "host:port" form.
sample_ratef32The fraction of finished requests whose latency is reported, in (0, 1]; zero means 1.
prefixStringThe metric name prefix; empty means "ucall".
tagsVec<String>Tags on every metric, each "key:value".
intervalDurationThe time between one event loop's datagrams; zero means one second.

Implements

Clone, Debug, Default
StructServerAn HTTP server over one or more interfaces, polled by application threads.18
server.rs:704 ↗
pub struct Server

Register every endpoint first, through &mut self, then share &Server with the threads that poll its event loops. Dropping it stops the server; the library frees it once the last BatchEndpoint and retained Request are dropped too.

use ucall::*;

let mut server = Server::new(&[Interface::new("127.0.0.1:0")])?;
server.get("/hello", |request: Request| {
    let _ = request.reply(HTTPStatus::OK, &["content-type: text/plain".into()], b"Hello!");
})?;
let address = server.address(0).unwrap().to_owned();

let mut client = Client::new(ClientOptions { connections: 1, ..Default::default() })?;
std::thread::scope(|scope| {
    for event_loop in 0..server.event_loops() {
        let server = &server;
        scope.spawn(move || server.take_calls(event_loop));
    }
    let response = client.get(format!("http://{address}/hello"));
    server.stop();
    assert_eq!(response?.body(), b"Hello!");
    Ok::<(), Error>(())
})?;
assert_eq!(server.event_loop_statistics(0)?.requests, 1);

Implements

Send, Sync, Drop
newStarts a server answering on interfaces, with the automatic backend and CPU class.
server.rs:720 ↗
pub fn new(interfaces: &[Interface]) -> Result<Server>
with_optionsStarts a server answering on interfaces with the backend, CPU class, log sink and StatsD endpoint options name.
server.rs:726 ↗
pub fn with_options(interfaces: &[Interface], options: ServerOptions) -> Result<Server>
event_loopsEvent loops this server runs, each to be polled by one thread at a time.
server.rs:829 ↗
pub fn event_loops(&self) -> usize
backendThe I/O backend the server resolved to.
server.rs:834 ↗
pub fn backend(&self) -> Backend
cpu_classThe CPU class the server resolved to.
server.rs:839 ↗
pub fn cpu_class(&self) -> CPUClass
addressThe numeric address interface index listens on, with the port it bound.
server.rs:844 ↗
pub fn address(&self, index: usize) -> Option<&str>
routeRoutes requests matching path and verb to handler, which runs on the polling thread.
server.rs:856 ↗
pub fn route(
    &mut self,
    path: &str,
    verb: Verb,
    handler: impl Fn(Request) + Send + Sync + 'static,
) -> Result<()>

Templates capture {name} segments and a greedy {*name} tail, read back through Request::path_parameter; literal segments match before captures, and captures before a tail. A handler that panics, or drops its request unreplied, answers 500. Registering the same template and verb twice fails with Status::AlreadyExists.

route_withRoutes like Server::route, running handler once the body is buffered, or with BodyDelivery::Streamed as soon as the headers arrived, the body still to be pulled through DetachedRequest::body_chunk.
server.rs:868 ↗
pub fn route_with(
    &mut self,
    path: &str,
    verb: Verb,
    delivery: BodyDelivery,
    handler: impl Fn(Request) + Send + Sync + 'static,
) -> Result<()>
getRoutes GET requests on path to handler.
server.rs:899 ↗
pub fn get(&mut self, path: &str, handler: impl Fn(Request) + Send + Sync + 'static) -> Result<()>
postRoutes POST requests on path to handler.
server.rs:908 ↗
pub fn post(&mut self, path: &str, handler: impl Fn(Request) + Send + Sync + 'static) -> Result<()>
putRoutes PUT requests on path to handler.
server.rs:917 ↗
pub fn put(&mut self, path: &str, handler: impl Fn(Request) + Send + Sync + 'static) -> Result<()>
deleteRoutes DELETE requests on path to handler.
server.rs:926 ↗
pub fn delete(
    &mut self,
    path: &str,
    handler: impl Fn(Request) + Send + Sync + 'static,
) -> Result<()>
patchRoutes PATCH requests on path to handler.
server.rs:935 ↗
pub fn patch(&mut self, path: &str, handler: impl Fn(Request) + Send + Sync + 'static) -> Result<()>
batchCollects requests matching path and verb from every event loop into one queue, which the returned endpoint drains into handler on whichever thread drains it.
server.rs:948 ↗
pub fn batch(
    &mut self,
    path: &str,
    verb: Verb,
    batching: Batching,
    handler: impl Fn(Batch<'_>) + Send + Sync + 'static,
) -> Result<BatchEndpoint>

Requests the handler neither replies to nor keeps answer 500 once it returns, as do all of them when it panics.

take_callAdvances one event loop, running the handlers of the requests it completes on this thread, and waits up to timeout for progress; zero performs one nonblocking round.
server.rs:999 ↗
pub fn take_call(&self, event_loop: usize, timeout: Duration) -> Result<Polling>

Answers Polling::Retired once the loop has nothing left to take after Server::stop.

Fails with Status::Contended while another thread polls the same loop.

take_callsDrives one event loop on this thread until Server::stop or an I/O failure.
server.rs:1018 ↗
pub fn take_calls(&self, event_loop: usize) -> Result<()>

Fails with Status::Contended while another thread polls the same loop.

stopRequests shutdown and wakes every polling thread; callable from any thread, including from inside a handler.
server.rs:1035 ↗
pub fn stop(&self)

The first call drains: no new connections, idle ones close, and the requests already held are answered before their connections close. A second call cuts the replies still in flight.

event_loop_statisticsA snapshot of one event loop's counters, readable from any thread while it is polled.
server.rs:1042 ↗
pub fn event_loop_statistics(&self, event_loop: usize) -> Result<EventLoopStatistics>

Fails with Status::InvalidArgument for an index past the last loop.

StructServerOptionsWhat a server runs on and reports through, beside its interfaces.
server.rs:564 ↗
pub struct ServerOptions

Fields

backendBackendThe requested I/O backend, failing rather than substituting when it is unavailable.
cpu_classOption<CPUClass>A ceiling on the CPU class, failing where the CPU does not run it; None takes what the constructing thread's CPU offers.
logOption<Log>Where log lines go; None keeps the server silent.
metricsOption<Metrics>Where StatsD datagrams go; None sends none.

Implements

Clone, Default
EnumBackendThe I/O implementation a server or client runs on.
types.rs:686 ↗
pub enum Backend

Prints and parses its stable name, like "linux-uring618".

Values

AutomaticThe best native backend this machine and build offer.
POSIXPortable readiness polling on any POSIX system.
LinuxEpollAIOLinux readiness polling over epoll, reading files through Linux AIO.
LinuxURing612Linux io_uring as kernel 6.12 offers it.
LinuxURing618Linux io_uring as kernel 6.18 offers it.
BSDKqueuemacOS and FreeBSD readiness polling over kqueue.
WindowsIOCPWindows I/O completion ports.

Implements

Clone, Copy, StructuralPartialEq, PartialEq, Eq, Debug, Hash, Default, Display, FromStr
StructLogThe one log sink of a server or a client; without one, the library writes nothing anywhere.
types.rs:926 ↗
pub struct Log

Fields

callbackArc<LogCallback>Receives every line at or above level.
formatLogFormatThe line format.
levelLogLevelLines below this severity are never formatted.

Implements

Clone
EnumLogFormatThe shape of each log line.
types.rs:912 ↗
pub enum LogFormat

Values

HumanHuman-readable lines with normalized units.
JSONOne JSON document per line.

Implements

Clone, Copy, StructuralPartialEq, PartialEq, Eq, Debug, Hash, Default
EnumLogLevelThe severity of a log line, from the most verbose up.
types.rs:884 ↗
pub enum LogLevel

Values

DebugDetail for debugging UCall itself.
InfoRoutine events.
WarningSomething recoverable went wrong.
ErrorAn operation failed.
FatalThe handle cannot go on.

Implements

Clone, Copy, StructuralPartialEq, PartialEq, Eq, PartialOrd, Ord, Debug, Hash, Default
EnumVerbAn HTTP method, for routing on the server and for calls on the client.
types.rs:641 ↗
pub enum Verb

Values

UnknownUnrecognized; a routed request never carries it.
GetGET, reading a resource.
PostPOST, creating a resource or calling a procedure.
PutPUT, replacing a resource.
DeleteDELETE, removing a resource.
PatchPATCH, updating part of a resource.
OptionsOPTIONS, asking which methods a resource supports.
AnyMatches every method; valid only when registering an endpoint.

Implements

Clone, Copy, StructuralPartialEq, PartialEq, Eq, Debug, Hash