Threads and Polling

The package starts no threads and needs no event loop of its own: every handler runs on a thread the application lent to a polling call.

• A route handler runs on the thread inside take_call, take_calls or run for the event loop that received the request.
• A batch handler runs on the thread inside take_batch or take_batches for its endpoint, never on an event loop.
• A server’s peer authorizer runs on the polling thread completing the handshake, and a client’s on the client’s own thread inside take_completed or a blocking call.

Each event loop and each batch endpoint has one polling owner at a time. take_calls and take_batches run until server.stop(), while take_call and take_batch return after one round, waiting up to their timeout, so a loop of them interleaves polling with other work on the same thread. server.stop() works from any thread, handlers included, and wakes every polling thread; join them before releasing the server. The first stop drains, answering every request already held before its connection closes, and a second one cuts the replies still in flight. An interface accepts on its own number of event loops, one by default, and server.event_loops reports the largest of those counts.

import threading

import ucall

server = ucall.Server(ucall.Interface("127.0.0.1:0", event_loops=4))
threads = [threading.Thread(target=server.take_calls, args=(loop,)) for loop in range(server.event_loops)]
threads += [threading.Thread(target=endpoint.take_batches) for endpoint in server.batch_endpoints]
for thread in threads:
    thread.start()
server.stop()
for thread in threads:
    thread.join()

Every polling call releases the interpreter lock while it waits on the network and takes it back only to run a Python handler, authorizer or batch handler. So loops parse, route and send in parallel, while Python handlers on several loops take turns on the lock unless they release it themselves, as NumPy does in its kernels. On a free-threaded interpreter there is no lock to take turns on, and handlers on several loops run at once. Signals reach Python only on the main thread between calls, so take_calls and take_batches belong on other threads.

server.run() covers the simplest server, one event loop and no batch endpoints, on the calling thread, and raises RuntimeError for anything larger. It polls in slices of 100 milliseconds and checks for signals between them, so Ctrl+C raises KeyboardInterrupt from it, and once stopped it lets the loop finish the requests in flight before returning. The blocking client calls wait in the same slices, so Ctrl+C interrupts them too, and the interrupted call completes later through take_completed.

A Client belongs to the thread that created it, and any other thread calling it gets ucall.Error with Status.INVALID_ARGUMENT, so a program calling from several threads makes one client per thread. A Request and a Response move freely between threads. Every thread that polls, replies or reads must run on a CPU supporting the CPU class the server or client resolved at start, so CPUClass.SERIAL is the safe choice on machines mixing CPU generations.