Rust · NUMA

NUMA

use forkunion::AllocationResult;use forkunion::DomainAllocator;use forkunion::PinnedVec;use forkunion::ReplicatedArray;use forkunion::ShardLocation;use forkunion::ShardedArray;use forkunion::*; 4.0.0-rc.1 · d927445f

Placing memory on a chosen domain is implemented on Linux via the mbind syscall, on FreeBSD via domainset policies, and on Windows via VirtualAllocExNuma, and degrades to a single domain elsewhere. The portable domain_allocator_t alias picks the right backend per platform - linux_numa_allocator_t, freebsd_numa_allocator_t, windows_numa_allocator_t, or a plain aligned fallback - each an STL-compatible allocator bound to one memory_domain_id_t.

Let’s say you are building a Big Data application — brute-forcing Vector Search with the NumKong library — on a 2 dual-socket CPU system, the kind that powers production USearch deployments. The first part of that program may be responsible for sharding the incoming stream of data between distinct memory regions. That part, in our simple example will be single-threaded:

#include <vector> // `std::vector`
#include <span> // `std::span`
#include <forkunion.hpp> // `linux_numa_allocator`, `machine_topology_t`, `distributed_pool_t`
#include <numkong/spatial.h> // `nk_angular_f32`, `nk_f64_t`

namespace fu = ashvardanian::forkunion;
using floats_alloc_t = fu::linux_numa_allocator<float>;

constexpr std::size_t dimensions = 768; /// Matches most BERT-like models
static std::vector<float, floats_alloc_t> first_half(floats_alloc_t(fu::memory_domain_id_t {0}));
static std::vector<float, floats_alloc_t> second_half(floats_alloc_t(fu::memory_domain_id_t {1}));
static fu::machine_topology_t machine_topology;
static fu::distributed_pool_t distributed_pool;

/// Dynamically shards incoming vectors across 2 nodes in a round-robin fashion.
void append(std::span<float, dimensions> vector) {
    bool put_in_second = first_half.size() > second_half.size();
    if (put_in_second) second_half.insert(second_half.end(), vector.begin(), vector.end());
    else first_half.insert(first_half.end(), vector.begin(), vector.end());
}

The concurrent part would involve spawning threads adjacent to every memory pool to find the best search_result_t. The primary search function, in ideal world would look like this:

  1. Each thread finds the best match within its “slice” of a compute domain, tracking the best distance and index in a local CPU register.
  2. All threads in each compute domain atomically synchronize using a domain-local instance of search_result_t.
  3. The main thread collects aggregates of partial results from all compute domains.

That is, however, overly complicated to implement. Such tree-like hierarchical reductions are optimal in a theoretical sense. Still, assuming the relative cost of spin-locking once at the end of a thread scope and the complexity of organizing the code, the more straightforward path is better. A minimal example would look like this:

/// On each compute domain we'll synchronize the threads
struct search_result_t {
    nk_f64_t best_distance {std::numeric_limits<nk_f64_t>::max()};
    std::size_t best_index {0};
};

inline search_result_t pick_best(search_result_t const& a, search_result_t const& b) noexcept {
    return a.best_distance < b.best_distance ? a : b;
}

/// Uses all CPU threads to search for the closest vector to the @p query.
search_result_t search(std::span<float, dimensions> query) {

    bool const need_to_spawn_threads = distributed_pool.threads_count() == 0;
    if (need_to_spawn_threads) {
        fu::status_t const harvested = machine_topology.harvest();
        assert(fu::succeeded(harvested) && "Failed to harvest NUMA topology");
        assert(machine_topology.memory_domains_count() == 2 && "Expected exactly 2 NUMA nodes");
        fu::status_t const spawned = distributed_pool.spawn(machine_topology);
        assert(fu::succeeded(spawned) && "Failed to spawn NUMA pools");
    }

    search_result_t result;
    fu::spin_mutex_t result_update; // ? Lighter `std::mutex` alternative w/out futexes

    std::size_t const first_vectors = first_half.size() / dimensions;
    std::size_t const total_vectors = first_vectors + second_half.size() / dimensions;

    auto slices = distributed_pool.for_slices(total_vectors,
        [&](fu::tasks_range_t tasks, fu::thread_in_domain_t) noexcept {

        search_result_t thread_local_result;
        for (std::size_t const task : tasks) {
            bool const in_second = task >= first_vectors;
            auto const &shard = in_second ? second_half : first_half;
            std::size_t const local_index = in_second ? task - first_vectors : task;

            nk_f64_t distance;
            nk_angular_f32(query.data(), shard.data() + local_index * dimensions, dimensions, &distance);
            thread_local_result = pick_best(thread_local_result, {distance, task});
        }

        // ! Still synchronizing over a shared mutex for brevity.
        std::lock_guard<fu::spin_mutex_t> lock(result_update);
        result = pick_best(result, thread_local_result);
    });
    slices.join();
    return result;
}

In a dream world, we would call distributed_pool.for_n, but there is no clean way to make the scheduling processes aware of the data distribution in an arbitrary application, so that’s left to the user. The for_slices helper pairs each run with a fu::thread_in_domain_t naming its compute domain, but slices follow the thread count rather than the shard boundary, so the callback above picks the shard per task. For more flexibility around building higher-level low-latency systems, there are unsafe APIs expecting you to manually “join” the broadcasted calls: unsafe_for_threads returns a generation token, is_complete polls it without blocking, and unsafe_join blocks until that generation completes.

The manual two-vector sharding above is what the symmetric allocators automate: symmetric_memory_allocator_t - fu_allocate_symmetric in C - maps one virtual range striped a slice per memory domain, and Rust, Zig, and Mojo wrap it as ShardedArray<T>, one shard per domain, and ReplicatedArray<T>, a full copy per domain for read-mostly data. To place and coordinate pools by hand, spawn_on - fu_pool_spawn_on in C - pins a pool to a single compute domain, while locate_thread_in and threads_count_in map a global thread index to its domain and count the workers living there.

API

StructAllocationResultResult of a memory-domain allocation; carries only the OS ID, so it can outlive the topology.8
allocators.rs:45 ↗
pub struct AllocationResult

Implements

Debug, Drop, Send, Sync
as_mut_sliceReturns the allocated memory as a mutable byte slice.
allocators.rs:57 ↗
pub fn as_mut_slice(&mut self) -> &mut [u8]
as_sliceReturns the allocated memory as an immutable byte slice.
allocators.rs:63 ↗
pub fn as_slice(&self) -> &[u8]
as_ptrReturns the raw pointer to the allocated memory.
allocators.rs:69 ↗
pub fn as_ptr(&self) -> *mut u8
allocated_bytesReturns the number of bytes actually allocated, which may be larger than requested.
allocators.rs:75 ↗
pub fn allocated_bytes(&self) -> usize
bytes_per_pageReturns the page size used for this allocation.
allocators.rs:81 ↗
pub fn bytes_per_page(&self) -> usize
memory_domain_idReturns the OS ID of the memory domain this memory was allocated on.
allocators.rs:87 ↗
pub fn memory_domain_id(&self) -> MemoryDomainId
as_mut_slice_ofConverts a typed slice into the allocation's memory space.
allocators.rs:100 ↗
pub unsafe fn as_mut_slice_of<T>(&mut self) -> &mut [T]

Safety

The caller must ensure that:

  • T has the correct alignment for the allocated memory
  • The allocation is large enough to hold the requested number of T elements
  • The memory is properly initialized before use
as_slice_ofConverts a typed slice into the allocation's memory space, immutably.
allocators.rs:114 ↗
pub unsafe fn as_slice_of<T>(&self) -> &[T]

Safety

The caller must ensure that:

  • T has the correct alignment for the allocated memory
  • The allocation contains valid data of type T
StructDomainAllocatorAn allocator bound to a single memory domain, so every block it hands out is domain-local.6
allocators.rs:160 ↗
pub struct DomainAllocator

On a machine whose memory access latency varies with the domain a block lives on, keeping an allocation on the domain of the cores that read it avoids the cross-domain penalty.

Examples

use forkunion::*;
let topology = Topology::new().unwrap();
let id = topology.memory_domain_id_at_index(MemoryDomain(0)).unwrap();
let allocator = DomainAllocator::new(id).expect("failed to bind allocator to memory domain 0");
let allocation = allocator.allocate(1024).expect("failed to allocate 1024 bytes");

// Access the allocated memory
let memory_slice = allocation.as_slice();
assert_eq!(memory_slice.len(), 1024);
println!("Allocated {} bytes on memory domain {}",
         allocation.allocated_bytes(), allocation.memory_domain_id().get());

Implements

Debug, Clone, Copy, Allocator
newCreates an allocator bound to the memory domain named by memory_domain_id.
allocators.rs:193 ↗
pub fn new(memory_domain_id: MemoryDomainId) -> Result<Self>

Arguments

Errors

Returns None if the ID is the -1 sentinel, i.e. it names no domain.

Examples

use forkunion::*;

let topology = Topology::new().unwrap();
// Bind an allocator to the first memory domain
let id = topology.memory_domain_id_at_index(MemoryDomain(0)).unwrap();
let allocator = DomainAllocator::new(id).expect("memory domain 0 should be available");

// Bind another to a second memory domain when the machine has one
let memory_domains = topology.memory_domains_count().unwrap();
if memory_domains > 1 {
    let id2 = topology.memory_domain_id_at_index(MemoryDomain(1)).unwrap();
    let allocator2 = DomainAllocator::new(id2).expect("memory domain 1 should be available");
    println!("Bound allocator to memory domain: {}", allocator2.memory_domain_id().get());
}
memory_domain_idReturns the OS ID of the memory domain this allocator is bound to.
allocators.rs:206 ↗
pub fn memory_domain_id(&self) -> MemoryDomainId
allocate_at_leastAllocates memory with at least the requested size on this allocator's memory domain.
allocators.rs:242 ↗
pub fn allocate_at_least(&self, minimum_bytes: usize) -> Result<AllocationResult>

Returns both the actual allocated size and page size information, which can be useful for optimizing memory access patterns.

Arguments

  • minimum_bytes - The minimum number of bytes to allocate

Errors

Reports InvalidArgument for a zero request, and the core's own status otherwise.

Examples

use forkunion::*;

let topology = Topology::new().unwrap();
let allocator = DomainAllocator::new(topology.memory_domain_id_at_index(MemoryDomain(0)).unwrap()).unwrap();
let allocation = allocator.allocate_at_least(1024).expect("Failed to allocate memory");

println!("Requested 1024 bytes, got {} bytes on {} byte pages",
         allocation.allocated_bytes(), allocation.bytes_per_page());

// The allocation might be larger than requested due to page alignment
assert!(allocation.allocated_bytes() >= 1024);

// Access the memory as a byte slice
let memory = allocation.as_slice();
println!("Can access {} bytes of memory", memory.len());
allocateAllocates exactly the requested number of bytes on this allocator's memory domain.
allocators.rs:303 ↗
pub fn allocate(&self, bytes: usize) -> Result<AllocationResult>

Arguments

  • bytes - The exact number of bytes to allocate

Errors

Reports InvalidArgument for a zero request, and the core's own status otherwise.

Examples

use forkunion::*;

let topology = Topology::new().unwrap();
let id = topology.memory_domain_id_at_index(MemoryDomain(0)).unwrap();
let allocator = DomainAllocator::new(id).unwrap();
let allocation = allocator.allocate(1024).expect("Failed to allocate memory");
assert_eq!(allocation.allocated_bytes(), 1024);

// Write some data to the allocated memory
let mut allocation = allocation; // Make mutable
let memory = allocation.as_mut_slice();
memory[0] = 42;
memory[1023] = 255;

// Verify the data was written
assert_eq!(memory[0], 42);
assert_eq!(memory[1023], 255);
allocate_forAllocates memory for a specific number of elements of type T.
allocators.rs:351 ↗
pub fn allocate_for<T>(&self, count: usize) -> Result<AllocationResult>

Arguments

  • count - The number of elements to allocate space for

Examples

use forkunion::*;

let topology = Topology::new().unwrap();
let allocator = DomainAllocator::new(topology.memory_domain_id_at_index(MemoryDomain(0)).unwrap()).unwrap();
let mut allocation = allocator.allocate_for::<u64>(100).expect("Failed to allocate");

// Verify the allocation size first
assert_eq!(allocation.allocated_bytes(), 100 * std::mem::size_of::<u64>());

// Access as typed slice
let slice = unsafe { allocation.as_mut_slice_of::<u64>() };
slice[0] = 42;
slice[99] = 12345;

// Read back the values
assert_eq!(slice[0], 42);
assert_eq!(slice[99], 12345);
allocate_for_at_leastAllocates memory for at least the specified number of elements of type T.
allocators.rs:395 ↗
pub fn allocate_for_at_least<T>(&self, min_count: usize) -> Result<AllocationResult>

This function may allocate more elements than requested for optimal page alignment.

Arguments

  • min_count - The minimum number of elements to allocate space for

Examples

use forkunion::*;

let topology = Topology::new().unwrap();
let allocator = DomainAllocator::new(topology.memory_domain_id_at_index(MemoryDomain(0)).unwrap()).unwrap();
let mut allocation = allocator.allocate_for_at_least::<u32>(1000).expect("Failed to allocate");
let actual_count = allocation.allocated_bytes() / std::mem::size_of::<u32>();
println!("Requested {} u32s, got space for {} u32s", 1000, actual_count);

// The allocation provides at least the requested number of elements
assert!(actual_count >= 1000);

// Initialize the allocated memory
let slice = unsafe { allocation.as_mut_slice_of::<u32>() };
for i in 0..1000 {
    slice[i] = i as u32;
}

// Verify initialization
assert_eq!(slice[0], 0);
assert_eq!(slice[999], 999);
StructPinnedVecA Vec-like container that uses NUMA-aware pinned memory allocation.34
allocators.rs:491 ↗
pub struct PinnedVec<T>

PinnedVec<T> provides a dynamic array that allocates memory on a specific memory domain, which should correspond to a compute_domain_index for optimal performance with ThreadPool. It automatically manages growth and shrinkage.

Examples

use forkunion::*;

// Create a vector on memory domain 0
let topology = Topology::new().unwrap();
let allocator = DomainAllocator::new(topology.memory_domain_id_at_index(MemoryDomain(0)).unwrap()).expect("Failed to create alloc");
let mut vec = PinnedVec::<u64>::new_in(allocator);

// Add elements
vec.push(42).expect("Failed to push");
vec.push(100).expect("Failed to push");

// Access elements
assert_eq!(vec.len(), 2);
assert_eq!(vec[0], 42);
assert_eq!(vec[1], 100);

// Iterate over elements
for (i, &value) in vec.iter().enumerate() {
    println!("Element {}: {}", i, value);
}

Implements

Debug, Index<usize>, IndexMut<usize>
new_inCreates a new empty PinnedVec using the specified allocator.
allocators.rs:515 ↗
pub fn new_in(allocator: DomainAllocator) -> Self

Arguments

Examples

use forkunion::*;

let topology = Topology::new().unwrap();
let allocator = DomainAllocator::new(topology.memory_domain_id_at_index(MemoryDomain(0)).unwrap()).expect("Failed to create alloc");
let vec = PinnedVec::<i32>::new_in(allocator);
assert_eq!(vec.len(), 0);
assert_eq!(vec.capacity(), 0);
with_capacity_inCreates a new PinnedVec with the specified capacity using the given allocator.
allocators.rs:543 ↗
pub fn with_capacity_in(allocator: DomainAllocator, capacity: usize) -> Result<Self>

Arguments

Errors

Returns None if allocation fails.

Examples

use forkunion::*;

let topology = Topology::new().unwrap();
let allocator = DomainAllocator::new(topology.memory_domain_id_at_index(MemoryDomain(0)).unwrap()).expect("Failed to create alloc");
let vec = PinnedVec::<i32>::with_capacity_in(allocator, 100).expect("Failed to create vec");
assert_eq!(vec.len(), 0);
assert_eq!(vec.capacity(), 100);
lenReturns the number of elements in the vector.
allocators.rs:551 ↗
pub fn len(&self) -> usize
is_emptyReturns true if the vector contains no elements.
allocators.rs:557 ↗
pub fn is_empty(&self) -> bool
capacityReturns the number of elements the vector can hold without reallocating.
allocators.rs:563 ↗
pub fn capacity(&self) -> usize
memory_domain_idReturns the OS ID of the memory domain this vector's memory is allocated on.
allocators.rs:569 ↗
pub fn memory_domain_id(&self) -> MemoryDomainId
reserveReserves capacity for at least additional more elements.
allocators.rs:594 ↗
pub fn reserve(&mut self, additional: usize) -> Result<()>

Arguments

  • additional - The number of additional elements to reserve space for

Errors

Returns an error if allocation fails.

Examples

use forkunion::*;

let topology = Topology::new().unwrap();
let allocator = DomainAllocator::new(topology.memory_domain_id_at_index(MemoryDomain(0)).unwrap()).expect("Failed to create alloc");
let mut vec = PinnedVec::<i32>::new_in(allocator);
vec.reserve(10).expect("Failed to reserve");
assert!(vec.capacity() >= 10);
pushAppends an element to the back of the vector.
allocators.rs:620 ↗
pub fn push(&mut self, value: T) -> Result<()>

Arguments

  • value - The element to append

Errors

Returns an error if allocation fails when growing the vector.

Examples

use forkunion::*;

let topology = Topology::new().unwrap();
let allocator = DomainAllocator::new(topology.memory_domain_id_at_index(MemoryDomain(0)).unwrap()).expect("Failed to create alloc");
let mut vec = PinnedVec::<i32>::new_in(allocator);
vec.push(42).expect("Failed to push");
assert_eq!(vec.len(), 1);
assert_eq!(vec[0], 42);
popRemoves the last element from the vector and returns it.
allocators.rs:644 ↗
pub fn pop(&mut self) -> Option<T>

Returns

The last element, or None if the vector is empty.

Examples

use forkunion::*;

let topology = Topology::new().unwrap();
let allocator = DomainAllocator::new(topology.memory_domain_id_at_index(MemoryDomain(0)).unwrap()).expect("Failed to create alloc");
let mut vec = PinnedVec::<i32>::new_in(allocator);
vec.push(42).expect("Failed to push");
assert_eq!(vec.pop(), Some(42));
assert_eq!(vec.pop(), None);
clearClears the vector, removing all values.
allocators.rs:662 ↗
pub fn clear(&mut self)

Examples

use forkunion::*;

let topology = Topology::new().unwrap();
let allocator = DomainAllocator::new(topology.memory_domain_id_at_index(MemoryDomain(0)).unwrap()).expect("Failed to create alloc");
let mut vec = PinnedVec::<i32>::new_in(allocator);
vec.push(42).expect("Failed to push");
vec.clear();
assert_eq!(vec.len(), 0);
as_ptrReturns a raw pointer to the vector's buffer.
allocators.rs:668 ↗
pub fn as_ptr(&self) -> *const T
as_mut_ptrReturns a mutable raw pointer to the vector's buffer.
allocators.rs:674 ↗
pub fn as_mut_ptr(&mut self) -> *mut T
sync_ptrReturns a synchronization-friendly mutable pointer wrapper.
allocators.rs:682 ↗
pub fn sync_ptr(&self) -> SyncMutPtr<T>

The pointer may be shared between threads as long as each accesses disjoint indices.

as_sliceReturns a slice containing the entire vector.
allocators.rs:688 ↗
pub fn as_slice(&self) -> &[T]
as_mut_sliceReturns a mutable slice containing the entire vector.
allocators.rs:694 ↗
pub fn as_mut_slice(&mut self) -> &mut [T]
iterReturns an iterator over the vector.
allocators.rs:699 ↗
pub fn iter(&self) -> Iter<'_, T>
iter_mutReturns a mutable iterator over the vector.
allocators.rs:704 ↗
pub fn iter_mut(&mut self) -> IterMut<'_, T>
par_iterCreates a read-only parallel slice view over the vector.
allocators.rs:710 ↗
pub fn par_iter(&self) -> ParallelSlice<'_, T>
par_iter_mutCreates a mutable parallel slice view over the vector.
allocators.rs:716 ↗
pub fn par_iter_mut(&mut self) -> ParallelSliceMut<'_, T>
where
    T: Send,
insertInserts an element at position index, shifting all elements after it to the right.
allocators.rs:737 ↗
pub fn insert(&mut self, index: usize, element: T) -> Result<()>

Arguments

  • index - The position to insert at
  • element - The element to insert

Panics

Panics if index > len.

Errors

Returns an error if allocation fails when growing the vector.

removeRemoves and returns the element at position index, shifting later elements left.
allocators.rs:760 ↗
pub fn remove(&mut self, index: usize) -> T

Arguments

  • index - The position to remove from

Panics

Panics if index >= len.

extend_from_sliceExtend the vector by cloning elements from a slice.
allocators.rs:773 ↗
pub fn extend_from_slice(&mut self, other: &[T]) -> Result<()>
where
    T: Clone,

Arguments

  • other - The slice to copy elements from

Errors

Returns an error if allocation fails when growing the vector.

getReturns a reference to an element or subslice depending on the type of index.
allocators.rs:783 ↗
pub fn get<I>(&self, index: I) -> Option<&<I as SliceIndex<[T]>>::Output>
where
    I: SliceIndex<[T]>,
get_mutReturns a mutable reference to an element or subslice depending on the type of index.
allocators.rs:791 ↗
pub fn get_mut<I>(&mut self, index: I) -> Option<&mut <I as SliceIndex<[T]>>::Output>
where
    I: SliceIndex<[T]>,
*× 2Returns a reference to the first element of the vector, or None if it is empty.
allocators.rs:799 ↗
pub fn first(&self) -> Option<&T>
Symbol
first
last
*_mut× 2Returns a mutable reference to the first element of the vector, or None if it is empty.
allocators.rs:804 ↗
pub fn first_mut(&mut self) -> Option<&mut T>
Symbol
first_mut
last_mut
swapSwaps two elements in the vector.
allocators.rs:819 ↗
pub fn swap(&mut self, a: usize, b: usize)
reverseReverses the order of elements in the vector, in place.
allocators.rs:824 ↗
pub fn reverse(&mut self)
containsReturns true if the vector contains an element with the given value.
allocators.rs:830 ↗
pub fn contains(&self, x: &T) -> bool
where
    T: PartialEq,
truncateShortens the vector, keeping the first len elements and dropping the rest.
allocators.rs:838 ↗
pub fn truncate(&mut self, len: usize)
resizeResizes the vector in-place so that len is equal to new_len.
allocators.rs:843 ↗
pub fn resize(&mut self, new_len: usize, value: T) -> Result<()>
where
    T: Clone,
resize_withResizes the vector in-place so that len is equal to new_len.
allocators.rs:853 ↗
pub fn resize_with<F>(&mut self, new_len: usize, f: F) -> Result<()>
where
    F: FnMut() -> T,
fillFills the vector with copies of the given value.
allocators.rs:880 ↗
pub fn fill(&mut self, value: T)
where
    T: Clone,

Arguments

  • value - The value to fill the vector with

Examples

use forkunion::*;

let topology = Topology::new().unwrap();
let allocator = DomainAllocator::new(topology.memory_domain_id_at_index(MemoryDomain(0)).unwrap()).expect("Failed to create alloc");
let mut vec = PinnedVec::<i32>::with_capacity_in(allocator, 5).expect("Failed to create vec");
vec.resize(5, 0).expect("Failed to resize");
vec.fill(42);
assert_eq!(vec.as_slice(), &[42, 42, 42, 42, 42]);
fill_withFills the vector with values generated by calling a closure repeatedly.
allocators.rs:905 ↗
pub fn fill_with<F>(&mut self, f: F)
where
    F: FnMut() -> T,

Arguments

  • f - A closure that generates values to fill the vector with

Examples

use forkunion::*;

let topology = Topology::new().unwrap();
let allocator = DomainAllocator::new(topology.memory_domain_id_at_index(MemoryDomain(0)).unwrap()).expect("Failed to create alloc");
let mut vec = PinnedVec::<i32>::with_capacity_in(allocator, 5).expect("Failed to create vec");
vec.resize(5, 0).expect("Failed to resize");
vec.fill_with(|| 42);
assert_eq!(vec.as_slice(), &[42, 42, 42, 42, 42]);
StructReplicatedArrayOne length-n copy of a sequence per memory domain, so every thread reads a node-local replica.10
allocators.rs:990 ↗
pub struct ReplicatedArray<T>

A thin owner of one symmetric mapping - replica d lives at base + d * stride_bytes() and holds n elements, each starting as T::default(): the caller fills every replica and keeps them coherent, mirroring the C++ replicated_array.

Implements

Default, Send, Sync
new_inAllocates one length-n replica per domain, each element T::default().
allocators.rs:998 ↗
pub fn new_in(topology: &Topology, n: usize) -> Result<Self>
newAn empty array holding no mapping.
allocators.rs:1020 ↗
pub const fn new() -> Self
lenThe logical length of each replica.
allocators.rs:1030 ↗
pub fn len(&self) -> usize
is_*_empty× 2Whether the array holds no elements.
allocators.rs:1036 ↗
pub fn is__empty(&self) -> bool
memory_domains_countThe number of per-domain replicas.
allocators.rs:1042 ↗
pub fn memory_domains_count(&self) -> usize
stride_bytesThe page-aligned byte distance between consecutive replicas.
allocators.rs:1048 ↗
pub fn stride_bytes(&self) -> usize
replica_ptrRaw pointer to the replica start on memory_domain, for concurrent first-touch fills.
allocators.rs:1054 ↗
pub fn replica_ptr(&self, memory_domain: MemoryDomain) -> *mut T
on_memory_domainThe whole replica living on memory_domain.
allocators.rs:1061 ↗
pub fn on_memory_domain(&self, memory_domain: MemoryDomain) -> &[T]
on_memory_domain_mutThe whole replica living on memory_domain, mutably.
allocators.rs:1067 ↗
pub fn on_memory_domain_mut(&mut self, memory_domain: MemoryDomain) -> &mut [T]
atOne element of the replica on memory_domain.
allocators.rs:1073 ↗
pub fn at(&self, memory_domain: MemoryDomain, local_index: usize) -> &T
StructShardLocationWhere a logical element lives - which memory domain, and its index within that shard.
allocators.rs:1089 ↗
pub struct ShardLocation

Fields

memory_domainMemoryDomainThe memory domain holding the element.
local_indexusizeThe element's index within that domain's shard.

Implements

Copy, Clone, Debug, StructuralPartialEq, PartialEq, Eq
StructShardedArrayA sequence partitioned across memory domains as contiguous segments, each element stored once.13
allocators.rs:1102 ↗
pub struct ShardedArray<T>

Each domain owns a contiguous logical segment of segment() elements, so element i lives at {i / segment(), i % segment()} - see location_of - and a scan of a domain's shard is sequential in memory. Backed by one symmetric mapping; the trailing shard may be short, and each element starts as T::default(), mirroring ReplicatedArray.

Implements

Default, Send, Sync
new_inAllocates n elements partitioned round-robin across the domains, each T::default().
allocators.rs:1111 ↗
pub fn new_in(topology: &Topology, n: usize) -> Result<Self>
newAn empty array holding no mapping.
allocators.rs:1142 ↗
pub const fn new() -> Self
lenThe logical length, summed across the shards.
allocators.rs:1153 ↗
pub fn len(&self) -> usize
memory_domains_countThe number of shards, one per memory domain.
allocators.rs:1165 ↗
pub fn memory_domains_count(&self) -> usize
stride_bytesThe page-aligned byte distance between consecutive shards.
allocators.rs:1171 ↗
pub fn stride_bytes(&self) -> usize
segmentThe contiguous logical segment size per domain - ceil(n / memory_domains_count()).
allocators.rs:1177 ↗
pub fn segment(&self) -> usize
length_on_memory_domainHow many elements the shard on memory_domain holds - a trailing shard may be shorter.
allocators.rs:1183 ↗
pub fn length_on_memory_domain(&self, memory_domain: MemoryDomain) -> usize
location_ofThe memory domain and local index that store logical element logical_index.
allocators.rs:1196 ↗
pub fn location_of(&self, logical_index: usize) -> ShardLocation

Requires a non-empty array - segment() is the divisor and is zero when empty.

logical_index_ofLogical index of the element at local_index on memory_domain - inverse of location_of.
allocators.rs:1205 ↗
pub fn logical_index_of(&self, memory_domain: MemoryDomain, local_index: usize) -> usize
shard_ptrRaw start of the shard on memory_domain, for concurrent fills through a raw pointer.
allocators.rs:1211 ↗
pub fn shard_ptr(&self, memory_domain: MemoryDomain) -> *mut T
on_memory_domainThe whole shard living on memory_domain.
allocators.rs:1218 ↗
pub fn on_memory_domain(&self, memory_domain: MemoryDomain) -> &[T]
on_memory_domain_mutThe whole shard living on memory_domain, mutably.
allocators.rs:1229 ↗
pub fn on_memory_domain_mut(&mut self, memory_domain: MemoryDomain) -> &mut [T]
atThe single home of a logical element.
allocators.rs:1236 ↗
pub fn at(&self, memory_domain: MemoryDomain, local_index: usize) -> &T

Functions

default_pinned_allocatorCreates an allocator for memory domain 0, used when the specific domain does not matter.
allocators.rs:451 ↗
pub fn default_pinned_allocator(topology: &Topology) -> Result<DomainAllocator>

Examples

use forkunion::*;

let topology = Topology::new().unwrap();
let allocator = default_pinned_allocator(&topology).expect("No memory domains available");
let allocation = allocator.allocate(1024).expect("Failed to allocate");

// The default allocator uses the first memory domain
assert_eq!(allocation.memory_domain_id(), topology.memory_domain_id_at_index(MemoryDomain(0)).unwrap());

// For more control, pin to a specific domain
let domains = topology.memory_domains_count().unwrap();
println!("System has {} memory domains available", domains);

if domains > 1 {
    let allocator_domain1 = DomainAllocator::new(topology.memory_domain_id_at_index(MemoryDomain(1)).unwrap()).expect("domain 1 available");
    let allocation2 = allocator_domain1.allocate(2048).expect("Failed to allocate on domain 1");
    assert_eq!(allocation2.memory_domain_id(), topology.memory_domain_id_at_index(MemoryDomain(1)).unwrap());
}