typedef uint64_t ustore_size_t; § Conventions
Every call takes a single pointer to an argument struct named after it with a _t suffix, so arguments are named and every omitted one is zero.
Every struct opens with its handle, then executor, outcome and completion, so every call reports through a ustore_outcome_t whose status names the result and whose static message is never freed.
Statuses read ustore_status_<name>_k, from ustore_status_success_k through every failure.
Every closed set is a C enum, and each one a binding spells has a pair of functions: <prefix>_<enum>_name, such as ustore_isolation_name, answers a static lowercase spelling like strict-serializable, or unrecognized for a value no enumerator carries, and <prefix>_<enum>_from_name reads a ustore_bytes_view_t back into its output, failing with ustore_status_invalid_argument_k for a name it does not know. ustore_version_name spells the version of the library the program runs against, like 1.0.0.
Every call is a batch of count tasks, and each of its inputs takes one of three shapes:
- One run of bytes, a
ustore_bytes_view_tof a pointer and a length, carries a path or a name, never null-terminated by contract. - A strided run of fixed-width elements, such as a
ustore_keys_view_tof a pointer and a stride in bytes, carries one element per task, and a zero stride repeats the first element for every task. - A ragged batch, a
ustore_strings_view_t, carries one string per task as an Apache Arrow tape of 32-bit or 64-bit offsets, or through an accessor callback over any other shape.
A zeroed view of any shape stands for the input’s default.
Every output whose length scales with the call lands packed in the arena the call is given, so no output carries a capacity or a stride, and ragged bytes come back as a tape.
Keys are ustore_key_t, 16 bytes of a UUID compared bytewise, and a non-negative integer spells as its 128-bit big-endian bytes, as Python’s uuid.UUID(int=...) does, so keys made from integers order as the integers.
The examples below share these handles, the arena made once for calls on the calling thread:
ustore_database_t database = NULL;
ustore_executor_t executor = NULL;
ustore_outcome_t outcome = {0};
ustore_arena_t arena = NULL;
ustore_arena_init(&(ustore_arena_init_t){.arena = &arena, .outcome = &outcome});§ Opening the Database
#include <ustore/ustore.h>
char const directory[] = "/var/lib/ustore";
ustore_database_open_rocksdb(&(ustore_database_open_rocksdb_t){
.database = &database,
.outcome = &outcome,
.directory = {(uint8_t const*)directory, sizeof(directory) - 1},
});
if (outcome.status != ustore_status_success_k)
fprintf(stderr, "%s: %s\n", ustore_status_name(outcome.status), outcome.message);
ustore_arena_free(arena);
ustore_database_free(database);
ustore_executor_free(executor);Each engine opens through its own calls, whose arguments hold what that engine can be asked, and a zero field keeps the engine’s default:
ustore_database_init_ramstarts an empty database in memory alone, whileustore_database_open_ramserves a directory as one image, loading it at open and rewriting it whole on a flushed commit and on close.ustore_database_open_rocksdbandustore_database_open_leveldbserve a directory in place, creating the database when the directory holds none.ustore_database_open_nvmeserves the database its manifest names,<database>.ustore.jsonin an existing directory or the raw address of its first slot, such as/dev/nvme0n1@64GiB, creating the database when the manifest is absent.ustore_database_view_rocksdbserves an existing RocksDB directory read-only, refusing every call that would change it withustore_status_read_only_k: writes, removes, transactions, collection creates and drops, clears and compactions, while snapshots and their saves still serve.
The opens of an engine the build lacks answer ustore_status_unsupported_k, so a program links against every build.
Every freeing function accepts NULL.
API
Types
ustore_size_tAn unsigned count or size, 64 bits wide on every platform.
ustore_bytes_view_tA borrowed run of bytes and its length, never null-terminated by contract, so a caller holding a string without a terminator passes its own storage.
struct ustore_bytes_view_t Fields
data | uint8_t const * | First byte of the run. |
|---|---|---|
length | ustore_size_t | Bytes the run spans, with no terminator assumed or written. |
ustore_database_tAn opaque handle to an open database, safe to share across threads from open until free.
typedef struct ustore_database_handle_t * ustore_database_t; An embedded database owns its directory, so no two processes may open the same one at once. A database holds no collection until one is created, and every call addresses the collections it created by identifier.
ustore_byte_tThe elementary piece of a binary value.
typedef uint8_t ustore_byte_t; ustore_octet_tThe smallest bitset, holding eight bits.
typedef uint8_t ustore_octet_t; ustore_engine_tThe engine serving a database, as ustore_database_describe reports it.
enum ustore_engine_t Values
ustore_engine_ram_k | 0 | The in-memory engine of ustore_database_init_ram and ustore_database_open_ram. |
|---|---|---|
ustore_engine_rocksdb_k | 1 | The RocksDB engine of ustore_database_open_rocksdb and ustore_database_view_rocksdb. |
ustore_engine_leveldb_k | 2 | The LevelDB engine of ustore_database_open_leveldb. |
ustore_engine_nvme_k | 3 | The NVMe engine of ustore_database_open_nvme. |
Spellings
char const * ustore_engine_name(ustore_engine_t engine);ustore_engine_t ustore_engine_from_name(ustore_bytes_view_t name);ustore_data_directory_tA directory the data files may occupy, with the bytes they may take there.
struct ustore_data_directory_t Fields
path | ustore_bytes_view_t | The directory, which must exist; on NVMe also a raw device from a base, as in /dev/nvme0n1@64GiB. |
|---|---|---|
max_bytes | ustore_size_t | Bytes the files may take there, zero being unlimited on RocksDB and the whole medium on NVMe. |
Functions
ustore_database_init_ramStarts an empty database of the in-memory engine, which persists nowhere.
void ustore_database_init_ram(ustore_database_init_ram_t *arguments); Parameters
databaseustore_database_t * | Output for the new database, left NULL on failure. | |
|---|---|---|
executorustore_executor_t | Executor that runs the call, where NULL runs it on the calling thread. | |
outcomeustore_outcome_t * | Output for the outcome, final once its status leaves ustore_status_deferred_k. | |
completionustore_completion_t | Whether the call returns final, the default, or once submitted. | |
isolationustore_isolation_t | How transactions see each other. | |
cpu_classustore_cpu_class_t | The CPU class the database confines its waits, atomics and string kernels to, where zero takes what the CPU offers. |
Statuses
ustore_status_device_kThe executor computes on a device.ustore_status_contended_kThe executor's deferred table is full, or a callback blocks on the executor running it.ustore_status_invalid_argument_kThedatabaseoutput isNULL, or the isolation or the CPU class is unknown.ustore_status_unsupported_kThe CPU does not run thecpu_class.
ustore_database_open_ramServes a directory in place through the in-memory engine, loading the image it holds, if any, and rewriting it whole on a flushed commit and on close.
void ustore_database_open_ram(ustore_database_open_ram_t *arguments); Parameters
databaseustore_database_t * | Output for the opened database, left NULL on failure. | |
|---|---|---|
executorustore_executor_t | Executor that runs the call, where NULL runs it on the calling thread. | |
outcomeustore_outcome_t * | Output for the outcome, final once its status leaves ustore_status_deferred_k. | |
completionustore_completion_t | Whether the call returns final, the default, or once submitted. | |
directoryustore_bytes_view_t | Directory the database persists in as one image, which must exist. | |
isolationustore_isolation_t | How transactions see each other. | |
cpu_classustore_cpu_class_t | The CPU class the database confines its waits, atomics and string kernels to, where zero takes what the CPU offers. |
Statuses
ustore_status_device_kThe executor computes on a device.ustore_status_contended_kThe executor's deferred table is full, or a callback blocks on the executor running it.ustore_status_invalid_argument_kThedirectoryis empty, thedatabaseoutput isNULL, or the isolation or the CPU class is unknown.ustore_status_unsupported_kThe CPU does not run thecpu_class.ustore_status_not_found_kThe directory does not exist.ustore_status_malformed_kThe image has an unknown format, is truncated or names an unknown modality.
ustore_database_open_rocksdbServes a directory in place through the RocksDB engine, creating the database when it holds none.
void ustore_database_open_rocksdb(
ustore_database_open_rocksdb_t *arguments); Parameters
databaseustore_database_t * | Output for the opened database, left NULL on failure. | |
|---|---|---|
executorustore_executor_t | Executor that runs the call, where NULL runs it on the calling thread. | |
outcomeustore_outcome_t * | Output for the outcome, final once its status leaves ustore_status_deferred_k. | |
completionustore_completion_t | Whether the call returns final, the default, or once submitted. | |
directoryustore_bytes_view_t | Directory of the database, which must exist. | |
data_directoriesustore_data_directory_t const * | Directories the table files fill in order, or null to keep them in directory. | |
data_directories_countustore_size_t | Entries of data_directories. |
Statuses
ustore_status_unsupported_kThe library was built without the engine.ustore_status_device_kThe executor computes on a device.ustore_status_contended_kThe executor's deferred table is full, or a callback blocks on the executor running it.ustore_status_invalid_argument_kThedatabaseoutput isNULL, thedirectoryis empty, or the counteddata_directoriesareNULLor one has an empty path.ustore_status_not_found_kThe directory does not exist.ustore_status_malformed_kA column family or a collections entry is of another layout or corrupt.
ustore_database_view_rocksdbServes a directory in place through the RocksDB engine, read-only.
void ustore_database_view_rocksdb(
ustore_database_view_rocksdb_t *arguments); The database reads the state its directory held at open, and refuses every call that would change it with ustore_status_read_only_k: writes, removes, transactions, collection creates and drops, clears and compactions. Snapshots and their saves still serve.
Parameters
databaseustore_database_t * | Output for the opened database, left NULL on failure. | |
|---|---|---|
executorustore_executor_t | Executor that runs the call, where NULL runs it on the calling thread. | |
outcomeustore_outcome_t * | Output for the outcome, final once its status leaves ustore_status_deferred_k. | |
completionustore_completion_t | Whether the call returns final, the default, or once submitted. | |
directoryustore_bytes_view_t | Directory of an existing database. | |
data_directoriesustore_data_directory_t const * | Directories the table files fill, as the database was opened with, or null to find them in directory. | |
data_directories_countustore_size_t | Entries of data_directories. |
Statuses
ustore_status_unsupported_kThe library was built without the engine.ustore_status_device_kThe executor computes on a device.ustore_status_contended_kThe executor's deferred table is full, or a callback blocks on the executor running it.ustore_status_invalid_argument_kThedatabaseoutput isNULL, thedirectoryis empty, or the counteddata_directoriesareNULLor one has an empty path.ustore_status_not_found_kThe directory does not exist.ustore_status_malformed_kA column family or a collections entry is of another layout or corrupt, the system collections are missing, or a crashed create or drop left a family behind.
ustore_database_open_leveldbServes a directory in place through the LevelDB engine, creating the database when it holds none.
void ustore_database_open_leveldb(
ustore_database_open_leveldb_t *arguments); Parameters
databaseustore_database_t * | Output for the opened database, left NULL on failure. | |
|---|---|---|
executorustore_executor_t | Executor that runs the call, where NULL runs it on the calling thread. | |
outcomeustore_outcome_t * | Output for the outcome, final once its status leaves ustore_status_deferred_k. | |
completionustore_completion_t | Whether the call returns final, the default, or once submitted. | |
directoryustore_bytes_view_t | Directory of the database, which must exist. |
Statuses
ustore_status_unsupported_kThe library was built without the engine.ustore_status_device_kThe executor computes on a device.ustore_status_contended_kThe executor's deferred table is full, or a callback blocks on the executor running it.ustore_status_invalid_argument_kThedatabaseoutput isNULL, or thedirectoryis empty.ustore_status_not_found_kThe directory does not exist.
ustore_database_open_nvmeServes the database a manifest names through the NVMe engine, creating it when the manifest is absent.
void ustore_database_open_nvme(ustore_database_open_nvme_t *arguments); ustore_status_device_kThe executor computes on a device, or thecompactionsexecutor is no VRAM one.ustore_status_contended_kThe executor's deferred table is full, or a callback blocks on the executor running it.ustore_status_invalid_argument_kThedatabaseoutput isNULL, thedirectoryis empty or names no.ustore.json file nor raw address, the counteddata_directoriesareNULL, one has an empty path or a path past the platform's limit, the transport or the CPU class is unknown, or a size or count is out of range.ustore_status_out_of_memory_kThememory_limitrefuses the arena or the channels.ustore_status_not_found_kThe manifest's directory does not exist.ustore_status_already_exists_kThe first slot of a raw medium belongs to something else.ustore_status_capacity_exhausted_kThe drives need more queue pairs than a disk holds, thememory_limitleaves no block per channel, or every shared channel is taken.ustore_status_malformed_kThe manifest is corrupt or names a SHA-256 a run's bytes miss, or raw data directories are listed in another order than the database was created with.
Parameters
databaseustore_database_t * | Output for the opened database, left NULL on failure. | |
|---|---|---|
executorustore_executor_t | Executor that runs the call, where NULL runs it on the calling thread. | |
outcomeustore_outcome_t * | Output for the outcome, final once its status leaves ustore_status_deferred_k. | |
completionustore_completion_t | Whether the call returns final, the default, or once submitted. | |
directoryustore_bytes_view_t | The manifest, <database>.ustore.json, whose directory must exist and holds the database's first data directory, or the raw address of its first slot, as in /dev/nvme0n1@64GiB. | |
data_directoriesustore_data_directory_t const * | Further data directories every run may land on, the one with the most free bytes taking it, or null. | |
data_directories_countustore_size_t | Entries of data_directories. | |
compactionsustore_executor_t | Executor of ustore_executor_for_vram whose device, scratch budget and GPUDirect Storage mode every merge of the database takes, read once here, so it may be freed afterwards; NULL merges on host threads. | |
memory_limitustore_size_t | Bytes of host memory the engine may pin and cache. | |
transaction_max_bytesustore_size_t | Bytes a transaction may buffer before it must commit. | |
transaction_max_deltasustore_size_t | Deltas a transaction may buffer before it must commit. | |
collection_concurrency_limitustore_size_t | Operations that may run on one collection at once. | |
collection_cache_limitustore_size_t | Bytes of values each collection may cache. | |
collection_memory_runs_max_bytesustore_size_t | Bytes of memory runs a collection holds before the commit that crosses them flushes level 0. | |
collection_memory_runs_max_deltasustore_size_t | Deltas of memory runs a collection holds before the commit that crosses them flushes level 0. | |
collection_level_enlarge_factorustore_size_t | How many times larger each level of a collection is than the one above it. | |
collection_value_sizeustore_size_t | Bytes of every value of a collection whose values share one size, or zero for varying sizes. | |
collection_value_max_sizeustore_size_t | Bytes of the largest value of a collection whose values vary in size. | |
cpu_classustore_cpu_class_t | The CPU class the database confines its waits, atomics and string kernels to, where zero takes what the CPU offers. |
Statuses
ustore_status_unsupported_k means the library was built without the engine, SPDK is asked for, the manifest or a slot has another format version, the compactions executor is given to a library built without a CUDA compiler or drives an AMD device, the CPU does not run the cpu_class, or the io_uring transport was named on a kernel without one.
ustore_database_freeWaits until every deferred call naming the database finishes, then closes it and frees its memory once no transaction, snapshot or cursor of it holds it open, while the data persists on disk.
void ustore_database_free(ustore_database_t database); Parameters
database | inout | The database to be closed; NULL is safe. |
|---|