C · Quickstart

Quickstart

#include <ustore/types.h>#include <ustore/db.h> 1.0.0-rc.1 · d6d40961

§ Conventions

Every call takes a single pointer to an argument struct named after it with a _t suffix, so arguments are named and every omitted one is zero. Every struct opens with its handle, then executor, outcome and completion, so every call reports through a ustore_outcome_t whose status names the result and whose static message is never freed. Statuses read ustore_status_<name>_k, from ustore_status_success_k through every failure. Every closed set is a C enum, and each one a binding spells has a pair of functions: <prefix>_<enum>_name, such as ustore_isolation_name, answers a static lowercase spelling like strict-serializable, or unrecognized for a value no enumerator carries, and <prefix>_<enum>_from_name reads a ustore_bytes_view_t back into its output, failing with ustore_status_invalid_argument_k for a name it does not know. ustore_version_name spells the version of the library the program runs against, like 1.0.0. Every call is a batch of count tasks, and each of its inputs takes one of three shapes:

  • One run of bytes, a ustore_bytes_view_t of a pointer and a length, carries a path or a name, never null-terminated by contract.
  • A strided run of fixed-width elements, such as a ustore_keys_view_t of a pointer and a stride in bytes, carries one element per task, and a zero stride repeats the first element for every task.
  • A ragged batch, a ustore_strings_view_t, carries one string per task as an Apache Arrow tape of 32-bit or 64-bit offsets, or through an accessor callback over any other shape.

A zeroed view of any shape stands for the input’s default. Every output whose length scales with the call lands packed in the arena the call is given, so no output carries a capacity or a stride, and ragged bytes come back as a tape. Keys are ustore_key_t, 16 bytes of a UUID compared bytewise, and a non-negative integer spells as its 128-bit big-endian bytes, as Python’s uuid.UUID(int=...) does, so keys made from integers order as the integers.

The examples below share these handles, the arena made once for calls on the calling thread:

ustore_database_t database = NULL;
ustore_executor_t executor = NULL;
ustore_outcome_t outcome = {0};
ustore_arena_t arena = NULL;
ustore_arena_init(&(ustore_arena_init_t){.arena = &arena, .outcome = &outcome});

§ Opening the Database

#include <ustore/ustore.h>

char const directory[] = "/var/lib/ustore";
ustore_database_open_rocksdb(&(ustore_database_open_rocksdb_t){
    .database = &database,
    .outcome = &outcome,
    .directory = {(uint8_t const*)directory, sizeof(directory) - 1},
});
if (outcome.status != ustore_status_success_k)
    fprintf(stderr, "%s: %s\n", ustore_status_name(outcome.status), outcome.message);

ustore_arena_free(arena);
ustore_database_free(database);
ustore_executor_free(executor);

Each engine opens through its own calls, whose arguments hold what that engine can be asked, and a zero field keeps the engine’s default:

  • ustore_database_init_ram starts an empty database in memory alone, while ustore_database_open_ram serves a directory as one image, loading it at open and rewriting it whole on a flushed commit and on close.
  • ustore_database_open_rocksdb and ustore_database_open_leveldb serve a directory in place, creating the database when the directory holds none.
  • ustore_database_open_nvme serves the database its manifest names, <database>.ustore.json in an existing directory or the raw address of its first slot, such as /dev/nvme0n1@64GiB, creating the database when the manifest is absent.
  • ustore_database_view_rocksdb serves an existing RocksDB directory read-only, refusing every call that would change it with ustore_status_read_only_k: writes, removes, transactions, collection creates and drops, clears and compactions, while snapshots and their saves still serve.

The opens of an engine the build lacks answer ustore_status_unsupported_k, so a program links against every build. Every freeing function accepts NULL.

API

Types

ustore_size_tAn unsigned count or size, 64 bits wide on every platform.
types.h:195 ↗
typedef uint64_t ustore_size_t;
ustore_bytes_view_tA borrowed run of bytes and its length, never null-terminated by contract, so a caller holding a string without a terminator passes its own storage.
types.h:199 ↗
struct ustore_bytes_view_t

Fields

datauint8_t const *First byte of the run.
lengthustore_size_tBytes the run spans, with no terminator assumed or written.
ustore_database_tAn opaque handle to an open database, safe to share across threads from open until free.
db.h:24 ↗
typedef struct ustore_database_handle_t * ustore_database_t;

An embedded database owns its directory, so no two processes may open the same one at once. A database holds no collection until one is created, and every call addresses the collections it created by identifier.

ustore_byte_tThe elementary piece of a binary value.
db.h:69 ↗
typedef uint8_t ustore_byte_t;
ustore_octet_tThe smallest bitset, holding eight bits.
db.h:75 ↗
typedef uint8_t ustore_octet_t;
ustore_engine_tThe engine serving a database, as ustore_database_describe reports it.
db.h:254 ↗
enum ustore_engine_t

Values

ustore_engine_ram_k0The in-memory engine of ustore_database_init_ram and ustore_database_open_ram.
ustore_engine_rocksdb_k1The RocksDB engine of ustore_database_open_rocksdb and ustore_database_view_rocksdb.
ustore_engine_leveldb_k2The LevelDB engine of ustore_database_open_leveldb.
ustore_engine_nvme_k3The NVMe engine of ustore_database_open_nvme.

Spellings

char const * ustore_engine_name(ustore_engine_t engine);
ustore_engine_t ustore_engine_from_name(ustore_bytes_view_t name);
ustore_data_directory_tA directory the data files may occupy, with the bytes they may take there.
db.h:273 ↗
struct ustore_data_directory_t

Fields

pathustore_bytes_view_tThe directory, which must exist; on NVMe also a raw device from a base, as in /dev/nvme0n1@64GiB.
max_bytesustore_size_tBytes the files may take there, zero being unlimited on RocksDB and the whole medium on NVMe.

Functions

ustore_database_init_ramStarts an empty database of the in-memory engine, which persists nowhere.
db.h:321 ↗
void ustore_database_init_ram(ustore_database_init_ram_t *arguments);

Parameters

databaseustore_database_t *Output for the new database, left NULL on failure.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
isolationustore_isolation_tHow transactions see each other.
cpu_classustore_cpu_class_tThe CPU class the database confines its waits, atomics and string kernels to, where zero takes what the CPU offers.

Statuses

ustore_database_open_ramServes a directory in place through the in-memory engine, loading the image it holds, if any, and rewriting it whole on a flushed commit and on close.
db.h:366 ↗
void ustore_database_open_ram(ustore_database_open_ram_t *arguments);

Parameters

databaseustore_database_t *Output for the opened database, left NULL on failure.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
directoryustore_bytes_view_tDirectory the database persists in as one image, which must exist.
isolationustore_isolation_tHow transactions see each other.
cpu_classustore_cpu_class_tThe CPU class the database confines its waits, atomics and string kernels to, where zero takes what the CPU offers.

Statuses

ustore_database_open_rocksdbServes a directory in place through the RocksDB engine, creating the database when it holds none.
db.h:408 ↗
void ustore_database_open_rocksdb(
    ustore_database_open_rocksdb_t *arguments);

Parameters

databaseustore_database_t *Output for the opened database, left NULL on failure.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
directoryustore_bytes_view_tDirectory of the database, which must exist.
data_directoriesustore_data_directory_t const *Directories the table files fill in order, or null to keep them in directory.
data_directories_countustore_size_tEntries of data_directories.

Statuses

ustore_database_view_rocksdbServes a directory in place through the RocksDB engine, read-only.
db.h:455 ↗
void ustore_database_view_rocksdb(
    ustore_database_view_rocksdb_t *arguments);

The database reads the state its directory held at open, and refuses every call that would change it with ustore_status_read_only_k: writes, removes, transactions, collection creates and drops, clears and compactions. Snapshots and their saves still serve.

Parameters

databaseustore_database_t *Output for the opened database, left NULL on failure.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
directoryustore_bytes_view_tDirectory of an existing database.
data_directoriesustore_data_directory_t const *Directories the table files fill, as the database was opened with, or null to find them in directory.
data_directories_countustore_size_tEntries of data_directories.

Statuses

ustore_database_open_leveldbServes a directory in place through the LevelDB engine, creating the database when it holds none.
db.h:489 ↗
void ustore_database_open_leveldb(
    ustore_database_open_leveldb_t *arguments);

Parameters

databaseustore_database_t *Output for the opened database, left NULL on failure.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
directoryustore_bytes_view_tDirectory of the database, which must exist.

Statuses

ustore_database_open_nvmeServes the database a manifest names through the NVMe engine, creating it when the manifest is absent.
db.h:585 ↗
void ustore_database_open_nvme(ustore_database_open_nvme_t *arguments);

Parameters

databaseustore_database_t *Output for the opened database, left NULL on failure.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
directoryustore_bytes_view_tThe manifest, <database>.ustore.json, whose directory must exist and holds the database's first data directory, or the raw address of its first slot, as in /dev/nvme0n1@64GiB.
data_directoriesustore_data_directory_t const *Further data directories every run may land on, the one with the most free bytes taking it, or null.
data_directories_countustore_size_tEntries of data_directories.
compactionsustore_executor_tExecutor of ustore_executor_for_vram whose device, scratch budget and GPUDirect Storage mode every merge of the database takes, read once here, so it may be freed afterwards; NULL merges on host threads.
memory_limitustore_size_tBytes of host memory the engine may pin and cache.
transaction_max_bytesustore_size_tBytes a transaction may buffer before it must commit.
transaction_max_deltasustore_size_tDeltas a transaction may buffer before it must commit.
collection_concurrency_limitustore_size_tOperations that may run on one collection at once.
collection_cache_limitustore_size_tBytes of values each collection may cache.
collection_memory_runs_max_bytesustore_size_tBytes of memory runs a collection holds before the commit that crosses them flushes level 0.
collection_memory_runs_max_deltasustore_size_tDeltas of memory runs a collection holds before the commit that crosses them flushes level 0.
collection_level_enlarge_factorustore_size_tHow many times larger each level of a collection is than the one above it.
collection_value_sizeustore_size_tBytes of every value of a collection whose values share one size, or zero for varying sizes.
collection_value_max_sizeustore_size_tBytes of the largest value of a collection whose values vary in size.
cpu_classustore_cpu_class_tThe CPU class the database confines its waits, atomics and string kernels to, where zero takes what the CPU offers.

Statuses

ustore_status_unsupported_k means the library was built without the engine, SPDK is asked for, the manifest or a slot has another format version, the compactions executor is given to a library built without a CUDA compiler or drives an AMD device, the CPU does not run the cpu_class, or the io_uring transport was named on a kernel without one.

ustore_database_freeWaits until every deferred call naming the database finishes, then closes it and frees its memory once no transaction, snapshot or cursor of it holds it open, while the data persists on disk.
db.h:769 ↗
void ustore_database_free(ustore_database_t database);

Parameters

databaseinoutThe database to be closed; NULL is safe.