struct ustore_arena_description_t Fields
residency | ustore_residency_t | RAM for host pages, VRAM for one device's memory. |
|---|---|---|
device | int32_t | The UStore ordinal of the device as ustore_gpu_devices lists it, or -1 for host pages. |
A database holds no collection until one is created, and every call names the collections it addresses, refusing a collections view with NULL data with ustore_status_invalid_argument_k.
A collection is created with an optional name and the ustore_modality_t it holds, and addressed afterwards by the identifier the call answers, a UUIDv7 its create mints once and never reuses:
char const name[] = "movies";
ustore_collection_id_t movies = {0};
ustore_collection_create(&(ustore_collection_create_t){
.database = database,
.outcome = &outcome,
.name = {(uint8_t const*)name, sizeof(name) - 1},
.modality = ustore_modality_docs_k,
.collection = &movies,
});
ustore_collection_id_t values = {0};
ustore_collection_create(&(ustore_collection_create_t){
.database = database,
.outcome = &outcome,
.modality = ustore_modality_blobs_k,
.collection = &values,
});The examples below write binary values into values and documents into movies.
Each engine records names in the _names system collection and every collection’s modality and name in _collections, which no identifier reaches and no listing shows, so a name starting with _ is refused, and a name resolves in two phases: once through ustore_collection_find, which answers the identifier and the modality of every name of a batch at the head or at a snapshot, and by identifier from then on, as no other call carries one.
Collections are created and dropped only at the head, so their names are never read inside a transaction, and resolving one never joins the conflicts of the work that follows it.
A create records the name in the same step as the collection, refusing a name another collection carries even under a concurrent creator.
A name no collection carries resolves to the nil UUID with its presence bit cleared, and ustore_collection_drop with ustore_drop_collection_k forgets a collection’s name before the collection goes. ustore_collection_list exports the identifier of every collection and, where asked, its modality and its name, as a tape where a collection created without one has an empty name.
A listing describes the database rather than a transaction, so it reads the head or a snapshot.
ustore_database_describe reports the engine of an open database, what it supports and how much it holds, while ustore_database_clear and ustore_database_compact work on every collection at once:
ustore_database_description_t description = {0};
ustore_database_describe(&(ustore_database_describe_t){
.database = database,
.outcome = &outcome,
.description = &description,
});
printf("%s holds %llu keys in %llu bytes on disk\n", ustore_engine_name(description.engine),
(unsigned long long)description.keys, (unsigned long long)description.disk_bytes);
ustore_database_clear(&(ustore_database_clear_t){.database = database, .outcome = &outcome, .mode = ustore_drop_pairs_k});
ustore_database_compact(&(ustore_database_compact_t){.database = database, .outcome = &outcome});A description’s capabilities say whether the engine supports transactions, collections and snapshots, engine names the engine, and isolation says how its transactions see each other: as the ram engine was opened, and ustore_isolation_snapshot_k elsewhere. keys, memory_bytes and disk_bytes are the engine’s estimates of the keys across every collection and of the bytes the data takes, and zero where it keeps none, as LevelDB keeps no count of its keys. pin_refused_bytes and crossing_requests report where the NVMe engine went unpinned: the arena bytes the locked-memory cap refused to pin, and the requests that crossed a gibibyte of a pinned arena. ustore_database_clear empties every collection as its mode names:
ustore_drop_pairs_k removes every pair, keeping the collections and their names;ustore_drop_collection_k drops every collection but the main one with its name, and empties the main one;ustore_drop_values_k empties every value, keeping the keys.The ram and RocksDB engines refuse to drop the collections while a snapshot is open, with ustore_status_invalid_argument_k, since the snapshot belongs to its holder. ustore_database_compact reclaims the space overwritten and removed data still takes, one collection at a time: RocksDB and LevelDB flush and compact their trees, and the ram engine frees the versions no reader needs and rewrites its image.
ustore_arena_description_tWhere an arena's outputs live.struct ustore_arena_description_t residency | ustore_residency_t | RAM for host pages, VRAM for one device's memory. |
|---|---|---|
device | int32_t | The UStore ordinal of the device as ustore_gpu_devices lists it, or -1 for host pages. |
ustore_uuid_tSixteen bytes naming one thing everywhere, ordered as their bytes and compared with memcmp.struct ustore_uuid_t An integer spells as its 128-bit big-endian bytes, as Python's uuid.UUID(int=...) does, so identifiers made from integers order as the integers.
bytes | uint8_t |
|---|
ustore_collection_id_tThe identifier of a collection: a UUIDv7 its create mints once and never reuses, so it stays the same across restarts and never names two collections.ustore_key_tThe identifier of a value within one collection.ustore_keys_view_tA strided run of keys, one per task, where a zero stride repeats the first.struct ustore_keys_view_t data | ustore_key_t const * | First key of the run. |
|---|---|---|
stride | ustore_size_t | Bytes between consecutive keys, where zero repeats the first one. |
ustore_collections_view_tA strided run of collections, one per task, whose NULL data a call refuses with ustore_status_invalid_argument_k, as no collection was named.struct ustore_collections_view_t data | ustore_collection_id_t const * | First collection of the run. |
|---|---|---|
stride | ustore_size_t | Bytes between consecutive collections, where zero repeats the first one. |
ustore_drop_mode_tWhat dropping a collection removes.enum ustore_drop_mode_t ustore_drop_collection_k | 0 | The collection together with its name and every pair it holds. |
|---|---|---|
ustore_drop_pairs_k | 1 | Every key and value, keeping the collection. |
ustore_drop_values_k | 2 | Every value, keeping the keys, which a strings collection refuses, as its values are keys. |
char const * ustore_drop_mode_name(ustore_drop_mode_t drop_mode);ustore_drop_mode_t ustore_drop_mode_from_name(ustore_bytes_view_t name);ustore_capabilities_tCapabilities an engine reports.enum ustore_capabilities_t ustore_capabilities_transactions_k | 1 | The engine supports transactions. |
|---|---|---|
ustore_capabilities_snapshots_k | 4 | The engine supports snapshots. |
ustore_database_description_tWhat an open database reports about itself in one call, where every size is the engine's estimate, and zero where the engine keeps none.struct ustore_database_description_t capabilities | ustore_capabilities_t | The capabilities of the engine. |
|---|---|---|
engine | ustore_engine_t | The engine serving the database. |
isolation | ustore_isolation_t | How transactions see each other: what the RAM engine was opened with, and ustore_isolation_snapshot_k elsewhere. |
keys | ustore_size_t | Keys across every collection. |
memory_bytes | ustore_size_t | Bytes of memory the engine holds for the data. |
disk_bytes | ustore_size_t | Bytes the files of the database take on disk. |
ustore_modality_tWhat a collection holds, recorded beside its name so a view of another kind is refused.enum ustore_modality_t Zero names no modality.
ustore_modality_blobs_k | 1 | Binary values of blobs.h. |
|---|---|---|
ustore_modality_docs_k | 2 | JSON documents of docs.h. |
ustore_modality_graph_k | 3 | Vertices and their edges of graph.h. |
ustore_modality_strings_k | 4 | Values keyed by strings of strings.h. |
ustore_modality_pages_k | 5 | Ordered documents of pages.h. |
char const * ustore_modality_name(ustore_modality_t modality);ustore_modality_t ustore_modality_from_name(ustore_bytes_view_t name);ustore_database_describeReports the engine of an open database, what it supports and how much it holds.void ustore_database_describe(ustore_database_describe_t *arguments); databaseustore_database_t | An open database. | |
|---|---|---|
executorustore_executor_t | Executor that runs the call, where NULL runs it on the calling thread. | |
outcomeustore_outcome_t * | Output for the outcome, final once its status leaves ustore_status_deferred_k. | |
completionustore_completion_t | Whether the call returns final, the default, or once submitted. | |
descriptionustore_database_description_t * | Output for the description, filled whole; zeroed on failure. |
ustore_status_contended_k The executor's deferred table is full, or a callback blocks on the executor running it.ustore_status_uninitialized_k The database is NULL.ustore_status_unsupported_k The executor computes on a device.ustore_status_invalid_argument_k The description output is NULL, on the RAM and NVMe engines.ustore_database_clearEmpties every collection at once, as ustore_collection_drop empties one.void ustore_database_clear(ustore_database_clear_t *arguments); The RAM and RocksDB engines refuse to drop the collections while a snapshot is open, since the snapshot belongs to its holder.
databaseustore_database_t | An open database. | |
|---|---|---|
executorustore_executor_t | Executor that runs the call, where NULL runs it on the calling thread. | |
outcomeustore_outcome_t * | Output for the outcome, final once its status leaves ustore_status_deferred_k. | |
completionustore_completion_t | Whether the call returns final, the default, or once submitted. | |
modeustore_drop_mode_t | What goes: every pair, keeping the collections and their names; every collection with its name; or every value, keeping the keys where the engine can. |
ustore_status_contended_k The executor's deferred table is full, or a callback blocks on the executor running it.ustore_status_uninitialized_k The database is NULL.ustore_status_read_only_k The database was opened read-only.ustore_status_unsupported_k The executor computes on a device.ustore_status_invalid_argument_k The mode is unknown, or a snapshot is open while ustore_drop_collection_k drops the collections on the RAM and RocksDB engines.ustore_status_not_found_k A listed collection was dropped meanwhile, on the NVMe engine.ustore_status_write_conflict_k Concurrent commits kept conflicting, on the RAM and NVMe engines.ustore_database_compactReclaims the space overwritten and removed data still takes: RocksDB and LevelDB flush and compact their trees, and the RAM engine frees the versions no reader needs and rewrites its image.void ustore_database_compact(ustore_database_compact_t *arguments); The work runs one collection at a time, and its progress counts the bytes of the collections compacted where the engine measures them, as RocksDB does, and one unit per collection elsewhere. A cancelled compaction stops between collections, keeping what it compacted.
databaseustore_database_t | An open database. | |
|---|---|---|
executorustore_executor_t | Executor that runs the call, where NULL runs it on the calling thread. | |
outcomeustore_outcome_t * | Output for the outcome, final once its status leaves ustore_status_deferred_k. | |
completionustore_completion_t | Whether the call returns final, the default, or once submitted. | |
progressustore_progress_callback_t | Optional callback reporting the compaction's advance. | |
progress_statevoid * | Opaque pointer passed to progress. |
ustore_status_contended_k The executor's deferred table is full, or a callback blocks on the executor running it.ustore_status_uninitialized_k The database is NULL.ustore_status_read_only_k The database was opened read-only.ustore_status_unsupported_k The executor computes on a device.ustore_database_flushMakes every commit that returned before this call durable, as if each had asked for ustore_durability_flushed_k.void ustore_database_flush(ustore_database_flush_t *arguments); LevelDB syncs only its current log, so a commit in a log it already rotated stays unsynced until its compaction lands.
databaseustore_database_t | An open database. | |
|---|---|---|
executorustore_executor_t | Executor that runs the call, where NULL runs it on the calling thread. | |
outcomeustore_outcome_t * | Output for the outcome, final once its status leaves ustore_status_deferred_k. | |
completionustore_completion_t | Whether the call returns final, the default, or once submitted. |
ustore_status_contended_k The executor's deferred table is full.ustore_status_uninitialized_k The database is NULL.ustore_status_read_only_k The database was opened read-only.ustore_status_unsupported_k The executor computes on a device.ustore_collection_listLists the collections of the database.void ustore_collection_list(ustore_collection_list_t *arguments); databaseustore_database_t | An open database. | |
|---|---|---|
executorustore_executor_t | Executor that runs the call, where NULL runs it on the calling thread. | |
outcomeustore_outcome_t * | Output for the outcome, final once its status leaves ustore_status_deferred_k. | |
completionustore_completion_t | Whether the call returns final, the default, or once submitted. | |
snapshotustore_snapshot_t | Optional snapshot whose collections are listed, where NULL lists the head. | |
collectionsustore_collection_id_t const ** | Optional output for the identifiers of the collections, inside arena. | |
modalitiesustore_modality_t const ** | Optional output for what every collection holds, inside arena. | |
namesustore_strings_view_t * | Optional output for the names as a tape inside arena, in the order of the identifiers, where a collection without a name has an empty one. | |
writtenustore_size_t * | Output for the collections written; zeroed on failure. | |
arenaustore_arena_t | Rewound by this call, then holding every output it writes; NULL is refused. |
ustore_status_invalid_argument_k The arena or the written output is NULL, or the snapshot belongs to another engine.ustore_status_device_k The arena lives where another executor computes.ustore_status_contended_k The arena serves another call, the executor's deferred table is full, or a callback blocks on the executor running it.ustore_status_uninitialized_k The database is NULL.ustore_status_unsupported_k The executor computes on a device.ustore_status_not_found_k A listed collection was dropped meanwhile, on the NVMe engine.ustore_collection_createCreates an empty collection, named or not.void ustore_collection_create(ustore_collection_create_t *arguments); databaseustore_database_t | An open database. | |
|---|---|---|
executorustore_executor_t | Executor that runs the call, where NULL runs it on the calling thread. | |
outcomeustore_outcome_t * | Output for the outcome, final once its status leaves ustore_status_deferred_k. | |
completionustore_completion_t | Whether the call returns final, the default, or once submitted. | |
nameustore_bytes_view_t | Optional name no other collection carries and not starting with _, where a zeroed view creates a collection without one. | |
modalityustore_modality_t | What the collection holds, which fixes how it is keyed and so which calls reach it. | |
collectionustore_collection_id_t * | Output for the identifier of the collection, zeroed on failure. |
ustore_status_contended_k The executor's deferred table is full, or a callback blocks on the executor running it.ustore_status_uninitialized_k The database is NULL.ustore_status_read_only_k The database was opened read-only.ustore_status_invalid_argument_k The modality is unknown, the name starts with _ or is empty with non-null data, or the collection output is NULL.ustore_status_unsupported_k The executor computes on a device, or a documents, graph or pages collection is asked of an NVMe database whose values share one size.ustore_status_already_exists_k Another collection carries the name.ustore_status_write_conflict_k Concurrent commits kept conflicting, on the RAM and NVMe engines.ustore_status_capacity_exhausted_k The name outgrows the NVMe schema's record, or every shared channel is taken.ustore_collection_findReads the collections a batch of names address.void ustore_collection_find(ustore_collection_find_t *arguments); databaseustore_database_t | An open database. | |
|---|---|---|
executorustore_executor_t | Executor that runs the call, where NULL runs it on the calling thread. | |
outcomeustore_outcome_t * | Output for the outcome, final once its status leaves ustore_status_deferred_k. | |
completionustore_completion_t | Whether the call returns final, the default, or once submitted. | |
snapshotustore_snapshot_t | Optional snapshot whose names are read, where NULL reads the head. | |
countustore_size_t | Number of tasks, one per name. | |
namesustore_strings_view_t | Names to resolve, one per task. | |
presencesustore_octet_t const ** | Optional output for a bitset of count bits marking the names a collection carries, addressed as Apache Arrow does, inside arena. | |
collectionsustore_collection_id_t const ** | Output for the identifiers of the collections the names address, inside arena. | |
modalitiesustore_modality_t const ** | Optional output for what the collections hold, inside arena. | |
arenaustore_arena_t | Rewound by this call, then holding every output it writes; NULL is refused. |
ustore_status_invalid_argument_k The arena or the collections output is NULL, the names come in an unknown layout, or the snapshot belongs to another engine.ustore_status_device_k The arena lives where another executor computes.ustore_status_contended_k The arena serves another call, the executor's deferred table is full, or a callback blocks on the executor running it.ustore_status_uninitialized_k The database is NULL.ustore_status_unsupported_k The executor computes on a device.ustore_collection_dropRemoves or clears a collection.void ustore_collection_drop(ustore_collection_drop_t *arguments); databaseustore_database_t | An open database. | |
|---|---|---|
executorustore_executor_t | Executor that runs the call, where NULL runs it on the calling thread. | |
outcomeustore_outcome_t * | Output for the outcome, final once its status leaves ustore_status_deferred_k. | |
completionustore_completion_t | Whether the call returns final, the default, or once submitted. | |
collectionustore_collection_id_t | Identifier of an existing collection. | |
modeustore_drop_mode_t | Whether values, pairs or the whole collection go. |
ustore_status_contended_k The executor's deferred table is full, or a callback blocks on the executor running it.ustore_status_uninitialized_k The database is NULL.ustore_status_read_only_k The database was opened read-only.ustore_status_unsupported_k The executor computes on a device.ustore_status_invalid_argument_k The mode is unknown, ustore_drop_values_k empties a strings collection, or a snapshot is open while ustore_drop_collection_k drops the collection on the RAM and RocksDB engines.ustore_status_not_found_k No collection carries the identifier, a system one included.ustore_status_write_conflict_k Concurrent commits kept conflicting, on the RAM and NVMe engines.ustore_status_capacity_exhausted_k Every shared channel is taken on the NVMe engine.ustore_key_unknown_kThe key that stands for no key, every bit of it set.ustore_key_t const ustore_key_unknown_k;