Documents | UStore for C

A document collection stores JSON text, so reading a document back through ustore_blobs_read may not return the exact bytes written. A field addresses a value inside a document, either as a top-level key such as _id or as an RFC 6901 JSON Pointer such as /posts/0/text, while an empty field or a zeroed fields view addresses the whole document.

ustore_docs_write upserts, updates or inserts whole documents or fields, or applies RFC 6902 patches or RFC 7386 merge patches, as its modification selects.
ustore_docs_read reads whole documents or fields, as JSON or as values printed into text.
ustore_docs_gist lists the unique JSON Pointers to the leaves of the requested documents.
ustore_docs_gather reads many fields of many documents into typed columns, with validity, conversion and collision bitmaps per column.
ustore_docs_remove removes whole documents, and ustore_docs_scan lists their keys as ustore_blobs_scan lists the keys of blobs.
ustore_docs_sample draws document keys at random as ustore_blobs_sample draws the keys of blobs.

ustore_key_t key = {.uuid.bytes[15] = 1};
char const document[] = "{\"person\": \"Alice\", \"age\": 27}";
uint32_t const document_offsets[2] = {0, sizeof(document) - 1};
ustore_docs_write(&(ustore_docs_write_t){
    .database = database,
    .outcome = &outcome,
    .count = 1,
    .collections = {&movies, 0},
    .keys = {&key, sizeof(ustore_key_t)},
    .values = {.layout = ustore_strings_u32_tape_k, .data.u32_tape = {(uint8_t const*)document, document_offsets}},
    .type = ustore_docs_field_type_json_k,
    .modification = ustore_docs_modify_upsert_k,
});

char const field[] = "/person";
uint32_t const field_offsets[2] = {0, sizeof(field) - 1};
ustore_strings_view_t person = {0};
ustore_docs_read(&(ustore_docs_read_t){
    .database = database,
    .outcome = &outcome,
    .count = 1,
    .collections = {&movies, 0},
    .keys = {&key, sizeof(ustore_key_t)},
    .fields = {.layout = ustore_strings_u32_tape_k, .data.u32_tape = {(uint8_t const*)field, field_offsets}},
    .values = &person,
    .arena = arena,
    .outputs_type = ustore_docs_field_type_string_k,
});

A gather takes fields_count fields as one strings view and the type of every column in outputs_types, and answers one ustore_docs_column_view_t per field in the arena. A column holds the validity, conversion and collision bitmaps of its rows beside either the scalars or a tape of the strings.

API

Types

ustore_docs_field_type_tType of a document, or of a value extracted from one.

docs.h:31 ↗

enum ustore_docs_field_type_t

Every type but JSON text, strings and binary text is a fixed-width scalar.

Values

`ustore_docs_field_type_json_k`	`0`	JSON text.
`ustore_docs_field_type_null_k`	`10`	The JSON null.
`ustore_docs_field_type_bool_k`	`11`	A boolean.
`ustore_docs_field_type_uuid_k`	`12`	A 128-bit UUID.
`ustore_docs_field_type_f64_k`	`42`	A 64-bit float.
`ustore_docs_field_type_f32_k`	`41`	A 32-bit float.
`ustore_docs_field_type_f16_k`	`40`	A 16-bit float.
`ustore_docs_field_type_i64_k`	`23`	A signed 64-bit integer.
`ustore_docs_field_type_i32_k`	`22`	A signed 32-bit integer.
`ustore_docs_field_type_i16_k`	`21`	A signed 16-bit integer.
`ustore_docs_field_type_i8_k`	`20`	A signed 8-bit integer.
`ustore_docs_field_type_u64_k`	`33`	An unsigned 64-bit integer.
`ustore_docs_field_type_u32_k`	`32`	An unsigned 32-bit integer.
`ustore_docs_field_type_u16_k`	`31`	An unsigned 16-bit integer.
`ustore_docs_field_type_u8_k`	`30`	An unsigned 8-bit integer.
`ustore_docs_field_type_binary_k`	`50`	The text of a value without a null terminator.
`ustore_docs_field_type_string_k`	`51`	A value printed as text.

Spellings

char const * ustore_docs_field_type_name(
    ustore_docs_field_type_t field_type);

ustore_docs_field_type_t ustore_docs_field_type_from_name(
    ustore_bytes_view_t name);

ustore_docs_modification_tModification ustore_docs_write applies: an upsert, update or insert, an RFC 6902 patch, or an RFC 7386 merge.

docs.h:87 ↗

enum ustore_docs_modification_t

Values

`ustore_docs_modify_upsert_k`	`0`
`ustore_docs_modify_update_k`	`1`
`ustore_docs_modify_insert_k`	`2`
`ustore_docs_modify_patch_k`	`3`
`ustore_docs_modify_merge_k`	`4`

Spellings

char const * ustore_docs_modification_name(
    ustore_docs_modification_t modification);

ustore_docs_modification_t ustore_docs_modification_from_name(
    ustore_bytes_view_t name);

ustore_docs_column_view_tOne column of ustore_docs_gather: count rows of one field, as an Apache Arrow array of its type.

docs.h:337 ↗

struct ustore_docs_column_view_t

Validity, conversion and collision columns are bitmaps, least significant bit first. A value is valid when it was found and converted, converted when the conversion changed its semantics, such as a string parsed into an integer, and colliding when a value of an inconvertible type sits under the field.

Fields

`validities`	`ustore_octet_t const *`	Bits marking the rows whose value was found and converted.
`conversions`	`ustore_octet_t const *`	Bits marking the rows whose conversion changed the value's semantics.
`collisions`	`ustore_octet_t const *`	Bits marking the rows where a value of an inconvertible type sits under the field.
`scalars`	`ustore_byte_t const *`	The scalars, each as wide as the column's type and aligned to it, or `NULL` for a string column.
`strings`	`ustore_strings_view_t`	The `count` strings of a string column as a tape, or a zeroed view for a scalar column.

Functions

ustore_docs_writeWrites whole documents or fields inside them.

docs.h:194 ↗

void ustore_docs_write(ustore_docs_write_t *arguments);

ustore_status_invalid_argument_k means the collections or values are missing, so are both the keys and the id_field, the fields or values come in an unknown layout, the transaction belongs to another engine or is staged, the durability, watch or modification is unknown, a flushed write names a transaction, ustore_watch_none_k comes without one, a collection holds another modality, a patch or a modifier does not apply to its field, or a field or value is past 4 GiB.

ustore_status_unsupported_k The executor computes on a device, or the type is not supported.
ustore_status_not_found_k A collection does not exist or was dropped meanwhile, a field path does not resolve, or the document a field modification, an update or a patch needs is missing.
ustore_status_already_exists_k An inserted document already exists.
ustore_status_malformed_k A document or a value does not parse, is empty or holds NaN or Infinity, an id field is neither an unsigned integer nor a UUID, or a document outgrows what its NVMe collection admits.
ustore_status_write_conflict_k Concurrent commits kept conflicting, or a staged RocksDB transaction holds a key.
ustore_status_capacity_exhausted_k The documents outgrow the NVMe transaction's buffer.

Parameters

`databaseustore_database_t`		An open database.
`executorustore_executor_t`		Executor that runs the call, where `NULL` runs it on the calling thread.
`outcomeustore_outcome_t *`		Output for the outcome, final once its status leaves `ustore_status_deferred_k`.
`completionustore_completion_t`		Whether the call returns final, the default, or once submitted.
`transactionustore_transaction_t`		Transaction the writes join, where `NULL` writes to the head.
`countustore_size_t`		Number of documents or fields to write.
`collectionsustore_collections_view_t`		Collections of the documents.
`keysustore_keys_view_t`		Keys of the documents, where `NULL` data reads every key from `id_field`.
`id_fieldustore_bytes_view_t`		Field holding the key of every document when `keys` has no data, such as "_id".
`fieldsustore_strings_view_t`		Fields to modify inside the documents, one per task, where a zeroed view modifies whole documents.
`valuesustore_strings_view_t`		Values to write, one per task.
`typeustore_docs_field_type_t`		Format of the written values.
`modificationustore_docs_modification_t`		Modification applied to every document.
`durabilityustore_durability_t`		How durable the writes are once the call finishes at the head; a transaction's commit decides its own.
`watchustore_watch_t`		Whether the `transaction` records the documents it reads and writes for conflicts.

Statuses

ustore_status_contended_k The transaction serves another call, the executor's deferred table is full, or a callback blocks on the executor running it.
ustore_status_uninitialized_k The database is NULL.
ustore_status_read_only_k The database was opened read-only.

ustore_docs_readReads whole documents or fields inside them.

docs.h:268 ↗

void ustore_docs_read(ustore_docs_read_t *arguments);

Parameters

`databaseustore_database_t`		An open database.
`executorustore_executor_t`		Executor that runs the call, where `NULL` runs it on the calling thread.
`outcomeustore_outcome_t *`		Output for the outcome, final once its status leaves `ustore_status_deferred_k`.
`completionustore_completion_t`		Whether the call returns final, the default, or once submitted.
`viewustore_view_t`		What the call sees, where `NULL` is the head.
`countustore_size_t`		Number of documents or fields to read.
`collectionsustore_collections_view_t`		Collections of the documents.
`keysustore_keys_view_t`		Keys of the documents.
`fieldsustore_strings_view_t`		Fields to read inside the documents, one per task, where a zeroed view reads whole documents.
`presencesustore_octet_t const **`		Optional output for a bitset of `count` bits marking found values, inside `arena`.
`lengthsustore_length_t const **`		Optional output for the length of every value, for reads that skip the bytes, inside `arena`.
`valuesustore_strings_view_t *`		Optional output for the values as a tape inside `arena`, where a missing value reads as an empty one.
`arenaustore_arena_t`		Rewound by this call, then holding every output it writes; `NULL` is refused.
`outputs_typeustore_docs_field_type_t`		Format the values are converted into on export, either JSON or a string.

Statuses

ustore_status_invalid_argument_k The arena, collections or keys are missing, the fields come in an unknown layout, the view is none of the engine's, a watching read goes through a staged transaction, or a collection holds another modality.
ustore_status_device_k The arena lives where another executor computes, or the device refused a copy or a launch.
ustore_status_contended_k The arena or the transaction the view reads through serves another call, the executor's deferred table is full, a callback blocks on the executor running it, or an NVMe value keeps changing size across reads.
ustore_status_uninitialized_k The database is NULL or an NVMe transfer was cancelled.
ustore_status_unsupported_k Fields are read as neither JSON nor strings.
ustore_status_not_found_k A collection does not exist or was dropped meanwhile.
ustore_status_malformed_k A document does not parse.
ustore_status_capacity_exhausted_k Every shared NVMe channel is taken.

ustore_docs_gistLists the unique paths to the leaves of the requested documents.

docs.h:328 ↗

void ustore_docs_gist(ustore_docs_gist_t *arguments);

Parameters

`databaseustore_database_t`		An open database.
`executorustore_executor_t`		Executor that runs the call, where `NULL` runs it on the calling thread.
`outcomeustore_outcome_t *`		Output for the outcome, final once its status leaves `ustore_status_deferred_k`.
`completionustore_completion_t`		Whether the call returns final, the default, or once submitted.
`viewustore_view_t`		What the call sees, where `NULL` is the head.
`countustore_size_t`		Number of documents to inspect.
`collectionsustore_collections_view_t`		Collections of the documents.
`keysustore_keys_view_t`		Keys of the documents.
`fieldsustore_strings_view_t *`		Output for the `written` unique JSON Pointers as a tape inside `arena`.
`writtenustore_size_t *`		Output for the unique paths written; zeroed on failure.
`arenaustore_arena_t`		Rewound by this call, then holding every output it writes; `NULL` is refused.

Statuses

ustore_status_invalid_argument_k The arena, collections or keys are missing, the view was made by neither view constructor or belongs to another engine, a watching read goes through a staged transaction, or a collection holds another modality.
ustore_status_device_k The arena lives where another executor computes.
ustore_status_contended_k The arena or the transaction the view reads through serves another call, the executor's deferred table is full, a callback blocks on the executor running it, or an NVMe value keeps changing size across reads.
ustore_status_uninitialized_k The database is NULL or an NVMe transfer was cancelled.
ustore_status_unsupported_k The executor computes on a device.
ustore_status_not_found_k A collection does not exist or was dropped meanwhile.
ustore_status_malformed_k A document does not parse.
ustore_status_capacity_exhausted_k A path is 512 bytes or longer, or every shared NVMe channel is taken.

ustore_docs_gatherReads, type-checks and converts fields_count fields of count documents into columns.

docs.h:426 ↗

void ustore_docs_gather(ustore_docs_gather_t *arguments);

Parameters

`databaseustore_database_t`		An open database.
`executorustore_executor_t`		Executor that runs the call, where `NULL` runs it on the calling thread.
`outcomeustore_outcome_t *`		Output for the outcome, final once its status leaves `ustore_status_deferred_k`.
`completionustore_completion_t`		Whether the call returns final, the default, or once submitted.
`viewustore_view_t`		What the call sees, where `NULL` is the head.
`countustore_size_t`		Number of documents, the rows of the output.
`collectionsustore_collections_view_t`		Collections of the documents.
`keysustore_keys_view_t`		Keys of the documents.
`fields_countustore_size_t`		Number of fields, the columns of the output.
`fieldsustore_strings_view_t`		Fields to gather from every document, `fields_count` of them.
`columnsustore_docs_column_view_t const **`		Output for `fields_count` columns of `count` rows, one per field, inside `arena`.
`arenaustore_arena_t`		Rewound by this call, then holding every output it writes; `NULL` is refused.
`outputs_typesustore_docs_field_type_t const *`		Types the columns are converted into, `fields_count` of them, one per field.

Statuses

ustore_status_invalid_argument_k The arena, collections, keys or columns are missing, the fields or their types are missing or unknown, the view is foreign, a watching read goes through a staged transaction, or a collection holds another modality.
ustore_status_device_k The arena lives where another executor computes, or the device refused a copy or a launch.
ustore_status_contended_k The arena or the transaction the view reads through serves another call, the executor's deferred table is full, a callback blocks on the executor running it, or an NVMe value keeps changing size across reads.
ustore_status_uninitialized_k The database is NULL or an NVMe transfer was cancelled.
ustore_status_not_found_k A collection does not exist or was dropped meanwhile.
ustore_status_malformed_k A document does not parse.
ustore_status_capacity_exhausted_k Every shared NVMe channel is taken.

ustore_docs_removeRemoves a batch of whole documents.

docs.h:488 ↗

void ustore_docs_remove(ustore_docs_remove_t *arguments);

ustore_status_invalid_argument_k means the collections or keys are missing, the transaction belongs to another engine or is staged, the durability or watch is unknown, a flushed remove names a transaction, ustore_watch_none_k comes without one, or a collection holds another modality.

ustore_status_unsupported_k The executor computes on a device.
ustore_status_not_found_k A collection does not exist or was dropped meanwhile.
ustore_status_write_conflict_k Concurrent commits kept conflicting, or a staged RocksDB transaction holds a key.
ustore_status_malformed_k A key falls outside what its NVMe collection admits.
ustore_status_capacity_exhausted_k The removes outgrow the NVMe transaction's buffer.

Parameters

`databaseustore_database_t`		An open database.
`executorustore_executor_t`		Executor that runs the call, where `NULL` runs it on the calling thread.
`outcomeustore_outcome_t *`		Output for the outcome, final once its status leaves `ustore_status_deferred_k`.
`completionustore_completion_t`		Whether the call returns final, the default, or once submitted.
`transactionustore_transaction_t`		Transaction the removals join, where `NULL` removes at the head.
`countustore_size_t`		Number of tasks, one per document.
`collectionsustore_collections_view_t`		Collections of the documents.
`keysustore_keys_view_t`		Keys of the documents to remove, or inclusive starts of ranges when `end_keys` is given.
`end_keysustore_keys_view_t`		Exclusive ends of the ranges to remove, or a zeroed view to remove `keys` alone.
`durabilityustore_durability_t`		How durable the removals are once the call finishes at the head; a transaction's commit decides its own.
`watchustore_watch_t`		Whether the `transaction` records the removed keys for conflicts.

Statuses

ustore_status_contended_k The transaction serves another call, the executor's deferred table is full, or a callback blocks on the executor running it.
ustore_status_uninitialized_k The database is NULL.
ustore_status_read_only_k The database was opened read-only.

ustore_docs_column_view_to_arrowExports one column of count rows a ustore_docs_gather of type wrote into arena as an Arrow column whose validity buffer is the column's validities, in place where Arrow lays the column out alike.

arrow.h:310 ↗

ustore_status_t ustore_docs_column_view_to_arrow(
    ustore_docs_column_view_t const *column,
    ustore_size_t count,
    ustore_docs_field_type_t type,
    ustore_arena_t arena,
    struct ArrowSchema *schema,
    struct ArrowDeviceArray *array);

Scalars export as the Arrow type of their width, UUIDs as fixed_size_binary(16) tagged as arrow.uuid, and binary strings as binary or large_binary by the width of their offsets, all in place. Booleans are packed into bits, and strings copied without the null byte ending every row into utf8 or large_utf8, behind the call's outputs in arena under the claim a call takes, on the arena's device for a device arena, through an executor the export spawns there, which also reads a column the gather left on the device. The array lives until arena's next call or free as ustore_keys_view_to_arrow does, and leaves its null count unknown.

Statuses

ustore_status_invalid_argument_k An output or the column is missing, type is no column type, or the column names memory outside arena.
ustore_status_contended_k A call fills arena, or another export carves it.
ustore_status_device_k No executor spawns on the arena's device, or it has no kernels.
ustore_status_out_of_memory_k An allocation failed.