# Reads and Writes

> UStore for C

### Writes and Removals

`ustore_blobs_write` takes its keys as a `ustore_keys_view_t` and its values as a `ustore_strings_view_t`, whose `layout` names how the values lie:

- `ustore_strings_u32_tape_k` is Apache Arrow's `binary`: the bytes end to end, and `count + 1` offsets of 32 bits, where value `index` spans from `offsets[index]` up to `offsets[index + 1]`.
- `ustore_strings_u64_tape_k` is Arrow's `large_binary`: the same with 64-bit offsets.
- `ustore_strings_accessor_k` holds a `ustore_strings_get_t` callback and the `handle` it reads, and the callback writes the view of every value whose index it is asked for.

A single write passes a tape of one value:

```c
ustore_key_t key = {.uuid.bytes[15] = 42};
char const value[] = "meaning of life";
uint32_t const offsets[2] = {0, 15};
ustore_blobs_write(&(ustore_blobs_write_t){
    .database = database,
    .outcome = &outcome,
    .count = 1,
    .collections = {&values, 0},
    .keys = {&key, sizeof(ustore_key_t)},
    .values = {.layout = ustore_strings_u32_tape_k, .data.u32_tape = {(uint8_t const*)value, offsets}},
});
```

A batch held in an array of structures needs no copy into separate arrays, since an accessor can read the values out of the records:

```c
typedef struct pair_t {
    ustore_key_t key;
    char const* text;
} pair_t;

static void get_texts(void const* handle, ustore_size_t const* indices, ustore_size_t count,
                      ustore_bytes_view_t* views) {
    pair_t const* pairs = (pair_t const*)handle;
    for (ustore_size_t index = 0; index != count; ++index) {
        char const* text = pairs[indices[index]].text;
        views[index] = (ustore_bytes_view_t){(uint8_t const*)text, strlen(text)};
    }
}
```

The write then takes the keys as a strided run over the same records:

```c
pair_t const pairs[2] = {
    {{.uuid.bytes[15] = 42}, "meaning of life"},
    {{.uuid.bytes[15] = 43}, "is unknown"},
};
ustore_blobs_write(&(ustore_blobs_write_t){
    .database = database,
    .outcome = &outcome,
    .count = 2,
    .collections = {&values, 0},
    .keys = {&pairs[0].key, sizeof(pair_t)},
    .values = {.layout = ustore_strings_accessor_k, .data.accessor = {get_texts, pairs}},
});
```

The accessor runs on the thread that runs the call, in blocks of up to 64 indices, never concurrently with itself and never after the call is final.
An accessor writing the same view for every index writes one value under every key, as a zero stride repeats one key.
Removing keys is its own call, so no value ever deletes, and an empty value is still a value:

```c
ustore_blobs_remove(&(ustore_blobs_remove_t){
    .database = database, .outcome = &outcome, .count = 1, .collections = {&values, 0}, .keys = {&key, 0}});
```

A batch is atomic: if one task fails, none is applied.
With `end_keys`, every task removes the range from its key, inclusive, to its end key, exclusive, in the same atomic batch; graphs name the field `end_vertices`, strings `end_strings`, and a range of pages removes whole pages.
`durability` chooses whether the call returns once the write is buffered or once it is flushed.

### Reads and Arenas

Every output whose length scales with the call, the values, their lengths and the bitmap of presences alike, lands in an arena.
`ustore_arena_init` makes one where its executor computes, or on host pages for a `NULL` executor, and every call given it rewinds it and refills it, growing only when the new outputs outsize everything it held, so a steady loop allocates once.
Only fixed-size reports, such as descriptions, stamps, the `written` count of a listing and created handles, stay caller fields, and a call that fails zeroes the scalars among them and leaves the handles `NULL`.

```c
ustore_key_t key = {.uuid.bytes[15] = 42};
ustore_octet_t const* presences = NULL;
ustore_strings_view_t values = {0};
ustore_blobs_read(&(ustore_blobs_read_t){
    .database = database,
    .outcome = &outcome,
    .count = 1,
    .collections = {&values, 0},
    .keys = {&key, sizeof(ustore_key_t)},
    .presences = &presences,
    .values = &values,
    .arena = arena,
});
uint32_t const* offsets = values.data.u32_tape.offsets;
assert(values.layout == ustore_strings_u32_tape_k && offsets[1] - offsets[0] == 15);
assert(memcmp(values.data.u32_tape.data + offsets[0], "meaning of life", 15) == 0);
```

Values are joined into one tape as Apache Arrow lays them out: `count + 1` offsets, the last one closing the final value, and the bytes end to end.
The offsets are 32 bits wide while the bytes fit in 4 GiB and 64 bits beyond, so a reader checks `layout` once per call, and a tape a read wrote is written back as it is, without a copy.
Every output is optional: `.lengths` exports the length of every value without its bytes, `.presences` a bitmap of the keys that are present, and a missing key reads as an empty value.
`offsets` and `limits` read the bytes of every value from its offset, at most its limit of them, and report that slice's length, where an offset past the end reads empty.
The outputs stay valid until the next call given `arena`; to keep them past it, pass that call another arena.
A `NULL` arena is refused with `ustore_status_invalid_argument_k`, an arena serves one call at a time, and a call that fails, is cancelled or has nothing to report leaves its outputs `NULL`.
`ustore_arena_free` waits until every deferred call naming the arena is final, and `ustore_arena_describe` reports whether an arena's outputs live on host pages or on which device.

`#include <ustore/types.h>`

`#include <ustore/db.h>`

`#include <ustore/blobs.h>`

`#include <ustore/strings.h>`

`#include <ustore/arrow.h>`

## ustore_strings_get_t

Writes the view of every string `indices[i]` into `views[i]`, for strings no tape holds, and accepts any `count`.

```c
typedef void(*) ustore_strings_get_t(void const *handle, ustore_size_t const *indices, ustore_size_t count, ustore_bytes_view_t *views);
```

## ustore_strings_layout_t

How a batch of strings holds its bytes, naming the member of its `data` union to read.

```c
enum ustore_strings_layout_t
```

- `ustore_strings_u32_tape_k` = `0`: Apache Arrow `Binary`, where string `i` spans ____[offsets[i],offsets[i+1]) of the bytes.
- `ustore_strings_u64_tape_k` = `1`: Apache Arrow `LargeBinary`, the same with 64-bit offsets.
- `ustore_strings_accessor_k` = `2`: Strings found through a `[ustore_strings_get_t](#ustore_strings_get_t)` accessor, which inputs alone use.

## ustore_strings_view_t

A ragged batch of byte strings in one of three layouts: an Arrow tape of 32-bit or 64-bit offsets, or an accessor writing views over any other shape, like an array of runtime objects or of records.

```c
struct ustore_strings_view_t
```

Keys of a strings collection, binary values, documents, names and field paths all arrive in it. As an input it holds one string per task, or as many as its verb counts otherwise, and a zeroed view, a u32 tape with `NULL` offsets, stands for the input's default. The accessor runs on the thread that runs the call, in blocks of up to 64 indices, never concurrently with itself and never after the call is final. As an output the call writes a tape, with 32-bit offsets while its bytes fit in 4 GiB and 64-bit beyond.

- `layout` `ustore_strings_layout_t`: Which member of `data` describes the strings.
- `data` `union ustore_strings_view_t::@033110322220322170162205100253033163301270125152`

## ustore_length_t

The length of a value in bytes, or an offset into a run of keys, wide enough for a run past 4 GB.

```c
typedef uint64_t ustore_length_t;
```

## ustore_lengths_view_t

A strided run of lengths or positions, one per task.

```c
struct ustore_lengths_view_t
```

- `data` `ustore_length_t const *`: First length of the run.
- `stride` `ustore_size_t`: Bytes between consecutive lengths, where zero repeats the first one.

## ustore_durability_t

How durable the writes a call publishes are once the call finishes.

```c
enum ustore_durability_t
```

- `ustore_durability_buffered_k` = `0`: The writes are visible, and reach the disk as the engine's buffers fill.
- `ustore_durability_flushed_k` = `1`: The writes are on disk, at a cost that varies by engine and drive.

Spellings: ```
char const * ustore_durability_name(ustore_durability_t durability);
```

```
ustore_durability_t ustore_durability_from_name(
    ustore_bytes_view_t name);
```

## ustore_ingest_order_t

In what order an ingest's rows arrive, which decides whether its appends land as they come or wait to be sorted.

```c
enum ustore_ingest_order_t
```

- `ustore_ingest_sorted_k` = `0`: Keys strictly ascend across every append, checked as each one is written into runs of its own.
- `ustore_ingest_unsorted_k` = `1`: Keys arrive in any order and may repeat, the last row of a key winning; every append is held until finished.

Spellings: ```
char const * ustore_ingest_order_name(ustore_ingest_order_t order);
```

```
ustore_ingest_order_t ustore_ingest_order_from_name(
    ustore_bytes_view_t name);
```

## ustore_ingest_t

An ingest under way into one collection, which lands every row appended to it as one commit of its own.

```c
typedef struct ustore_ingest_handle_t * ustore_ingest_t;
```

It joins no transaction and watches nothing: every row lands at one stamp, so a reader sees all of the ingest or none of it, and a transaction that began before it and wrote a key it lands fails at commit. Its calls run on the calling thread, one at a time. On NVMe it holds the database's maintenance claim from its init to its finish or free, so flushes, compactions, creates and drops wait for it meanwhile.

## ustore_strings_cursor_t

An opaque handle to a walk over the strings of one prefix, which every pull resumes past the last string the pull before it handed out.

```c
typedef struct ustore_strings_cursor_handle_t * ustore_strings_cursor_t;
```

It keeps a copy of its prefix and of the string its next pull starts at, and holds its database open, and the transaction or snapshot it reads through, until [`ustore_strings_cursor_free`](#ustore_strings_cursor_free), as `[ustore_blobs_cursor_t](https://www.unum.cloud/docs/ustore/c/scans.md#ustore_blobs_cursor_t)` does. A cursor serves one pull at a time, so a second pull naming it finishes [`ustore_status_contended_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_contended_k).

## ustore_ingest_init

Opens an ingest into one collection.

```c
void ustore_ingest_init(ustore_ingest_init_t *arguments);
```

- `ingest`: Output for the ingest, freed with [`ustore_ingest_free`](#ustore_ingest_free); `NULL` on failure.
- `database`: An open database, which stays open until the ingest is freed.
- `outcome`: Output for the outcome, final on return.
- `collection`: Collection the rows land in.
- `order`: In what order the rows arrive.

Statuses: - [`ustore_status_uninitialized_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_uninitialized_k) The `database` is `NULL`.
- [`ustore_status_invalid_argument_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_invalid_argument_k) The `ingest` output is `NULL`, or the `order` is unknown.
- [`ustore_status_read_only_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_read_only_k) The database was opened read-only.
- [`ustore_status_unsupported_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_unsupported_k) The engine ingests nothing, or not into that collection's modality.
- [`ustore_status_not_found_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_not_found_k) The collection does not exist.

## ustore_ingest_append

Hands an ingest one batch of rows.

```c
void ustore_ingest_append(ustore_ingest_append_t *arguments);
```

- `ingest`: The ingest, which a failed append leaves as it was.
- `outcome`: Output for the outcome, final on return.
- `count`: Number of rows.
- `keys`: Keys of blobs, documents and pages, or the keys strings map onto.
- `values`: Values of blobs, JSON documents and XML pages, or the strings of a strings collection.
- `sources`: Sources of edges.
- `targets`: Targets of edges.
- `edges`: Identifiers of edges, where `NULL` data takes [`ustore_edge_id_default_k`](https://www.unum.cloud/docs/ustore/c/graphs.md#ustore_edge_id_default_k) for each.

Statuses: - [`ustore_status_invalid_argument_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_invalid_argument_k) The `ingest` is `NULL` or finished, a view the modality needs is missing, the `values` come in an unknown layout, or a key does not ascend past every key before it under [`ustore_ingest_sorted_k`](#ustore_ingest_sorted_k).
- [`ustore_status_malformed_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_malformed_k) A value is past what the collection admits, a document no JSON, or a page no XML.
- [`ustore_status_capacity_exhausted_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_capacity_exhausted_k) The volume is full.

## ustore_ingest_finish

Lands every row appended to an ingest as one commit.

```c
void ustore_ingest_finish(ustore_ingest_finish_t *arguments);
```

- `ingest`: The ingest, finished on return whatever the outcome, which only [`ustore_ingest_free`](#ustore_ingest_free) still takes.
- `outcome`: Output for the outcome, final on return.
- `durability`: Buffered: visible on return, durable at the next flush, dropped whole by a crash before it.
- `commit_stamp`: Optional output for the one stamp every row landed at; zeroed on failure.

Statuses: - [`ustore_status_invalid_argument_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_invalid_argument_k) The `ingest` is `NULL` or finished, or the `durability` is unknown.
- [`ustore_status_not_found_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_not_found_k) The collection was dropped meanwhile.
- [`ustore_status_contended_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_contended_k) Memory runs kept refilling while the ingest waited them out.
- [`ustore_status_capacity_exhausted_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_capacity_exhausted_k) The volume is full.

## ustore_ingest_free

Frees an ingest, dropping every row of an unfinished one; `NULL` is safe.

```c
void ustore_ingest_free(ustore_ingest_t ingest);
```

## ustore_blobs_write

Writes a batch of values.

```c
void ustore_blobs_write(ustore_blobs_write_t *arguments);
```

[`ustore_status_invalid_argument_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_invalid_argument_k) means the `collections`, `keys` or `values` are missing, the values come in an unknown layout, the `transaction` belongs to another engine or is staged, the `durability` or `watch` is unknown, a flushed write names a `transaction`, [`ustore_watch_none_k`](https://www.unum.cloud/docs/ustore/c/transactions.md#ustore_watch_none_k) comes without one, or a collection holds another modality.

- [`ustore_status_unsupported_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_unsupported_k) The executor computes on a device.
- [`ustore_status_not_found_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_not_found_k) A collection does not exist or was dropped meanwhile.
- [`ustore_status_write_conflict_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_write_conflict_k) Concurrent commits kept conflicting, or a staged RocksDB transaction holds a key.
- [`ustore_status_malformed_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_malformed_k) A key or value falls outside what its NVMe collection admits.
- [`ustore_status_capacity_exhausted_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_capacity_exhausted_k) The values outgrow the NVMe transaction's buffer.

- `database`: An open database.
- `executor`: Executor that runs the call, where `NULL` runs it on the calling thread.
- `outcome`: Output for the outcome, final once its status leaves [`ustore_status_deferred_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_deferred_k).
- `completion`: Whether the call returns final, the default, or once submitted.
- `transaction`: Transaction the writes join, where `NULL` writes to the head.
- `count`: Number of tasks, one per key.
- `collections`: Collections owning the keys.
- `keys`: Keys to write.
- `values`: Values to write, one per key; a tape [`ustore_blobs_read`](#ustore_blobs_read) wrote is written back as it is.
- `durability`: How durable the writes are once the call finishes at the head; a transaction's commit decides its own.
- `watch`: Whether the `transaction` records the written keys for conflicts.

Statuses: - [`ustore_status_contended_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_contended_k) The `transaction` serves another call, the executor's deferred table is full, or a callback blocks on the executor running it.
- [`ustore_status_uninitialized_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_uninitialized_k) The `database` is `NULL`.
- [`ustore_status_read_only_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_read_only_k) The database was opened read-only.

## ustore_blobs_remove

Removes a batch of keys with their values.

```c
void ustore_blobs_remove(ustore_blobs_remove_t *arguments);
```

[`ustore_status_invalid_argument_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_invalid_argument_k) means the `collections` or `keys` are missing, the `transaction` belongs to another engine or is staged, the `durability` or `watch` is unknown, a flushed remove names a `transaction`, [`ustore_watch_none_k`](https://www.unum.cloud/docs/ustore/c/transactions.md#ustore_watch_none_k) comes without one, or a collection holds another modality.

- [`ustore_status_unsupported_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_unsupported_k) The executor computes on a device.
- [`ustore_status_not_found_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_not_found_k) A collection does not exist or was dropped meanwhile.
- [`ustore_status_write_conflict_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_write_conflict_k) Concurrent commits kept conflicting, or a staged RocksDB transaction holds a key.
- [`ustore_status_malformed_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_malformed_k) A key falls outside what its NVMe collection admits.
- [`ustore_status_capacity_exhausted_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_capacity_exhausted_k) The removes outgrow the NVMe transaction's buffer.

- `database`: An open database.
- `executor`: Executor that runs the call, where `NULL` runs it on the calling thread.
- `outcome`: Output for the outcome, final once its status leaves [`ustore_status_deferred_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_deferred_k).
- `completion`: Whether the call returns final, the default, or once submitted.
- `transaction`: Transaction the removals join, where `NULL` removes at the head.
- `count`: Number of tasks, one per key.
- `collections`: Collections owning the keys.
- `keys`: Keys to remove, or the inclusive starts of ranges when `end_keys` is given.
- `end_keys`: Exclusive ends of the ranges to remove, or a zeroed view to remove `keys` alone.
- `durability`: How durable the removals are once the call finishes at the head; a transaction's commit decides its own.
- `watch`: Whether the `transaction` records the removed keys for conflicts.

Statuses: - [`ustore_status_contended_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_contended_k) The `transaction` serves another call, the executor's deferred table is full, or a callback blocks on the executor running it.
- [`ustore_status_uninitialized_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_uninitialized_k) The `database` is `NULL`.
- [`ustore_status_read_only_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_read_only_k) The database was opened read-only.

## ustore_blobs_read

Reads a batch of values.

```c
void ustore_blobs_read(ustore_blobs_read_t *arguments);
```

- `database`: An open database.
- `executor`: Executor that runs the call, where `NULL` runs it on the calling thread.
- `outcome`: Output for the outcome, final once its status leaves [`ustore_status_deferred_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_deferred_k).
- `completion`: Whether the call returns final, the default, or once submitted.
- `view`: What the call sees, where `NULL` is the head.
- `count`: Number of tasks, one per key.
- `collections`: Collections owning the keys.
- `keys`: Keys to read.
- `offsets`: Byte offset into each value the read starts at, or a zeroed view for whole values.
- `limits`: Bytes to read of each value from its offset, clipped, or a zeroed view for the rest.
- `presences`: Optional output for a bitset of `count` bits marking present keys, inside `arena`.
- `lengths`: Optional output for the length of every value, for reads that skip the bytes, inside `arena`.
- `values`: Optional output for the values as a tape inside `arena`, where a missing key reads as an empty value.
- `arena`: Rewound by this call, then holding every output it writes; `NULL` is refused.

Statuses: - [`ustore_status_invalid_argument_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_invalid_argument_k) The `arena`, `collections` or `keys` are missing, the `view` was made by neither view constructor or belongs to another engine, a watching read goes through a staged transaction, or a collection holds another modality.
- [`ustore_status_device_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_device_k) The arena lives where another executor computes.
- [`ustore_status_contended_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_contended_k) The arena or the transaction the `view` reads through serves another call, the executor's deferred table is full, a callback blocks on the executor running it, or an NVMe value keeps changing size across reads.
- [`ustore_status_uninitialized_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_uninitialized_k) The `database` is `NULL` or an NVMe transfer was cancelled.
- [`ustore_status_unsupported_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_unsupported_k) The executor computes on a device, where no engine serves the call yet.
- [`ustore_status_not_found_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_not_found_k) A collection does not exist or was dropped meanwhile.
- [`ustore_status_capacity_exhausted_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_capacity_exhausted_k) Every shared NVMe channel is taken.

## ustore_strings_write

Writes a batch of strings and the keys they map onto.

```c
void ustore_strings_write(ustore_strings_write_t *arguments);
```

- `database`: An open database.
- `executor`: Executor that runs the call, where `NULL` runs it on the calling thread.
- `outcome`: Output for the outcome, final once its status leaves [`ustore_status_deferred_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_deferred_k).
- `completion`: Whether the call returns final, the default, or once submitted.
- `transaction`: Transaction the writes join, where `NULL` writes to the head.
- `count`: Number of tasks, one per string.
- `collections`: Strings collections owning the strings.
- `strings`: Strings to write, one per task.
- `values`: The key every string maps onto, one per string.
- `durability`: How durable the writes are once the call finishes at the head; a transaction's commit decides its own.
- `watch`: Whether the `transaction` records the strings it reads and writes for conflicts.

Statuses: - [`ustore_status_contended_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_contended_k) The `transaction` serves another call, the executor's deferred table is full, or a callback blocks on the executor running it.
- [`ustore_status_uninitialized_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_uninitialized_k) The `database` is `NULL`.
- [`ustore_status_read_only_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_read_only_k) The database was opened read-only.
- [`ustore_status_invalid_argument_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_invalid_argument_k) The `collections`, `strings` or `values` are missing or malformed, the `transaction` is foreign or staged, the `durability` or `watch` is unknown, a flushed write names a `transaction`, or a collection holds another modality.
- [`ustore_status_unsupported_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_unsupported_k) The executor computes on a device.
- [`ustore_status_not_found_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_not_found_k) A collection does not exist or was dropped meanwhile.
- [`ustore_status_write_conflict_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_write_conflict_k) Concurrent commits kept conflicting, or a staged RocksDB transaction holds a string.
- [`ustore_status_capacity_exhausted_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_capacity_exhausted_k) The strings outgrow one NVMe run file.

## ustore_strings_remove

Removes a batch of strings with their values.

```c
void ustore_strings_remove(ustore_strings_remove_t *arguments);
```

[`ustore_status_invalid_argument_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_invalid_argument_k) means the `collections` or `strings` are missing, the strings come in an unknown layout, the `transaction` belongs to another engine or is staged, the `durability` or `watch` is unknown, a flushed remove names a `transaction`, or a collection holds another modality.

- [`ustore_status_unsupported_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_unsupported_k) The executor computes on a device.
- [`ustore_status_not_found_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_not_found_k) A collection does not exist or was dropped meanwhile.
- [`ustore_status_write_conflict_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_write_conflict_k) Concurrent commits kept conflicting, or a staged RocksDB transaction holds a string.
- [`ustore_status_capacity_exhausted_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_capacity_exhausted_k) The strings outgrow one NVMe run file.

- `database`: An open database.
- `executor`: Executor that runs the call, where `NULL` runs it on the calling thread.
- `outcome`: Output for the outcome, final once its status leaves [`ustore_status_deferred_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_deferred_k).
- `completion`: Whether the call returns final, the default, or once submitted.
- `transaction`: Transaction the removals join, where `NULL` removes at the head.
- `count`: Number of tasks, one per string.
- `collections`: Strings collections owning the strings.
- `strings`: Strings to remove, one per task, or inclusive range starts when `end_strings` is given.
- `end_strings`: Exclusive ends of the ranges to remove, where an empty one bounds nothing, or a zeroed view to remove `strings` alone.
- `durability`: How durable the removals are once the call finishes at the head; a transaction's commit decides its own.
- `watch`: Whether the `transaction` records the strings it reads and writes for conflicts.

Statuses: - [`ustore_status_contended_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_contended_k) The `transaction` serves another call, the executor's deferred table is full, or a callback blocks on the executor running it.
- [`ustore_status_uninitialized_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_uninitialized_k) The `database` is `NULL`.
- [`ustore_status_read_only_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_read_only_k) The database was opened read-only.

## ustore_strings_read

Reads the keys a batch of strings map onto.

```c
void ustore_strings_read(ustore_strings_read_t *arguments);
```

- `database`: An open database.
- `executor`: Executor that runs the call, where `NULL` runs it on the calling thread.
- `outcome`: Output for the outcome, final once its status leaves [`ustore_status_deferred_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_deferred_k).
- `completion`: Whether the call returns final, the default, or once submitted.
- `view`: What the call sees, where `NULL` is the head.
- `count`: Number of tasks, one per string.
- `collections`: Strings collections owning the strings.
- `strings`: Strings to read, one per task.
- `presences`: Optional output for a bitset of `count` bits marking present strings, addressed as Apache Arrow does, inside `arena`.
- `values`: Optional output for the key of every string inside `arena`, in the order of the strings, where a missing string reads as the nil UUID.
- `arena`: Rewound by this call, then holding every output it writes; `NULL` is refused.

Statuses: - [`ustore_status_invalid_argument_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_invalid_argument_k) The `arena`, `collections` or `strings` are missing, the strings come in an unknown layout, the `view` was made by neither view constructor or belongs to another engine, a watching read goes through a staged transaction, or a collection holds another modality.
- [`ustore_status_device_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_device_k) The arena lives where another executor computes.
- [`ustore_status_contended_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_contended_k) The arena or the transaction the `view` reads through serves another call, the executor's deferred table is full, or a callback blocks on the executor running it.
- [`ustore_status_uninitialized_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_uninitialized_k) The `database` is `NULL`.
- [`ustore_status_not_found_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_not_found_k) A collection does not exist or was dropped meanwhile.

## ustore_strings_scan

Scans a batch of prefixes.

```c
void ustore_strings_scan(ustore_strings_scan_t *arguments);
```

- [`ustore_status_device_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_device_k) The arena lives where another executor computes.
- [`ustore_status_contended_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_contended_k) The arena or the transaction the `view` reads through serves another call, the executor's deferred table is full, or a callback blocks on the executor running it.
- [`ustore_status_uninitialized_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_uninitialized_k) The `database` is `NULL`.
- [`ustore_status_unsupported_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_unsupported_k) The executor's device loaded no bundle of the library's kernels and `patterns` are given.
- [`ustore_status_not_found_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_not_found_k) A collection does not exist or was dropped meanwhile.

- `database`: An open database.
- `executor`: Executor that runs the call, where `NULL` runs it on the calling thread.
- `outcome`: Output for the outcome, final once its status leaves [`ustore_status_deferred_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_deferred_k).
- `completion`: Whether the call returns final, the default, or once submitted.
- `view`: What the call sees, where `NULL` is the head.
- `count`: Number of scans.
- `collections`: Strings collections to scan.
- `prefixes`: Prefixes every scan matches, where a zeroed view matches every string.
- `start_strings`: String every scan starts from, inclusive, which paginates a walk of one prefix; a zeroed view starts at the first string in the `direction`.
- `end_strings`: Exclusive string every scan stops before, where empty or zeroed views bound nothing.
- `direction`: Ascending, the default, or descending from the start string.
- `patterns`: Optional patterns every listed string matches, shared by every scan of the call; zeroed lists every string.
- `predicate`: Optional verdict on the keys the listed strings map onto, asked once per page as its `cost` says; host executors only.
- `offsets`: Output for `count` plus one offsets into `strings`, the last one closing the final scan, inside `arena`.
- `strings`: Output for the found strings as a tape inside `arena`, scan after scan.
- `scores`: Optional output for one BM25 score per listed string when `patterns` carry weights, inside `arena`; `NULL` skips them.
- `arena`: Rewound by this call, then holding every output it writes; `NULL` is refused.
- `wanted`: Most strings every scan returns.
- `mode`: In what order the strings come.
- `progress`: Optional callback reporting the scans finished.
- `progress_state`: Opaque pointer passed to `progress`.

Statuses: [`ustore_status_invalid_argument_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_invalid_argument_k) means the `arena`, `collections` or an output is missing, the prefixes, start or end strings come in an unknown layout, the `direction`, a pattern kind, a case sensitivity or a predicate cost is unknown, the `mode` is unknown under `patterns` or a predicate, the `patterns` do not compile, the `view` was made by neither view constructor or belongs to another engine, a watching scan goes through a staged transaction, or a collection holds a modality other than strings.

## ustore_strings_cursor_init

Opens a walk over the strings of one prefix, reading nothing until its first pull.

```c
void ustore_strings_cursor_init(
    ustore_strings_cursor_init_t *arguments);
```

- `cursor`: Output for the new cursor, which the caller frees with [`ustore_strings_cursor_free`](#ustore_strings_cursor_free); `NULL` on failure.
- `database`: An open database.
- `executor`: Executor of the kind every pull runs on, which picks the side the walk computes on, where `NULL` walks on host executors; a pull on another kind of executor reports [`ustore_status_device_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_device_k).
- `outcome`: Output for the outcome, final on return.
- `view`: What the walk sees, where `NULL` is the head, read pull by pull.
- `collection`: Strings collection to walk.
- `prefix`: Prefix every string the walk returns starts with, copied, where a zeroed view matches every string.
- `start_string`: String the walk starts from, inclusive, copied, where a zeroed view starts at the first string of the prefix in the `direction`.
- `end_string`: String the walk stops before, exclusive, copied, or a zeroed view for no bound.
- `direction`: Ascending, the default, or descending from the start string.
- `patterns`: Optional patterns every string the walk hands out matches, compiled here once; zeroed walks every string.
- `predicate`: Optional verdict on the keys of the strings of every page, asked as its `cost` says; host executors only.

Statuses: - [`ustore_status_invalid_argument_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_invalid_argument_k) The `cursor` output is `NULL`, the `view` was made by neither view constructor or reads another database, the `direction`, a pattern kind, a case sensitivity or a predicate cost is unknown, or the `patterns` do not compile.
- [`ustore_status_uninitialized_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_uninitialized_k) The `database` is `NULL`.

## ustore_strings_cursor_next

Pulls the next page of a walk, and the values of its strings when asked.

```c
void ustore_strings_cursor_next(
    ustore_strings_cursor_next_t *arguments);
```

- `database`: The database the cursor was opened on.
- `executor`: Executor that runs the call, where `NULL` runs it on the calling thread.
- `outcome`: Output for the outcome, final once its status leaves [`ustore_status_deferred_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_deferred_k).
- `completion`: Whether the call returns final, the default, or once submitted.
- `cursor`: The cursor to advance, which a pull that fails or is cancelled leaves where it was.
- `strings`: Output for the strings of the page in the order of their text, as a tape inside `arena`.
- `values`: Optional output for the keys of the page's strings inside `arena`, where a string removed at the head between the listing and the read of one pull reads as the nil UUID; `NULL` skips them.
- `written`: Output for the strings written, zero once the walk is exhausted; zeroed on failure.
- `arena`: Rewound by this call, then holding every output it writes; `NULL` is refused.
- `wanted`: Most strings the page holds, where a page of fewer exhausts the walk.
- `progress`: Optional callback reporting the one pull, which may cancel it before the cursor advances.
- `progress_state`: Opaque pointer passed to `progress`.

Statuses: - [`ustore_status_invalid_argument_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_invalid_argument_k) The `cursor`, the `arena` or an output is `NULL`, the cursor walks another database, a watching pull goes through a staged transaction, or the collection holds another modality.
- [`ustore_status_device_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_device_k) The arena lives where another executor computes, a cursor opened for a device is pulled on host threads, or a host cursor is pulled on a device.
- [`ustore_status_contended_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_contended_k) The arena, the cursor or the transaction it reads through serves another call, the executor's deferred table is full, or a callback blocks on the executor running it.
- [`ustore_status_uninitialized_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_uninitialized_k) The `database` is `NULL`.
- [`ustore_status_unsupported_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_unsupported_k) The pulling executor's device loaded no bundle of the library's kernels and the walk has `patterns`.
- [`ustore_status_not_found_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_not_found_k) The collection does not exist or was dropped meanwhile.

## ustore_strings_cursor_free

Waits until every deferred call naming the cursor finishes, then frees it, giving back the holds it kept on its database and on the transaction or snapshot it reads through.

```c
void ustore_strings_cursor_free(ustore_strings_cursor_t cursor);
```

- `cursor` (inout): The cursor to be freed; `NULL` is safe.

## ustore_strings_sample

Samples strings from a batch of collections.

```c
void ustore_strings_sample(ustore_strings_sample_t *arguments);
```

- `database`: An open database.
- `executor`: Executor that runs the call, where `NULL` runs it on the calling thread.
- `outcome`: Output for the outcome, final once its status leaves [`ustore_status_deferred_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_deferred_k).
- `completion`: Whether the call returns final, the default, or once submitted.
- `view`: What the call sees, where `NULL` is the head.
- `count`: Number of samples.
- `collections`: Strings collections to sample.
- `offsets`: Output for `count` plus one offsets into `strings`, the last one closing the final sample, inside `arena`.
- `strings`: Output for the sampled strings as a tape, sample after sample, inside `arena`.
- `arena`: Rewound by this call, then holding every output it writes; `NULL` is refused.
- `wanted`: Most strings every sample draws.

Statuses: - [`ustore_status_invalid_argument_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_invalid_argument_k) The `arena`, `collections` or an output is missing, the `view` was made by neither view constructor or belongs to another engine, a watching sample goes through a staged transaction, or a collection holds another modality.
- [`ustore_status_device_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_device_k) The arena lives where another executor computes.
- [`ustore_status_contended_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_contended_k) The arena or the transaction the `view` reads through serves another call, the executor's deferred table is full, or a callback blocks on the executor running it.
- [`ustore_status_uninitialized_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_uninitialized_k) The `database` is `NULL` or an NVMe transfer was cancelled.
- [`ustore_status_unsupported_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_unsupported_k) The executor computes on a device.
- [`ustore_status_not_found_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_not_found_k) A collection does not exist or was dropped meanwhile.
- [`ustore_status_capacity_exhausted_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_capacity_exhausted_k) Every shared NVMe channel is taken.

## ustore_strings_measure

Estimates the strings and the space inside a batch of string ranges.

```c
void ustore_strings_measure(ustore_strings_measure_t *arguments);
```

[`ustore_status_invalid_argument_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_invalid_argument_k) means the `arena` or the `collections` are missing, the strings come in an unknown layout, the `direction` is unknown, the `view` was made by neither view constructor or belongs to another engine, a watching measure goes through a staged transaction, or a collection holds another modality.

- `database`: An open database.
- `executor`: Executor that runs the call, where `NULL` runs it on the calling thread.
- `outcome`: Output for the outcome, final once its status leaves [`ustore_status_deferred_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_deferred_k).
- `completion`: Whether the call returns final, the default, or once submitted.
- `view`: What the call sees, where `NULL` is the head.
- `count`: Number of measured ranges.
- `collections`: Strings collections to measure.
- `start_strings`: String every range starts from, inclusive, where a zeroed view starts at the first string in the `direction`.
- `end_strings`: String every range stops before, exclusive, where an empty one bounds nothing, or a zeroed view for no bound.
- `direction`: Ascending, the default, or descending from the start string.
- `min_cardinalities`: Optional output for the lower bound on the strings in every range, inside `arena`.
- `max_cardinalities`: Optional output for the upper bound on the strings in every range, inside `arena`.
- `min_value_bytes`: Optional output for the lower bound on the bytes of the strings and their keys in every range, inside `arena`.
- `max_value_bytes`: Optional output for the upper bound on the bytes of the strings and their keys in every range, inside `arena`.
- `min_space_usages`: Optional output for the lower bound on the disk space every range takes, inside `arena`.
- `max_space_usages`: Optional output for the upper bound on the disk space every range takes, inside `arena`.
- `arena`: Rewound by this call, then holding every output it writes; `NULL` is refused.

Statuses: - [`ustore_status_device_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_device_k) The arena lives where another executor computes.
- [`ustore_status_contended_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_contended_k) The arena or the transaction the `view` reads through serves another call, the executor's deferred table is full, or a callback blocks on the executor running it.
- [`ustore_status_uninitialized_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_uninitialized_k) The `database` is `NULL` or an NVMe transfer was cancelled.
- [`ustore_status_unsupported_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_unsupported_k) The executor computes on a device.
- [`ustore_status_not_found_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_not_found_k) A collection does not exist or was dropped meanwhile.
- [`ustore_status_capacity_exhausted_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_capacity_exhausted_k) Every shared NVMe channel is taken.

## ustore_keys_view_from_arrow

Views the 16-byte keys of one Arrow `fixed_size_binary(16)` column in place, tagged `arrow.uuid` or not.

```c
ustore_status_t ustore_keys_view_from_arrow(
    struct ArrowSchema const *schema,
    struct ArrowArray const *array,
    ustore_keys_view_t *keys,
    ustore_size_t *count);
```

A `null_count` of -1 with a validity buffer may hold a null.

- `schema` (in): The column's schema.
- `array` (in): The column, whose `offset` is applied; only its lengths and pointers are read, never its buffers, so the `array` of an `ArrowDeviceArray` converts as well.
- `keys` (out): The keys, one every 16 bytes.
- `count` (out): The number of keys.

## ustore_strings_view_from_arrow

Views the byte strings of one Arrow column in place: `binary` or `utf8` as a tape of 32-bit offsets, and `large_binary` or `large_utf8` as one of 64-bit offsets, starting at the column's `offset`.

```c
ustore_status_t ustore_strings_view_from_arrow(
    struct ArrowSchema const *schema,
    struct ArrowArray const *array,
    ustore_strings_view_t *strings,
    ustore_size_t *count);
```

## ustore_keys_view_to_arrow

Exports `count` keys a call wrote into `arena` as one Arrow `fixed_size_binary(16)` column tagged as `arrow.uuid`, in place.

```c
ustore_status_t ustore_keys_view_to_arrow(
    ustore_keys_view_t keys,
    ustore_size_t count,
    ustore_arena_t arena,
    struct ArrowSchema *schema,
    struct ArrowDeviceArray *array);
```

The array reads `arena's` outputs until the arena's next call or free, like every other output view, and its `release` frees only the array's own nodes. Arenas on host pages export as `ARROW_DEVICE_CPU`, and those of a VRAM executor as `ARROW_DEVICE_CUDA` or `ARROW_DEVICE_ROCM` with the vendor's ordinal of their device and no `sync_event`, as the call that filled one waited for its device. The schema's `name` is empty, for the caller to point at its own.

Keys of another stride, like the triples of edges, are copied dense into `arena` first, under the claim a call takes, so no other call or carving export runs on `arena` meanwhile; a device arena's are copied on its device, through an executor the export spawns there.

Statuses: - [`ustore_status_invalid_argument_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_invalid_argument_k) An output is missing, or dense keys lie outside `arena`.
- [`ustore_status_contended_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_contended_k) Keys of another stride met a call filling `arena`, or another export carving it, while being copied dense.
- [`ustore_status_device_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_device_k) No executor spawns on the arena's device, or it has no kernels.
- [`ustore_status_out_of_memory_k`](https://www.unum.cloud/docs/ustore/c/errors.md#ustore_status_out_of_memory_k) An allocation failed.

## ustore_strings_view_to_arrow

Exports a tape of `count` strings a call wrote into `arena` as one Arrow column of `layout`, in place when the tape already has it.

```c
ustore_status_t ustore_strings_view_to_arrow(
    ustore_strings_view_t strings,
    ustore_size_t count,
    ustore_strings_layout_t layout,
    ustore_arena_t arena,
    struct ArrowSchema *schema,
    struct ArrowDeviceArray *array);
```

A [`ustore_strings_u32_tape_k`](#ustore_strings_u32_tape_k) layout exports as `binary` and a [`ustore_strings_u64_tape_k`](#ustore_strings_u64_tape_k) one as `large_binary`, living until `arena's` next call or free as [`ustore_keys_view_to_arrow`](#ustore_keys_view_to_arrow) does. A tape of the other width has its offsets converted into `arena` first, under the claim, narrowed ones counted from the tape's first byte; a device arena's offsets cross to the host and back through an executor the export spawns on that device.