C · Reads and Writes

Reads and Writes

#include <ustore/types.h>#include <ustore/db.h>#include <ustore/blobs.h>#include <ustore/strings.h>#include <ustore/arrow.h> 1.0.0-rc.1 · d6d40961

§ Writes and Removals

ustore_blobs_write takes its keys as a ustore_keys_view_t and its values as a ustore_strings_view_t, whose layout names how the values lie:

  • ustore_strings_u32_tape_k is Apache Arrow’s binary: the bytes end to end, and count + 1 offsets of 32 bits, where value index spans from offsets[index] up to offsets[index + 1].
  • ustore_strings_u64_tape_k is Arrow’s large_binary: the same with 64-bit offsets.
  • ustore_strings_accessor_k holds a ustore_strings_get_t callback and the handle it reads, and the callback writes the view of every value whose index it is asked for.

A single write passes a tape of one value:

ustore_key_t key = {.uuid.bytes[15] = 42};
char const value[] = "meaning of life";
uint32_t const offsets[2] = {0, 15};
ustore_blobs_write(&(ustore_blobs_write_t){
    .database = database,
    .outcome = &outcome,
    .count = 1,
    .collections = {&values, 0},
    .keys = {&key, sizeof(ustore_key_t)},
    .values = {.layout = ustore_strings_u32_tape_k, .data.u32_tape = {(uint8_t const*)value, offsets}},
});

A batch held in an array of structures needs no copy into separate arrays, since an accessor can read the values out of the records:

typedef struct pair_t {
    ustore_key_t key;
    char const* text;
} pair_t;

static void get_texts(void const* handle, ustore_size_t const* indices, ustore_size_t count,
                      ustore_bytes_view_t* views) {
    pair_t const* pairs = (pair_t const*)handle;
    for (ustore_size_t index = 0; index != count; ++index) {
        char const* text = pairs[indices[index]].text;
        views[index] = (ustore_bytes_view_t){(uint8_t const*)text, strlen(text)};
    }
}

The write then takes the keys as a strided run over the same records:

pair_t const pairs[2] = {
    {{.uuid.bytes[15] = 42}, "meaning of life"},
    {{.uuid.bytes[15] = 43}, "is unknown"},
};
ustore_blobs_write(&(ustore_blobs_write_t){
    .database = database,
    .outcome = &outcome,
    .count = 2,
    .collections = {&values, 0},
    .keys = {&pairs[0].key, sizeof(pair_t)},
    .values = {.layout = ustore_strings_accessor_k, .data.accessor = {get_texts, pairs}},
});

The accessor runs on the thread that runs the call, in blocks of up to 64 indices, never concurrently with itself and never after the call is final. An accessor writing the same view for every index writes one value under every key, as a zero stride repeats one key. Removing keys is its own call, so no value ever deletes, and an empty value is still a value:

ustore_blobs_remove(&(ustore_blobs_remove_t){
    .database = database, .outcome = &outcome, .count = 1, .collections = {&values, 0}, .keys = {&key, 0}});

A batch is atomic: if one task fails, none is applied. With end_keys, every task removes the range from its key, inclusive, to its end key, exclusive, in the same atomic batch; graphs name the field end_vertices, strings end_strings, and a range of pages removes whole pages. durability chooses whether the call returns once the write is buffered or once it is flushed.

§ Reads and Arenas

Every output whose length scales with the call, the values, their lengths and the bitmap of presences alike, lands in an arena. ustore_arena_init makes one where its executor computes, or on host pages for a NULL executor, and every call given it rewinds it and refills it, growing only when the new outputs outsize everything it held, so a steady loop allocates once. Only fixed-size reports, such as descriptions, stamps, the written count of a listing and created handles, stay caller fields, and a call that fails zeroes the scalars among them and leaves the handles NULL.

ustore_key_t key = {.uuid.bytes[15] = 42};
ustore_octet_t const* presences = NULL;
ustore_strings_view_t values = {0};
ustore_blobs_read(&(ustore_blobs_read_t){
    .database = database,
    .outcome = &outcome,
    .count = 1,
    .collections = {&values, 0},
    .keys = {&key, sizeof(ustore_key_t)},
    .presences = &presences,
    .values = &values,
    .arena = arena,
});
uint32_t const* offsets = values.data.u32_tape.offsets;
assert(values.layout == ustore_strings_u32_tape_k && offsets[1] - offsets[0] == 15);
assert(memcmp(values.data.u32_tape.data + offsets[0], "meaning of life", 15) == 0);

Values are joined into one tape as Apache Arrow lays them out: count + 1 offsets, the last one closing the final value, and the bytes end to end. The offsets are 32 bits wide while the bytes fit in 4 GiB and 64 bits beyond, so a reader checks layout once per call, and a tape a read wrote is written back as it is, without a copy. Every output is optional: .lengths exports the length of every value without its bytes, .presences a bitmap of the keys that are present, and a missing key reads as an empty value. offsets and limits read the bytes of every value from its offset, at most its limit of them, and report that slice’s length, where an offset past the end reads empty. The outputs stay valid until the next call given arena; to keep them past it, pass that call another arena. A NULL arena is refused with ustore_status_invalid_argument_k, an arena serves one call at a time, and a call that fails, is cancelled or has nothing to report leaves its outputs NULL. ustore_arena_free waits until every deferred call naming the arena is final, and ustore_arena_describe reports whether an arena’s outputs live on host pages or on which device.

API

Types

ustore_strings_get_tWrites the view of every string indices[i] into views[i], for strings no tape holds, and accepts any count.
types.h:211 ↗
typedef void(*) ustore_strings_get_t(void const *handle, ustore_size_t const *indices, ustore_size_t count, ustore_bytes_view_t *views);
ustore_strings_layout_tHow a batch of strings holds its bytes, naming the member of its data union to read.
types.h:215 ↗
enum ustore_strings_layout_t

Values

ustore_strings_u32_tape_k0Apache Arrow Binary, where string i spans [offsets[i],offsets[i+1]) of the bytes.
ustore_strings_u64_tape_k1Apache Arrow LargeBinary, the same with 64-bit offsets.
ustore_strings_accessor_k2Strings found through a ustore_strings_get_t accessor, which inputs alone use.
ustore_strings_view_tA ragged batch of byte strings in one of three layouts: an Arrow tape of 32-bit or 64-bit offsets, or an accessor writing views over any other shape, like an array of runtime objects or of records.
types.h:238 ↗
struct ustore_strings_view_t

Keys of a strings collection, binary values, documents, names and field paths all arrive in it. As an input it holds one string per task, or as many as its verb counts otherwise, and a zeroed view, a u32 tape with NULL offsets, stands for the input's default. The accessor runs on the thread that runs the call, in blocks of up to 64 indices, never concurrently with itself and never after the call is final. As an output the call writes a tape, with 32-bit offsets while its bytes fit in 4 GiB and 64-bit beyond.

Fields

layoutustore_strings_layout_tWhich member of data describes the strings.
dataunion ustore_strings_view_t::@033110322220322170162205100253033163301270125152
ustore_length_tThe length of a value in bytes, or an offset into a run of keys, wide enough for a run past 4 GB.
db.h:72 ↗
typedef uint64_t ustore_length_t;
ustore_lengths_view_tA strided run of lengths or positions, one per task.
db.h:105 ↗
struct ustore_lengths_view_t

Fields

dataustore_length_t const *First length of the run.
strideustore_size_tBytes between consecutive lengths, where zero repeats the first one.
ustore_durability_tHow durable the writes a call publishes are once the call finishes.
db.h:167 ↗
enum ustore_durability_t

Values

ustore_durability_buffered_k0The writes are visible, and reach the disk as the engine's buffers fill.
ustore_durability_flushed_k1The writes are on disk, at a cost that varies by engine and drive.

Spellings

char const * ustore_durability_name(ustore_durability_t durability);
ustore_durability_t ustore_durability_from_name(
    ustore_bytes_view_t name);
ustore_ingest_order_tIn what order an ingest's rows arrive, which decides whether its appends land as they come or wait to be sorted.
db.h:1519 ↗
enum ustore_ingest_order_t

Values

ustore_ingest_sorted_k0Keys strictly ascend across every append, checked as each one is written into runs of its own.
ustore_ingest_unsorted_k1Keys arrive in any order and may repeat, the last row of a key winning; every append is held until finished.

Spellings

char const * ustore_ingest_order_name(ustore_ingest_order_t order);
ustore_ingest_order_t ustore_ingest_order_from_name(
    ustore_bytes_view_t name);
ustore_ingest_tAn ingest under way into one collection, which lands every row appended to it as one commit of its own.
db.h:1543 ↗
typedef struct ustore_ingest_handle_t * ustore_ingest_t;

It joins no transaction and watches nothing: every row lands at one stamp, so a reader sees all of the ingest or none of it, and a transaction that began before it and wrote a key it lands fails at commit. Its calls run on the calling thread, one at a time. On NVMe it holds the database's maintenance claim from its init to its finish or free, so flushes, compactions, creates and drops wait for it meanwhile.

ustore_strings_cursor_tAn opaque handle to a walk over the strings of one prefix, which every pull resumes past the last string the pull before it handed out.
strings.h:317 ↗
typedef struct ustore_strings_cursor_handle_t * ustore_strings_cursor_t;

It keeps a copy of its prefix and of the string its next pull starts at, and holds its database open, and the transaction or snapshot it reads through, until ustore_strings_cursor_free, as ustore_blobs_cursor_t does. A cursor serves one pull at a time, so a second pull naming it finishes ustore_status_contended_k.

Functions

ustore_ingest_initOpens an ingest into one collection.
db.h:1575 ↗
void ustore_ingest_init(ustore_ingest_init_t *arguments);

Parameters

ingestustore_ingest_t *Output for the ingest, freed with ustore_ingest_free; NULL on failure.
databaseustore_database_tAn open database, which stays open until the ingest is freed.
outcomeustore_outcome_t *Output for the outcome, final on return.
collectionustore_collection_id_tCollection the rows land in.
orderustore_ingest_order_tIn what order the rows arrive.

Statuses

ustore_ingest_appendHands an ingest one batch of rows.
db.h:1622 ↗
void ustore_ingest_append(ustore_ingest_append_t *arguments);

Parameters

ingestustore_ingest_tThe ingest, which a failed append leaves as it was.
outcomeustore_outcome_t *Output for the outcome, final on return.
countustore_size_tNumber of rows.
keysustore_keys_view_tKeys of blobs, documents and pages, or the keys strings map onto.
valuesustore_strings_view_tValues of blobs, JSON documents and XML pages, or the strings of a strings collection.
sourcesustore_keys_view_tSources of edges.
targetsustore_keys_view_tTargets of edges.
edgesustore_keys_view_tIdentifiers of edges, where NULL data takes ustore_edge_id_default_k for each.

Statuses

ustore_ingest_finishLands every row appended to an ingest as one commit.
db.h:1650 ↗
void ustore_ingest_finish(ustore_ingest_finish_t *arguments);

Parameters

ingestustore_ingest_tThe ingest, finished on return whatever the outcome, which only ustore_ingest_free still takes.
outcomeustore_outcome_t *Output for the outcome, final on return.
durabilityustore_durability_tBuffered: visible on return, durable at the next flush, dropped whole by a crash before it.
commit_stampustore_stamp_t *Optional output for the one stamp every row landed at; zeroed on failure.

Statuses

ustore_ingest_freeFrees an ingest, dropping every row of an unfinished one; NULL is safe.
db.h:1653 ↗
void ustore_ingest_free(ustore_ingest_t ingest);
ustore_blobs_writeWrites a batch of values.
blobs.h:90 ↗
void ustore_blobs_write(ustore_blobs_write_t *arguments);

ustore_status_invalid_argument_k means the collections, keys or values are missing, the values come in an unknown layout, the transaction belongs to another engine or is staged, the durability or watch is unknown, a flushed write names a transaction, ustore_watch_none_k comes without one, or a collection holds another modality.

Parameters

databaseustore_database_tAn open database.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
transactionustore_transaction_tTransaction the writes join, where NULL writes to the head.
countustore_size_tNumber of tasks, one per key.
collectionsustore_collections_view_tCollections owning the keys.
keysustore_keys_view_tKeys to write.
valuesustore_strings_view_tValues to write, one per key; a tape ustore_blobs_read wrote is written back as it is.
durabilityustore_durability_tHow durable the writes are once the call finishes at the head; a transaction's commit decides its own.
watchustore_watch_tWhether the transaction records the written keys for conflicts.

Statuses

ustore_blobs_removeRemoves a batch of keys with their values.
blobs.h:153 ↗
void ustore_blobs_remove(ustore_blobs_remove_t *arguments);

ustore_status_invalid_argument_k means the collections or keys are missing, the transaction belongs to another engine or is staged, the durability or watch is unknown, a flushed remove names a transaction, ustore_watch_none_k comes without one, or a collection holds another modality.

Parameters

databaseustore_database_tAn open database.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
transactionustore_transaction_tTransaction the removals join, where NULL removes at the head.
countustore_size_tNumber of tasks, one per key.
collectionsustore_collections_view_tCollections owning the keys.
keysustore_keys_view_tKeys to remove, or the inclusive starts of ranges when end_keys is given.
end_keysustore_keys_view_tExclusive ends of the ranges to remove, or a zeroed view to remove keys alone.
durabilityustore_durability_tHow durable the removals are once the call finishes at the head; a transaction's commit decides its own.
watchustore_watch_tWhether the transaction records the removed keys for conflicts.

Statuses

ustore_blobs_readReads a batch of values.
blobs.h:227 ↗
void ustore_blobs_read(ustore_blobs_read_t *arguments);

Parameters

databaseustore_database_tAn open database.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
viewustore_view_tWhat the call sees, where NULL is the head.
countustore_size_tNumber of tasks, one per key.
collectionsustore_collections_view_tCollections owning the keys.
keysustore_keys_view_tKeys to read.
offsetsustore_lengths_view_tByte offset into each value the read starts at, or a zeroed view for whole values.
limitsustore_lengths_view_tBytes to read of each value from its offset, clipped, or a zeroed view for the rest.
presencesustore_octet_t const **Optional output for a bitset of count bits marking present keys, inside arena.
lengthsustore_length_t const **Optional output for the length of every value, for reads that skip the bytes, inside arena.
valuesustore_strings_view_t *Optional output for the values as a tape inside arena, where a missing key reads as an empty value.
arenaustore_arena_tRewound by this call, then holding every output it writes; NULL is refused.

Statuses

ustore_strings_writeWrites a batch of strings and the keys they map onto.
strings.h:86 ↗
void ustore_strings_write(ustore_strings_write_t *arguments);

Parameters

databaseustore_database_tAn open database.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
transactionustore_transaction_tTransaction the writes join, where NULL writes to the head.
countustore_size_tNumber of tasks, one per string.
collectionsustore_collections_view_tStrings collections owning the strings.
stringsustore_strings_view_tStrings to write, one per task.
valuesustore_keys_view_tThe key every string maps onto, one per string.
durabilityustore_durability_tHow durable the writes are once the call finishes at the head; a transaction's commit decides its own.
watchustore_watch_tWhether the transaction records the strings it reads and writes for conflicts.

Statuses

ustore_strings_removeRemoves a batch of strings with their values.
strings.h:149 ↗
void ustore_strings_remove(ustore_strings_remove_t *arguments);

ustore_status_invalid_argument_k means the collections or strings are missing, the strings come in an unknown layout, the transaction belongs to another engine or is staged, the durability or watch is unknown, a flushed remove names a transaction, or a collection holds another modality.

Parameters

databaseustore_database_tAn open database.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
transactionustore_transaction_tTransaction the removals join, where NULL removes at the head.
countustore_size_tNumber of tasks, one per string.
collectionsustore_collections_view_tStrings collections owning the strings.
stringsustore_strings_view_tStrings to remove, one per task, or inclusive range starts when end_strings is given.
end_stringsustore_strings_view_tExclusive ends of the ranges to remove, where an empty one bounds nothing, or a zeroed view to remove strings alone.
durabilityustore_durability_tHow durable the removals are once the call finishes at the head; a transaction's commit decides its own.
watchustore_watch_tWhether the transaction records the strings it reads and writes for conflicts.

Statuses

ustore_strings_readReads the keys a batch of strings map onto.
strings.h:207 ↗
void ustore_strings_read(ustore_strings_read_t *arguments);

Parameters

databaseustore_database_tAn open database.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
viewustore_view_tWhat the call sees, where NULL is the head.
countustore_size_tNumber of tasks, one per string.
collectionsustore_collections_view_tStrings collections owning the strings.
stringsustore_strings_view_tStrings to read, one per task.
presencesustore_octet_t const **Optional output for a bitset of count bits marking present strings, addressed as Apache Arrow does, inside arena.
valuesustore_key_t const **Optional output for the key of every string inside arena, in the order of the strings, where a missing string reads as the nil UUID.
arenaustore_arena_tRewound by this call, then holding every output it writes; NULL is refused.

Statuses

  • ustore_status_invalid_argument_k The arena, collections or strings are missing, the strings come in an unknown layout, the view was made by neither view constructor or belongs to another engine, a watching read goes through a staged transaction, or a collection holds another modality.
  • ustore_status_device_k The arena lives where another executor computes.
  • ustore_status_contended_k The arena or the transaction the view reads through serves another call, the executor's deferred table is full, or a callback blocks on the executor running it.
  • ustore_status_uninitialized_k The database is NULL.
  • ustore_status_not_found_k A collection does not exist or was dropped meanwhile.
ustore_strings_scanScans a batch of prefixes.
strings.h:307 ↗
void ustore_strings_scan(ustore_strings_scan_t *arguments);

Parameters

databaseustore_database_tAn open database.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
viewustore_view_tWhat the call sees, where NULL is the head.
countustore_size_tNumber of scans.
collectionsustore_collections_view_tStrings collections to scan.
prefixesustore_strings_view_tPrefixes every scan matches, where a zeroed view matches every string.
start_stringsustore_strings_view_tString every scan starts from, inclusive, which paginates a walk of one prefix; a zeroed view starts at the first string in the direction.
end_stringsustore_strings_view_tExclusive string every scan stops before, where empty or zeroed views bound nothing.
directionustore_scan_direction_tAscending, the default, or descending from the start string.
patternsustore_patterns_tOptional patterns every listed string matches, shared by every scan of the call; zeroed lists every string.
predicateustore_predicate_tOptional verdict on the keys the listed strings map onto, asked once per page as its cost says; host executors only.
offsetsustore_length_t const **Output for count plus one offsets into strings, the last one closing the final scan, inside arena.
stringsustore_strings_view_t *Output for the found strings as a tape inside arena, scan after scan.
scoresfloat const **Optional output for one BM25 score per listed string when patterns carry weights, inside arena; NULL skips them.
arenaustore_arena_tRewound by this call, then holding every output it writes; NULL is refused.
wantedustore_size_tMost strings every scan returns.
modeustore_scan_mode_tIn what order the strings come.
progressustore_progress_callback_tOptional callback reporting the scans finished.
progress_statevoid *Opaque pointer passed to progress.

Statuses

ustore_status_invalid_argument_k means the arena, collections or an output is missing, the prefixes, start or end strings come in an unknown layout, the direction, a pattern kind, a case sensitivity or a predicate cost is unknown, the mode is unknown under patterns or a predicate, the patterns do not compile, the view was made by neither view constructor or belongs to another engine, a watching scan goes through a staged transaction, or a collection holds a modality other than strings.

ustore_strings_cursor_initOpens a walk over the strings of one prefix, reading nothing until its first pull.
strings.h:372 ↗
void ustore_strings_cursor_init(
    ustore_strings_cursor_init_t *arguments);

Parameters

cursorustore_strings_cursor_t *Output for the new cursor, which the caller frees with ustore_strings_cursor_free; NULL on failure.
databaseustore_database_tAn open database.
executorustore_executor_tExecutor of the kind every pull runs on, which picks the side the walk computes on, where NULL walks on host executors; a pull on another kind of executor reports ustore_status_device_k.
outcomeustore_outcome_t *Output for the outcome, final on return.
viewustore_view_tWhat the walk sees, where NULL is the head, read pull by pull.
collectionustore_collection_id_tStrings collection to walk.
prefixustore_bytes_view_tPrefix every string the walk returns starts with, copied, where a zeroed view matches every string.
start_stringustore_bytes_view_tString the walk starts from, inclusive, copied, where a zeroed view starts at the first string of the prefix in the direction.
end_stringustore_bytes_view_tString the walk stops before, exclusive, copied, or a zeroed view for no bound.
directionustore_scan_direction_tAscending, the default, or descending from the start string.
patternsustore_patterns_tOptional patterns every string the walk hands out matches, compiled here once; zeroed walks every string.
predicateustore_predicate_tOptional verdict on the keys of the strings of every page, asked as its cost says; host executors only.

Statuses

ustore_strings_cursor_nextPulls the next page of a walk, and the values of its strings when asked.
strings.h:435 ↗
void ustore_strings_cursor_next(
    ustore_strings_cursor_next_t *arguments);

Parameters

databaseustore_database_tThe database the cursor was opened on.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
cursorustore_strings_cursor_tThe cursor to advance, which a pull that fails or is cancelled leaves where it was.
stringsustore_strings_view_t *Output for the strings of the page in the order of their text, as a tape inside arena.
valuesustore_key_t const **Optional output for the keys of the page's strings inside arena, where a string removed at the head between the listing and the read of one pull reads as the nil UUID; NULL skips them.
writtenustore_size_t *Output for the strings written, zero once the walk is exhausted; zeroed on failure.
arenaustore_arena_tRewound by this call, then holding every output it writes; NULL is refused.
wantedustore_size_tMost strings the page holds, where a page of fewer exhausts the walk.
progressustore_progress_callback_tOptional callback reporting the one pull, which may cancel it before the cursor advances.
progress_statevoid *Opaque pointer passed to progress.

Statuses

  • ustore_status_invalid_argument_k The cursor, the arena or an output is NULL, the cursor walks another database, a watching pull goes through a staged transaction, or the collection holds another modality.
  • ustore_status_device_k The arena lives where another executor computes, a cursor opened for a device is pulled on host threads, or a host cursor is pulled on a device.
  • ustore_status_contended_k The arena, the cursor or the transaction it reads through serves another call, the executor's deferred table is full, or a callback blocks on the executor running it.
  • ustore_status_uninitialized_k The database is NULL.
  • ustore_status_unsupported_k The pulling executor's device loaded no bundle of the library's kernels and the walk has patterns.
  • ustore_status_not_found_k The collection does not exist or was dropped meanwhile.
ustore_strings_cursor_freeWaits until every deferred call naming the cursor finishes, then frees it, giving back the holds it kept on its database and on the transaction or snapshot it reads through.
strings.h:442 ↗
void ustore_strings_cursor_free(ustore_strings_cursor_t cursor);

Parameters

cursorinoutThe cursor to be freed; NULL is safe.
ustore_strings_sampleSamples strings from a batch of collections.
strings.h:501 ↗
void ustore_strings_sample(ustore_strings_sample_t *arguments);

Parameters

databaseustore_database_tAn open database.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
viewustore_view_tWhat the call sees, where NULL is the head.
countustore_size_tNumber of samples.
collectionsustore_collections_view_tStrings collections to sample.
offsetsustore_length_t const **Output for count plus one offsets into strings, the last one closing the final sample, inside arena.
stringsustore_strings_view_t *Output for the sampled strings as a tape, sample after sample, inside arena.
arenaustore_arena_tRewound by this call, then holding every output it writes; NULL is refused.
wantedustore_size_tMost strings every sample draws.

Statuses

ustore_strings_measureEstimates the strings and the space inside a batch of string ranges.
strings.h:583 ↗
void ustore_strings_measure(ustore_strings_measure_t *arguments);

ustore_status_invalid_argument_k means the arena or the collections are missing, the strings come in an unknown layout, the direction is unknown, the view was made by neither view constructor or belongs to another engine, a watching measure goes through a staged transaction, or a collection holds another modality.

Parameters

databaseustore_database_tAn open database.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
viewustore_view_tWhat the call sees, where NULL is the head.
countustore_size_tNumber of measured ranges.
collectionsustore_collections_view_tStrings collections to measure.
start_stringsustore_strings_view_tString every range starts from, inclusive, where a zeroed view starts at the first string in the direction.
end_stringsustore_strings_view_tString every range stops before, exclusive, where an empty one bounds nothing, or a zeroed view for no bound.
directionustore_scan_direction_tAscending, the default, or descending from the start string.
min_cardinalitiesustore_size_t const **Optional output for the lower bound on the strings in every range, inside arena.
max_cardinalitiesustore_size_t const **Optional output for the upper bound on the strings in every range, inside arena.
min_value_bytesustore_size_t const **Optional output for the lower bound on the bytes of the strings and their keys in every range, inside arena.
max_value_bytesustore_size_t const **Optional output for the upper bound on the bytes of the strings and their keys in every range, inside arena.
min_space_usagesustore_size_t const **Optional output for the lower bound on the disk space every range takes, inside arena.
max_space_usagesustore_size_t const **Optional output for the upper bound on the disk space every range takes, inside arena.
arenaustore_arena_tRewound by this call, then holding every output it writes; NULL is refused.

Statuses

ustore_keys_view_from_arrowViews the 16-byte keys of one Arrow fixed_size_binary(16) column in place, tagged arrow.uuid or not.
arrow.h:231 ↗
ustore_status_t ustore_keys_view_from_arrow(
    struct ArrowSchema const *schema,
    struct ArrowArray const *array,
    ustore_keys_view_t *keys,
    ustore_size_t *count);

A null_count of -1 with a validity buffer may hold a null.

Parameters

schemainThe column's schema.
arrayinThe column, whose offset is applied; only its lengths and pointers are read, never its buffers, so the array of an ArrowDeviceArray converts as well.
keysoutThe keys, one every 16 bytes.
countoutThe number of keys.

Returns

ustore_status_invalid_argument_k for a missing or released input or one holding a null, ustore_status_unsupported_k for another format, a dictionary or a view, ustore_status_malformed_k for a negative length or offset or a missing buffer.

ustore_strings_view_from_arrowViews the byte strings of one Arrow column in place: binary or utf8 as a tape of 32-bit offsets, and large_binary or large_utf8 as one of 64-bit offsets, starting at the column's offset.
arrow.h:241 ↗
ustore_status_t ustore_strings_view_from_arrow(
    struct ArrowSchema const *schema,
    struct ArrowArray const *array,
    ustore_strings_view_t *strings,
    ustore_size_t *count);

Returns

The statuses of ustore_keys_view_from_arrow.

ustore_keys_view_to_arrowExports count keys a call wrote into arena as one Arrow fixed_size_binary(16) column tagged as arrow.uuid, in place.
arrow.h:267 ↗
ustore_status_t ustore_keys_view_to_arrow(
    ustore_keys_view_t keys,
    ustore_size_t count,
    ustore_arena_t arena,
    struct ArrowSchema *schema,
    struct ArrowDeviceArray *array);

The array reads arena's outputs until the arena's next call or free, like every other output view, and its release frees only the array's own nodes. Arenas on host pages export as ARROW_DEVICE_CPU, and those of a VRAM executor as ARROW_DEVICE_CUDA or ARROW_DEVICE_ROCM with the vendor's ordinal of their device and no sync_event, as the call that filled one waited for its device. The schema's name is empty, for the caller to point at its own.

Keys of another stride, like the triples of edges, are copied dense into arena first, under the claim a call takes, so no other call or carving export runs on arena meanwhile; a device arena's are copied on its device, through an executor the export spawns there.

Statuses

ustore_strings_view_to_arrowExports a tape of count strings a call wrote into arena as one Arrow column of layout, in place when the tape already has it.
arrow.h:285 ↗
ustore_status_t ustore_strings_view_to_arrow(
    ustore_strings_view_t strings,
    ustore_size_t count,
    ustore_strings_layout_t layout,
    ustore_arena_t arena,
    struct ArrowSchema *schema,
    struct ArrowDeviceArray *array);

A ustore_strings_u32_tape_k layout exports as binary and a ustore_strings_u64_tape_k one as large_binary, living until arena's next call or free as ustore_keys_view_to_arrow does. A tape of the other width has its offsets converted into arena first, under the claim, narrowed ones counted from the tape's first byte; a device arena's offsets cross to the host and back through an executor the export spawns on that device.

Returns

The statuses of ustore_keys_view_to_arrow, contended only while converting a tape of the other width, with an accessor or an unknown layout refused as an invalid argument, and ustore_status_capacity_exhausted_k for strings spanning more bytes than 32-bit offsets reach.