C · Platform Notes

Platform Notes

#include <ustore/pages.h> 1.0.0-rc.1 · d6d40961

§ Strings

A strings collection, created with ustore_modality_strings_k, keys its values by variable-length strings, kept in the order of their bytes. Its strings and their values both arrive as ustore_strings_view_t, so ustore_strings_write, _remove, _read, _scan and _cursor_* mirror the blob calls with strings in place of keys. A scan lists up to wanted strings under every prefix as one tape, split by task through count + 1 offsets, and resumes from start_strings, while a strings cursor walks one prefix as a blobs cursor walks keys.

§ Pages

A pages collection stores ordered documents, edited by one verb per kind of edit, each taking only its own fields: ustore_pages_import, _insert, _remove, _move, _splice, _set_attributes and _remove_attributes. ustore_pages_read reads documents back into an arena, and ustore_pages_scan lists the keys of the pages. A page stores only its current shape, so no call lists the edits that produced it. Every engine serves them, importing and rendering XML and plain text, while HTML, Markdown and the JSON tree answer ustore_status_unsupported_k. An edit watches the page it rewrites, except on a pages collection of the NVMe engine, which keeps the edits themselves, so two transactions editing one page both commit. An edit of a missing page answers ustore_status_not_found_k, an edit of a missing element inside a page does nothing, and a Nil element removes the whole page.

§ Design

  • This is the C ABI itself. Every name is the one c/ustore/ declares: a verb ustore_<noun>_<verb> takes a ustore_<noun>_<verb>_t, and closed sets are C enums.
  • The completion chooses, the verb stays. A NULL executor runs a call on the calling thread, an executor with the default completion blocks on it, and ustore_completion_deferred_k submits it, all with the same argument struct.
  • Deferred calls report through the same outcome. ustore_outcome_status polls and ustore_outcome_wait blocks, both with the acquire ordering the final store pairs with.
  • No life of its own. Threads live only in executors, each serving one call at a time, and every *_free waits for the calls naming its handle first.
  • Three shapes in, three out. An input is one run of bytes, a strided run of fixed-width elements, or a ragged batch of strings in an Arrow tape or behind an accessor, and an output is a packed array, a tape of bytes, or packed keys split by offsets. So Arrow arrays pass as they are, and a runtime’s own objects pass through an accessor, without copies on either side.
  • Arenas are lent and reused, never shared. Every output whose length scales with the call lands in an arena the caller passes, and each call rewinds it, so the previous call’s views go stale the moment the next one starts.
  • Nothing costs anything until it is used. A blocking call runs on its caller’s thread, and an arena allocates only when a call writes into it.
  • One shape for every call. Each verb takes one struct set by designator, so a call names only what it changes and the struct grows without breaking source.
  • Failures are statuses. Every failure is a status in the call’s outcome beside a static message, ustore_status_name and ustore_status_from_name spell statuses both ways, and an engine without a call answers ustore_status_unsupported_k rather than missing from the build.
  • Closed sets spell themselves. Every enum a binding names has its _name and _from_name functions, so a binding builds its own enums from them and keeps no table to drift.
  • Walks hold, scans paginate. A cursor holds its database and its view and resumes past its last key, while a scan holds nothing and starts wherever its caller says, so neither keeps an iterator open over concurrent state.
  • Only progress stops a call. A callback returning ustore_cancel_k is the one way to stop a running call, so every other call runs to a final status, and a cancelled one leaves its outputs NULL and its cursor where it was.

API

Types

ustore_page_encoding_tEncoding a page or a fragment is imported from or rendered into.
pages.h:40 ↗
enum ustore_page_encoding_t

Values

ustore_page_encoding_html_k0HTML, to be parsed into elements and rendered back as markup, not yet imported or rendered, answering ustore_status_unsupported_k.
ustore_page_encoding_xml_k1XML, with every element and attribute kept as written.
ustore_page_encoding_markdown_k2Markdown, to be parsed into headings, paragraphs, lists, tables and code, not yet imported or rendered, answering ustore_status_unsupported_k.
ustore_page_encoding_text_k3The text of every run in document order, with structure dropped.
ustore_page_encoding_json_k4The element tree as JSON of key, tag, attributes, text and children, not yet imported or rendered, answering ustore_status_unsupported_k.

Spellings

char const * ustore_page_encoding_name(ustore_page_encoding_t encoding);
ustore_page_encoding_t ustore_page_encoding_from_name(
    ustore_bytes_view_t name);

Functions

ustore_pages_importImports whole pages, replacing what they held and minting every element.
pages.h:143 ↗
void ustore_pages_import(ustore_pages_import_t *arguments);

ustore_status_invalid_argument_k means the collections, keys or values are missing, the values come in an unknown layout, the encoding is unknown, the transaction belongs to another engine or is staged, the durability or watch is unknown, a flushed import names a transaction, ustore_watch_none_k comes without one, one call imports past 4 Gi tasks, a markup or a page is past 4 GiB, or a collection holds another modality.

Parameters

databaseustore_database_tAn open database.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
transactionustore_transaction_tTransaction the edits join, where NULL writes to the head.
countustore_size_tNumber of pages.
collectionsustore_collections_view_tCollections of the pages.
keysustore_keys_view_tKeys of the pages.
valuesustore_strings_view_tMarkup of the pages, one per task.
encodingustore_page_encoding_tEncoding of the markup.
durabilityustore_durability_tHow durable the edits are once the call finishes at the head; a transaction's commit decides its own.
watchustore_watch_tWhether the transaction records the pages it reads and writes for conflicts.
progressustore_progress_callback_tOptional callback reporting the bytes of values parsed.
progress_statevoid *Opaque pointer passed to progress.

Statuses

ustore_pages_insertInserts parsed fragments as elements under parents after anchors, or at the front on a Nil anchor.
pages.h:228 ↗
void ustore_pages_insert(ustore_pages_insert_t *arguments);

Parameters

databaseustore_database_tAn open database.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
transactionustore_transaction_tTransaction the edits join, where NULL writes to the head.
countustore_size_tNumber of insertions.
collectionsustore_collections_view_tCollections of the pages.
keysustore_keys_view_tKeys of the pages.
elementsustore_keys_view_tKeys to give the inserted elements, where Nil or NULL data mints one.
parentsustore_keys_view_tParents the elements land under, where Nil stands for the root.
anchorsustore_keys_view_tSiblings after which the elements land, where Nil stands for the front.
valuesustore_strings_view_tFragments to parse into elements, one per task.
encodingustore_page_encoding_tEncoding of the fragments.
effective_keysustore_key_t const **Optional output for the key every inserted element ended up with, minted or given, inside arena.
arenaustore_arena_tRewound by this call, then holding every output it writes; NULL is refused.
durabilityustore_durability_tHow durable the edits are once the call finishes at the head; a transaction's commit decides its own.
watchustore_watch_tWhether the transaction records the pages it reads and writes for conflicts.

Statuses

  • ustore_status_invalid_argument_k The arena, collections, keys or values are missing, the values come in an unknown layout, the encoding is unknown, the transaction belongs to another engine or is staged, the durability or watch is unknown, a flushed insert names a transaction, ustore_watch_none_k comes without one, one call edits past 4 Gi tasks, a fragment or a page is past 4 GiB, or a collection holds another modality.
  • ustore_status_device_k The arena lives where another executor computes.
  • ustore_status_contended_k The arena or the transaction serves another call, the executor's deferred table is full, a callback blocks on the executor running it, or an NVMe value keeps changing size across reads.
  • ustore_status_uninitialized_k The database is NULL or an NVMe transfer was cancelled.
  • ustore_status_read_only_k The database was opened read-only.
  • ustore_status_unsupported_k The executor computes on a device, the encoding is HTML, Markdown or JSON, or a DOCTYPE carries an internal subset.
  • ustore_status_not_found_k A collection or an edited page does not exist, or a collection was dropped meanwhile.
  • ustore_status_malformed_k A fragment does not parse, a stored page does not decode, or a page outgrows what its NVMe collection admits.
  • ustore_status_write_conflict_k Concurrent commits kept conflicting, or a staged RocksDB transaction holds a page.
  • ustore_status_capacity_exhausted_k The pages outgrow the NVMe transaction's buffer, or every shared NVMe channel is taken.
ustore_pages_removeRemoves elements with their subtrees.
pages.h:301 ↗
void ustore_pages_remove(ustore_pages_remove_t *arguments);

ustore_status_invalid_argument_k means the collections, keys or elements are missing, the transaction belongs to another engine or is staged, the durability or watch is unknown, a flushed edit names a transaction, ustore_watch_none_k comes without one, one call edits past 4 Gi tasks, a page is past 4 GiB, or a collection holds a modality other than pages.

Parameters

databaseustore_database_tAn open database.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
transactionustore_transaction_tTransaction the edits join, where NULL writes to the head.
countustore_size_tNumber of removals.
collectionsustore_collections_view_tCollections of the pages.
keysustore_keys_view_tKeys of the pages, or the inclusive starts of ranges when end_keys is given.
end_keysustore_keys_view_tExclusive ends of the ranges of whole pages to remove, which leave elements zeroed, or a zeroed view to remove elements alone.
elementsustore_keys_view_tElements to remove with their subtrees.
durabilityustore_durability_tHow durable the edits are once the call finishes at the head; a transaction's commit decides its own.
watchustore_watch_tWhether the transaction records the pages it reads and writes for conflicts.

Statuses

ustore_pages_moveMoves elements under parents after anchors, keeping their keys.
pages.h:375 ↗
void ustore_pages_move(ustore_pages_move_t *arguments);

ustore_status_invalid_argument_k means the collections, keys or elements are missing, the transaction belongs to another engine or is staged, the durability or watch is unknown, a flushed edit names a transaction, ustore_watch_none_k comes without one, one call edits past 4 Gi tasks, a page is past 4 GiB, or a collection holds a modality other than pages.

Parameters

databaseustore_database_tAn open database.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
transactionustore_transaction_tTransaction the edits join, where NULL writes to the head.
countustore_size_tNumber of moves.
collectionsustore_collections_view_tCollections of the pages.
keysustore_keys_view_tKeys of the pages.
elementsustore_keys_view_tElements to move with their subtrees.
parentsustore_keys_view_tParents the elements land under, where Nil stands for the root.
anchorsustore_keys_view_tSiblings after which the elements land, where Nil stands for the front.
durabilityustore_durability_tHow durable the edits are once the call finishes at the head; a transaction's commit decides its own.
watchustore_watch_tWhether the transaction records the pages it reads and writes for conflicts.

Statuses

ustore_pages_spliceReplaces runs of text inside elements with new bytes.
pages.h:451 ↗
void ustore_pages_splice(ustore_pages_splice_t *arguments);

ustore_status_invalid_argument_k means the collections, keys or elements are missing, the values come in an unknown layout, the transaction belongs to another engine or is staged, the durability or watch is unknown, a flushed edit names a transaction, ustore_watch_none_k comes without one, one call edits past 4 Gi tasks, a page is past 4 GiB, or a collection holds another modality.

Parameters

databaseustore_database_tAn open database.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
transactionustore_transaction_tTransaction the edits join, where NULL writes to the head.
countustore_size_tNumber of splices.
collectionsustore_collections_view_tCollections of the pages.
keysustore_keys_view_tKeys of the pages.
elementsustore_keys_view_tElements whose text is spliced.
text_startsustore_lengths_view_tBytes into the text of every element where its splice starts.
text_lengthsustore_lengths_view_tBytes of text every splice replaces, where zero only inserts.
valuesustore_strings_view_tNew bytes of every splice, where an empty run only removes.
durabilityustore_durability_tHow durable the edits are once the call finishes at the head; a transaction's commit decides its own.
watchustore_watch_tWhether the transaction records the pages it reads and writes for conflicts.

Statuses

ustore_pages_set_attributesSets attributes of elements, adding the ones they lack.
pages.h:522 ↗
void ustore_pages_set_attributes(
    ustore_pages_set_attributes_t *arguments);

Parameters

databaseustore_database_tAn open database.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
transactionustore_transaction_tTransaction the edits join, where NULL writes to the head.
countustore_size_tNumber of attributes to set.
collectionsustore_collections_view_tCollections of the pages.
keysustore_keys_view_tKeys of the pages.
elementsustore_keys_view_tElements whose attributes are set.
fieldsustore_strings_view_tNames of the attributes to set, one per task.
valuesustore_strings_view_tValues of the attributes, one per task.
durabilityustore_durability_tHow durable the edits are once the call finishes at the head; a transaction's commit decides its own.
watchustore_watch_tWhether the transaction records the pages it reads and writes for conflicts.

Statuses

ustore_pages_remove_attributesRemoves attributes of elements.
pages.h:590 ↗
void ustore_pages_remove_attributes(
    ustore_pages_remove_attributes_t *arguments);

Parameters

databaseustore_database_tAn open database.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
transactionustore_transaction_tTransaction the edits join, where NULL writes to the head.
countustore_size_tNumber of attributes to remove.
collectionsustore_collections_view_tCollections of the pages.
keysustore_keys_view_tKeys of the pages.
elementsustore_keys_view_tElements whose attributes are removed.
fieldsustore_strings_view_tNames of the attributes to remove, one per task.
durabilityustore_durability_tHow durable the edits are once the call finishes at the head; a transaction's commit decides its own.
watchustore_watch_tWhether the transaction records the pages it reads and writes for conflicts.

Statuses

ustore_pages_readRenders pages or subtrees in one encoding.
pages.h:665 ↗
void ustore_pages_read(ustore_pages_read_t *arguments);

Parameters

databaseustore_database_tAn open database.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
viewustore_view_tWhat the call sees, where NULL is the head.
countustore_size_tNumber of pages or subtrees to render.
collectionsustore_collections_view_tCollections of the pages.
keysustore_keys_view_tKeys of the pages.
elementsustore_keys_view_tElements to render with their subtrees, where NULL data renders whole pages.
presencesustore_octet_t const **Optional output for a bitset of count bits marking found pages or elements, inside arena.
lengthsustore_length_t const **Optional output for the length of every rendering, for reads that skip the bytes, inside arena.
valuesustore_strings_view_t *Optional output for the renderings as a tape inside arena.
arenaustore_arena_tRewound by this call, then holding every output it writes; NULL is refused.
outputs_encodingustore_page_encoding_tEncoding the renderings are converted into.

Statuses

ustore_status_invalid_argument_k means the arena, collections or keys are missing, the outputs_encoding is unknown, the view was made by neither view constructor or belongs to another engine, a watching read goes through a staged transaction, or a collection holds a modality other than pages.

ustore_pages_scanScans a batch of page key ranges.
pages.h:758 ↗
void ustore_pages_scan(ustore_pages_scan_t *arguments);

Parameters

databaseustore_database_tAn open database.
executorustore_executor_tExecutor that runs the call, where NULL runs it on the calling thread.
outcomeustore_outcome_t *Output for the outcome, final once its status leaves ustore_status_deferred_k.
completionustore_completion_tWhether the call returns final, the default, or once submitted.
viewustore_view_tWhat the call sees, where NULL is the head.
countustore_size_tNumber of scans.
collectionsustore_collections_view_tCollections of the pages.
start_keysustore_keys_view_tPage key every scan starts from, inclusive, where NULL data starts at the first page in the direction.
end_keysustore_keys_view_tKey every scan stops before, exclusive, or a zeroed view for no bound.
directionustore_scan_direction_tAscending, the default, or descending from the start key.
filterustore_bytes_view_tOptional path in StringZilla's XPath subset that every listed page holds a node of, shared by every scan; a zeroed view lists every page.
predicateustore_predicate_tOptional verdict on the listed keys, asked on the host once per page as its cost says.
offsetsustore_length_t const **Output for count plus one offsets into keys, the last one closing the final scan, inside arena.
keysustore_key_t const **Output for the found page keys, scan after scan, inside arena.
arenaustore_arena_tRewound by this call, then holding every output it writes; NULL is refused.
wantedustore_size_tMost page keys every scan returns.
modeustore_scan_mode_tIn what order the page keys come.
progressustore_progress_callback_tOptional callback reporting the scans finished.
progress_statevoid *Opaque pointer passed to progress.

Statuses

ustore_status_invalid_argument_k means the arena, collections or an output is missing, the mode, direction or a predicate cost is unknown, the filter does not compile, the view was made by neither view constructor or belongs to another engine, a watching scan goes through a staged transaction, or a collection holds a modality other than pages.