Reads and Writes | UStore for Mojo

§ Binary Values

A cursor walks one collection page by page, every pull resuming past the last key the pull before it handed out:

var cursor = db.on(executor).cursor(people, Key(0))  # walks on the kind of executor it was opened on
var page = cursor.next(executor, arena, 1000)        # keys and their values, zero once exhausted

db.changes_cursor(collection, since) opens a feed of the changes committed after a stamp, and db.ingest(collection) lands every appended row as one commit of its own.

API

Types

LengthsViewA strided run of lengths or positions, ustore_lengths_view_t.

comptime LengthsView[origin: ImmOrigin = ImmUntrackedOrigin]: AnyStruct[StridedView[UInt64, origin]] = StridedView[UInt64, origin]

StructValuesWhat a read wrote into the arena it borrows: an Apache Arrow binary array of one value per task, a missing key reading as an unset presence bit and an empty value.5

struct Values[origin: MutOrigin]

Parameters

`originMutOrigin`	comptime	The arena the values live in, which this result borrows.

Fields

`arena`	`Pointer[Arena, origin]`	The arena holding the values.
`generation`	`Int`	The arena's call the values came from.
`written`	`ValuesWritten`	The views the engine wrote.
`count`	`Int`	Tasks the read carried.

Values

`Written`	`ValuesWritten`

Conforms to

Answer, Deinitable, Movable, Sized

__init__

def __init__(out self, arena: Int, generation: Int, written: ValuesWritten, count: Int)

__len__The tasks the read carried.

def __len__(self) -> Int

presentWhether task index found its value.

def present(self, index: Int) -> Bool

lengthsThe length of every value.

def lengths(ref self) -> Span[UInt64, origin_of(self)]

valuesThe values, empty for a missing key or when the read skipped the bytes.

def values(self) -> Tape[origin]

StructMeasuresThe bounds a measure estimated into the arena it borrows, one per range.8

struct Measures[origin: MutOrigin]

Parameters

`originMutOrigin`	comptime	The arena the bounds live in, which this result borrows.

Fields

`arena`	`Pointer[Arena, origin]`	The arena holding the bounds.
`generation`	`Int`	The arena's call the bounds came from.
`written`	`MeasuresWritten`	The views the engine wrote.
`count`	`Int`	Ranges measured.

Values

`Written`	`MeasuresWritten`

Conforms to

Answer, Deinitable, Movable, Sized

__init__

def __init__(out self, arena: Int, generation: Int, written: MeasuresWritten, count: Int)

__len__The ranges measured.

def __len__(self) -> Int

min_cardinalitiesThe lower bound on the entries in every range.

def min_cardinalities(ref self) -> Span[UInt64, origin_of(self)]

max_cardinalitiesThe upper bound on the entries in every range.

def max_cardinalities(ref self) -> Span[UInt64, origin_of(self)]

min_value_bytesThe lower bound on the bytes of values in every range.

def min_value_bytes(ref self) -> Span[UInt64, origin_of(self)]

max_value_bytesThe upper bound on the bytes of values in every range.

def max_value_bytes(ref self) -> Span[UInt64, origin_of(self)]

min_space_usagesThe lower bound on the disk space every range takes.

def min_space_usages(ref self) -> Span[UInt64, origin_of(self)]

max_space_usagesThe upper bound on the disk space every range takes.

def max_space_usages(ref self) -> Span[UInt64, origin_of(self)]

StructPulledOne page of a cursor's walk, read in place from the arena the pull borrows.4

struct Pulled[origin: MutOrigin]

Parameters

`originMutOrigin`	comptime	The arena the page lives in, which this result borrows.

Fields

`arena`	`Pointer[Arena, origin]`	The arena holding the page.
`generation`	`Int`	The arena's call the page came from.
`written`	`PulledWritten`	The views the engine wrote.

Values

`Written`	`PulledWritten`

Conforms to

Answer, Deinitable, Movable, Sized

__init__

def __init__(out self, arena: Int, generation: Int, written: PulledWritten, count: Int)

__len__The keys pulled, zero once the walk is exhausted.

def __len__(self) -> Int

keysThe keys of the page in their order.

def keys(ref self) -> Span[Key, origin_of(self)]

valuesTheir values, empty when the pull skipped them.

def values(self) -> Tape[origin]

StructBlobsReadConfigThe ranges a read takes of each value, and whether it copies the bytes.1

struct BlobsReadConfig[offsets_origin: ImmOrigin = ImmUntrackedOrigin, limits_origin: ImmOrigin = ImmUntrackedOrigin]

Parameters

`offsets_originImmOrigin`	comptime	The borrowed start offsets.
`limits_originImmOrigin`	comptime	The borrowed byte limits.

Fields

`offsets`	`StridedView[UInt64, offsets_origin]`	Byte offset into each value the read starts at; zeroed reads whole values.
`limits`	`StridedView[UInt64, limits_origin]`	Bytes to read of each value from its offset; zeroed reads the rest.
`values`	`Bool`	Whether the bytes come too, or only presences and lengths.

Conforms to

Copyable, Deinitable, ImplicitlyCopyable, Movable

__init__Names only the knobs that differ from the defaults.

def __init__(
    out self,
    *,
    offsets: StridedView[UInt64, offsets_origin] = StridedView(),
    limits: StridedView[UInt64, limits_origin] = StridedView(),
    values: Bool = True
)

StructBlobsScanConfigThe bounds, order and filters of a scan over keys.1

struct BlobsScanConfig[end_origin: ImmOrigin = ImmUntrackedOrigin, patterns_origin: ImmOrigin = ImmUntrackedOrigin]

Parameters

`end_originImmOrigin`	comptime	The borrowed end keys.
`patterns_originImmOrigin`	comptime	The borrowed patterns.

Fields

`end_keys`	`StridedView[Key, end_origin]`	Key every scan stops before, exclusive; zeroed bounds nothing.
`direction`	`ScanDirection`	Ascending or descending from the start keys.
`mode`	`ScanMode`	In what order the keys come.
`patterns`	`Patterns[patterns_origin]`	Patterns the value of every listed key matches; empty lists every key.
`predicate`	`Predicate`	The caller's verdict on the listed keys.
`progress`	`Progress`	Reports the scans finished, and stops the call where it says to.
`scores`	`Bool`	Whether one score per listed key comes too, when the patterns carry weights.

Conforms to

Copyable, Deinitable, ImplicitlyCopyable, Movable

__init__Names only the knobs that differ from the defaults.

def __init__(
    out self,
    *,
    end_keys: StridedView[Key, end_origin] = StridedView(),
    direction: ScanDirection = ScanDirection.ASCENDING,
    mode: ScanMode = ScanMode.ORDERED,
    patterns: Patterns[patterns_origin] = Patterns(),
    predicate: Predicate = Predicate(),
    progress: Progress = Progress(),
    scores: Bool = False
)

StructStridedViewA borrowed strided run of fixed-width elements, one per task, where a zero stride repeats the first.6

struct StridedView[T: Copyable, origin: ImmOrigin = ImmUntrackedOrigin]

Parameters

`TCopyable`	comptime	The element, laid out as the ABI declares it.
`originImmOrigin`	comptime	The borrowed elements, which the view keeps alive for as long as it is used.

Fields

`data`	`Optional[Pointer[T, origin]]`	The first element; absent for a zeroed view.
`stride`	`Int`	Bytes between consecutive elements; zero repeats the first.
`count`	`Int`	Elements the view holds: one for a repeated element, zero for a zeroed view.

Conforms to

Copyable, Deinitable, ImplicitlyCopyable, Movable, Sized

__init__A zeroed view, which stands for the input's default.

def __init__(out self)

__len__The elements the view holds.

def __len__(self) -> Int

tasksThe tasks the view can serve: its elements, minus one for a repeated element, which fits any batch.

def tasks(self) -> Int

ofBorrows elements packed one after another.

def of[items_origin: ImmOrigin, //](items: Span[T, items_origin]) -> StridedView[T, items_origin]

repeatingBorrows one element, repeated for every task.

def repeating[item_origin: ImmOrigin, //](ref[origin] item: T) -> StridedView[T, item_origin]

*× 2The C layout, erased only inside a verb so the borrow lives through the call.

def arguments(self) -> StridedArguments

StructDurabilityHow durable the writes a call publishes are once it finishes, ustore_durability_t.

struct Durability

Fields

`code`	`c_int`	The raw `ustore_durability_t` value the ABI exchanges.

Values

`BUFFERED`	`Durability(Int32(0))`	Visible, and on disk as the engine's buffers fill.
`FLUSHED`	`Durability(Int32(1))`	On disk.

Conforms to

Copyable, Deinitable, Equatable, ImplicitlyCopyable, Movable, RegisterPassable, TrivialRegisterPassable

StructIngestOrderIn what order an ingest's rows arrive, ustore_ingest_order_t.

struct IngestOrder

Fields

`code`	`c_int`	The raw `ustore_ingest_order_t` value the ABI exchanges.

Values

`SORTED`	`IngestOrder(Int32(0))`	Keys strictly ascend across every append.
`UNSORTED`	`IngestOrder(Int32(1))`	Keys arrive in any order and may repeat, the last row of a key winning.

Conforms to

Copyable, Deinitable, Equatable, ImplicitlyCopyable, Movable, RegisterPassable, TrivialRegisterPassable

StructIngestAn ingest under way into one collection, landing every row appended as one commit of its own, freed when it goes out of scope. Its calls run on the calling thread, one at a time.4

struct Ingest

Fields

`library`	`Library`	The loaded engine.
`address`	`Address`	The `ustore_ingest_t`.

Conforms to

Deinitable, Movable

__init__Adopts an ingest the engine just opened.

def __init__(out self, *, library: Library, address: Int)

__deinit__Frees the ingest, dropping every row of an unfinished one. Calls ustore_ingest_free.

def __deinit__(deinit self)

appendHands the ingest one batch of rows, copied before the call returns. Calls ustore_ingest_append.

def append[keys_origin: ImmOrigin, values_origin: ImmOrigin, sources_origin: ImmOrigin, targets_origin: ImmOrigin, edges_origin: ImmOrigin, //](
    self,
    keys: StridedView[Key, keys_origin] = StridedView(),
    values: StringsView[values_origin] = StringsView(),
    sources: StridedView[Key, sources_origin] = StridedView(),
    targets: StridedView[Key, targets_origin] = StridedView(),
    edges: StridedView[Key, edges_origin] = StridedView()
)

Blobs, documents and pages take keys and values, strings take their strings as values and the keys they map onto as keys, and edges take sources, targets and optionally edges.

Raises

Error if a view the modality needs is missing, or a key breaks a sorted order.

finishLands every row appended as one commit, answering its stamp. Calls ustore_ingest_finish.

def finish(self, durability: Durability = Durability.BUFFERED) -> Stamp

Raises

Error if the collection was dropped meanwhile.

StructStringRunsThe strings a scan or a sample listed into the arena it borrows, run after run, one run per task.5

struct StringRuns[origin: MutOrigin]

Parameters

`originMutOrigin`	comptime	The arena the strings live in, which this result borrows.

Fields

`arena`	`Pointer[Arena, origin]`	The arena holding the strings.
`generation`	`Int`	The arena's call the strings came from.
`written`	`StringRunsWritten`	The views the engine wrote.
`count`	`Int`	Runs, one per task.

Values

`Written`	`StringRunsWritten`

Conforms to

Answer, Deinitable, Movable, Sized

__init__

def __init__(out self, arena: Int, generation: Int, written: StringRunsWritten, count: Int)

__len__The runs, one per task.

def __len__(self) -> Int

offsetsThe runs' boundaries in strings, one more of them than runs.

def offsets(ref self) -> Span[UInt64, origin_of(self)]

stringsEvery run's strings end to end; run i spans entries offsets()[i] up to offsets()[i + 1].

def strings(self) -> Tape[origin]

scoresOne score per listed string, empty unless weighted patterns asked for them.

def scores(ref self) -> Span[Float32, origin_of(self)]

StructStringsPulledOne page of a strings cursor's walk, read in place from the arena the pull borrows.4

struct StringsPulled[origin: MutOrigin]

Parameters

`originMutOrigin`	comptime	The arena the page lives in, which this result borrows.

Fields

`arena`	`Pointer[Arena, origin]`	The arena holding the page.
`generation`	`Int`	The arena's call the page came from.
`written`	`StringsPulledWritten`	The views the engine wrote.

Values

`Written`	`StringsPulledWritten`

Conforms to

Answer, Deinitable, Movable, Sized

__init__

def __init__(out self, arena: Int, generation: Int, written: StringsPulledWritten, count: Int)

__len__The strings pulled, zero once the walk is exhausted.

def __len__(self) -> Int

stringsThe strings of the page in the order of their text.

def strings(self) -> Tape[origin]

keysThe keys of the page's strings, empty when the pull skipped them.

def keys(ref self) -> Span[Key, origin_of(self)]

StructStringsScanConfigThe bounds, order and filters of a prefix scan.1

struct StringsScanConfig[start_origin: ImmOrigin = ImmUntrackedOrigin, end_origin: ImmOrigin = ImmUntrackedOrigin, patterns_origin: ImmOrigin = ImmUntrackedOrigin]

Parameters

`start_originImmOrigin`	comptime	The borrowed start strings.
`end_originImmOrigin`	comptime	The borrowed end strings.
`patterns_originImmOrigin`	comptime	The borrowed patterns.

Fields

`start_strings`	`StringsView[start_origin]`	String every scan starts from, inclusive; zeroed starts at the first string in the direction.
`end_strings`	`StringsView[end_origin]`	String every scan stops before, exclusive; empty or zeroed bounds nothing.
`direction`	`ScanDirection`	Ascending or descending from the start string.
`mode`	`ScanMode`	In what order the strings come.
`patterns`	`Patterns[patterns_origin]`	Patterns every listed string matches; empty lists every string.
`predicate`	`Predicate`	The caller's verdict on the keys the listed strings map onto.
`progress`	`Progress`	Reports the scans finished, and stops the call where it says to.
`scores`	`Bool`	Whether one score per listed string comes too, when the patterns carry weights.

Conforms to

Copyable, Deinitable, ImplicitlyCopyable, Movable

__init__Names only the knobs that differ from the defaults.

def __init__(
    out self,
    *,
    start_strings: StringsView[start_origin] = StringsView(),
    end_strings: StringsView[end_origin] = StringsView(),
    direction: ScanDirection = ScanDirection.ASCENDING,
    mode: ScanMode = ScanMode.ORDERED,
    patterns: Patterns[patterns_origin] = Patterns(),
    predicate: Predicate = Predicate(),
    progress: Progress = Progress(),
    scores: Bool = False
)

StructStringsCursorConfigThe bounds, order and filters of a walk over one prefix, every string copied when the walk opens.1

struct StringsCursorConfig[start_origin: ImmOrigin = ImmUntrackedOrigin, end_origin: ImmOrigin = ImmUntrackedOrigin, patterns_origin: ImmOrigin = ImmUntrackedOrigin]

Parameters

`start_originImmOrigin`	comptime	The borrowed start string.
`end_originImmOrigin`	comptime	The borrowed end string.
`patterns_originImmOrigin`	comptime	The borrowed patterns.

Fields

`start_string`	`BytesView[start_origin]`	String the walk starts from, inclusive; zeroed starts at the first string of the prefix.
`end_string`	`BytesView[end_origin]`	String the walk stops before, exclusive; zeroed bounds nothing.
`direction`	`ScanDirection`	Ascending or descending from the start string.
`patterns`	`Patterns[patterns_origin]`	Patterns every string the walk hands out matches; empty walks every string.
`predicate`	`Predicate`	The caller's verdict on the keys of the strings of every page.

Conforms to

Copyable, Deinitable, ImplicitlyCopyable, Movable

__init__Names only the knobs that differ from the defaults.

def __init__(
    out self,
    *,
    start_string: BytesView[start_origin] = BytesView(),
    end_string: BytesView[end_origin] = BytesView(),
    direction: ScanDirection = ScanDirection.ASCENDING,
    patterns: Patterns[patterns_origin] = Patterns(),
    predicate: Predicate = Predicate()
)

StructStringsCursorA walk over the strings of one prefix, every pull resuming past the last string the pull before it handed out, freed when it goes out of scope. It holds its database open, and the transaction or snapshot it reads through.3

struct StringsCursor

Fields

`library`	`Library`	The loaded engine.
`database`	`Address`	The `ustore_database_t` it walks.
`address`	`Address`	The `ustore_strings_cursor_t`.

Conforms to

Deinitable, Movable

__init__Adopts a cursor the engine just opened.

def __init__(out self, *, library: Library, database: Int, address: Int)

__deinit__Frees the cursor and the holds it kept. Calls ustore_strings_cursor_free.

def __deinit__(deinit self)

nextPulls the next page of up to wanted strings, with their keys when asked. Calls ustore_strings_cursor_next.

def next(
    self,
    executor: Executor,
    mut arena: Arena,
    wanted: Int,
    values: Bool = True,
    progress: Progress = Progress()
) -> StringsPulled[origin_of(arena)]

Raises

Error with CANCELLED when progress stops the pull, which leaves the cursor where it was.

StructStringsViewA borrowed ragged batch of byte strings laid out as Apache Arrow lays one out: the bytes end to end, and one more offset than strings around them.4

struct StringsView[origin: ImmOrigin = ImmUntrackedOrigin]

Parameters

`originImmOrigin`	comptime	The borrowed bytes and offsets, which the view keeps alive for as long as it is used.

Fields

`data`	`Optional[Pointer[UInt8, origin]]`	The first byte, which every offset counts from; absent for a zeroed view.
`offsets`	`Address`	The strings' boundaries, one more of them than strings, as `layout` sizes them.
`layout`	`c_int`	`0` for 32-bit offsets, `1` for 64-bit offsets.
`count`	`Int`	Strings in the batch, or zero for a zeroed view.

Conforms to

Copyable, Deinitable, ImplicitlyCopyable, Movable, Sized

__init__A zeroed view, which stands for the input's default.

def __init__(out self)

__len__The strings in the batch.

def __len__(self) -> Int

tasksThe tasks the batch can serve, one per string.

def tasks(self) -> Int

ofBorrows an Apache Arrow Binary tape: string i spans offsets[i] up to offsets[i + 1].

def of[bytes_origin: ImmOrigin, offsets_origin: ImmOrigin, //](
    bytes: Span[UInt8, bytes_origin],
    offsets: Span[UInt32, offsets_origin]
) -> StringsView[origin_of(bytes_origin, offsets_origin)]

def of[bytes_origin: ImmOrigin, offsets_origin: ImmOrigin, //](
    bytes: Span[UInt8, bytes_origin],
    offsets: Span[UInt64, offsets_origin]
) -> StringsView[origin_of(bytes_origin, offsets_origin)]

Borrows an Apache Arrow LargeBinary tape, the same with 64-bit offsets.

StructTapeA ragged run of byte strings a call wrote into the arena it borrows, read in place.2

struct Tape[origin: MutOrigin]

Parameters

`originMutOrigin`	comptime	The arena the strings live in, which this result borrows.

Fields

`arena`	`Pointer[Arena, origin]`	The arena holding the strings.
`generation`	`Int`	The arena's call these strings came from, so a read after a later call is refused.
`view`	`StringsViewArguments`	The bytes and offsets, as the engine wrote them.
`count`	`Int`	Strings in the tape.

Conforms to

Copyable, Deinitable, ImplicitlyCopyable, Movable, Sized

__getitem__One string's bytes, borrowed from the arena.

def __getitem__(ref self, index: Int) -> Span[UInt8, origin_of(self)]

Raises

Error with INVALID_ARGUMENT once a later call rewound the arena, or DEVICE over device memory.

__len__The strings in the tape.

def __len__(self) -> Int