Quickstart | UStore for Mojo

from ustore import Arena, CollectionsView, Database, Executor, Key, KeysView, Library, Modality, StringsView

def main() raises:
    var library = Library()
    var db = Database.init_ram(Executor(library))
    var people = db.collection_create("people", Modality.BLOBS)

    var keys: List[Key] = [Key(1), Key(2)]
    var bytes = "AdaGrace".as_bytes()  # two values end to end
    var offsets: List[UInt32] = [0, 3, 8]  # one more offset than values
    db.write(CollectionsView.repeating(people), KeysView.of(keys), StringsView.of(bytes, offsets))

    var arena = Arena(library)
    var values = db.read(CollectionsView.repeating(people), KeysView.of(keys), arena)
    print(values.present(1), len(values.values()[1]))

A Database owns its engine; an Executor owns the threads or rings a call runs on; an Arena owns the outputs a call writes. Library() opens the engine once, and every handle created through it keeps it mapped.

API

StructEngineThe engine serving a database, ustore_engine_t.

struct Engine

Fields

`code`	`c_int`	The raw `ustore_engine_t` value the ABI exchanges.

Values

`RAM`	`Engine(Int32(0))`	The in-memory engine.
`ROCKSDB`	`Engine(Int32(1))`	The RocksDB engine.
`LEVELDB`	`Engine(Int32(2))`	The LevelDB engine.
`NVME`	`Engine(Int32(3))`	The NVMe engine.

Conforms to

Copyable, Deinitable, Equatable, ImplicitlyCopyable, Movable, RegisterPassable, TrivialRegisterPassable

StructDataDirectoryA directory the data files may occupy, with the bytes they may take there, ustore_data_directory_t.

struct DataDirectory[origin: ImmOrigin = ImmUntrackedOrigin]

Parameters

`originImmOrigin`	comptime	The borrowed path, kept alive for as long as the entry is used.

Fields

`path`	`BytesView[origin]`	The directory, which must exist; on NVMe also a raw device from a base.
`max_bytes`	`c_size_t`	Bytes the files may take there; zero is unlimited on RocksDB and the whole medium on NVMe.

Conforms to

Copyable, Deinitable, ImplicitlyCopyable, Movable

StructDatabaseNvmeOptionsThe sizes an NVMe database opens with, where every zero keeps the engine's default. Binding-only.1

struct DatabaseNvmeOptions

Fields

`memory_limit`	`c_size_t`	Bytes of host memory the engine may pin and cache.
`transaction_max_bytes`	`c_size_t`	Bytes a transaction may buffer before it must commit.
`transaction_max_deltas`	`c_size_t`	Deltas a transaction may buffer before it must commit.
`collection_concurrency_limit`	`c_size_t`	Operations that may run on one collection at once.
`collection_cache_limit`	`c_size_t`	Bytes of values each collection may cache.
`collection_memory_runs_max_bytes`	`c_size_t`	Bytes of memory runs a collection holds before level 0 flushes.
`collection_memory_runs_max_deltas`	`c_size_t`	Deltas of memory runs a collection holds before level 0 flushes.
`collection_level_enlarge_factor`	`c_size_t`	How many times larger each level is than the one above it.
`collection_value_size`	`c_size_t`	Bytes of every value of a collection whose values share one size, or zero.
`collection_value_max_size`	`c_size_t`	Bytes of the largest value of a collection whose values vary in size.
`cpu_class`	`CPUClass`	The class the database confines its waits, atomics and string kernels to; `UNKNOWN` derives.

Conforms to

Copyable, Defaultable, Deinitable, ImplicitlyCopyable, Movable

__init__Every engine default.

def __init__(out self)

StructDatabaseAn open database, closed when it goes out of scope while its data persists on disk.13

struct Database

Verbs called on the database run on the calling thread and address its head; db.on(executor) runs the same verbs blocking on an executor, and db.deferred(executor) submits them there. A transaction, snapshot or cursor holds the database open until it is freed itself.

Fields

`library`	`Library`	The loaded engine, held so its symbol table outlives every call made through the handle.
`address`	`Address`	The opaque `ustore_database_t`.

Conforms to

BlobsVerbs, DatabaseVerbs, Deinitable, DocsVerbs, GraphVerbs, Movable, PagesVerbs, StringsVerbs

__init__Adopts a handle the engine just opened.

def __init__(out self, *, library: Library, address: Int)

__deinit__Closes the database once nothing holds it open. Calls ustore_database_free.

def __deinit__(deinit self)

The loader reference is surrendered only after the call, so Mojo cannot dlclose the engine between the load of the entry point and the branch to it.

init_ramStarts an empty database of the in-memory engine, which persists nowhere. Calls ustore_database_init_ram.

def init_ram(
    executor: Executor,
    isolation: Isolation = Isolation.STRICT_SERIALIZABLE,
    cpu_class: CPUClass = CPUClass.UNKNOWN
) -> Self

Parameters

`executorExecutor`		The executor the open runs on; it also lends its engine.
`isolationIsolation`		How transactions see each other.
`cpu_classCPUClass`		The class the database confines its kernels to; `UNKNOWN` derives.

Raises

Error if the class is not one the CPU runs.

open_ramServes a directory through the in-memory engine, loading its image and rewriting it on flushed commits and on close. Calls ustore_database_open_ram.

def open_ram[directory_origin: ImmOrigin, //](
    executor: Executor,
    directory: StringSpan[directory_origin],
    isolation: Isolation = Isolation.STRICT_SERIALIZABLE,
    cpu_class: CPUClass = CPUClass.UNKNOWN
) -> Self

Raises

Error with NOT_FOUND if the directory does not exist, or MALFORMED for a corrupt image.

open_rocksdbServes a directory through the RocksDB engine, creating the database when it holds none. Calls ustore_database_open_rocksdb.

def open_rocksdb[directory_origin: ImmOrigin, paths_origin: ImmOrigin, directories_origin: ImmOrigin, //](
    executor: Executor,
    directory: StringSpan[directory_origin],
    data_directories: Span[DataDirectory[paths_origin], directories_origin] = Span()
) -> Self

Raises

Error with UNSUPPORTED in a build without the engine.

view_rocksdbServes a directory through the RocksDB engine, read-only. Calls ustore_database_view_rocksdb.

def view_rocksdb[directory_origin: ImmOrigin, paths_origin: ImmOrigin, directories_origin: ImmOrigin, //](
    executor: Executor,
    directory: StringSpan[directory_origin],
    data_directories: Span[DataDirectory[paths_origin], directories_origin] = Span()
) -> Self

Raises

Error with UNSUPPORTED in a build without the engine.

open_leveldbServes a directory through the LevelDB engine, creating the database when it holds none. Calls ustore_database_open_leveldb.

def open_leveldb[directory_origin: ImmOrigin, //](
    executor: Executor,
    directory: StringSpan[directory_origin]
) -> Self

Raises

Error with UNSUPPORTED in a build without the engine.

open_nvmeServes the database a manifest names through the NVMe engine, creating it when the manifest is absent. Calls ustore_database_open_nvme.

def open_nvme[directory_origin: ImmOrigin, paths_origin: ImmOrigin, directories_origin: ImmOrigin, //](
    executor: Executor,
    directory: StringSpan[directory_origin],
    options: DatabaseNvmeOptions = DatabaseNvmeOptions(),
    data_directories: Span[DataDirectory[paths_origin], directories_origin] = Span(),
    compactions: Int = Int(0)
) -> Self

Parameters

`executorExecutor`		The executor the open runs on, whose transport the database reads once.
`directoryStringSpan[directory_origin]`		The manifest, or the raw address of the database's first slot.
`optionsDatabaseNvmeOptions`		The sizes the database opens with.
`data_directoriesSpan[DataDirectory[paths_origin], directories_origin]`		Further directories every run may land on.
`compactionsInt`		The address of a VRAM executor every merge runs on, read once; null merges on host threads.

Raises

Error with UNSUPPORTED in a build without the engine.

onThe same verbs, blocking on executor after every call submitted there before them. Binding-only.

def on(ref self, ref executor: Executor) -> On[origin_of(executor, self)]

deferredThe same verbs, each submitted to executor and answering a Deferred at once. Binding-only.

def deferred(ref self, ref executor: Executor) -> DeferredOn[origin_of(executor, self)]

snapshotPins a point-in-time state of the database until the snapshot is freed. Calls ustore_snapshot_init.

def snapshot(self) -> Snapshot

Raises

Error with UNSUPPORTED on an engine without snapshots.

transactionBegins a transaction. Calls ustore_transaction_init.

def transaction(self) -> Transaction

Raises

Error with UNSUPPORTED on an engine without transactions.

ingestOpens an ingest into one collection. Calls ustore_ingest_init.

def ingest(self, collection: CollectionID, order: IngestOrder = IngestOrder.SORTED) -> Ingest

Raises

Error with UNSUPPORTED where the engine ingests nothing into that collection's modality.

StructBytesViewA borrowed run of bytes and its length, never null-terminated: every path, name and filter.3

struct BytesView[origin: ImmOrigin = ImmUntrackedOrigin]

Parameters

`originImmOrigin`	comptime	The borrowed storage, which the view keeps alive for as long as it is used.

Fields

`data`	`Optional[Pointer[UInt8, origin]]`	The first byte; absent for a zeroed view.
`length`	`Int`	How many bytes the run covers, since nothing terminates it.

Conforms to

Copyable, Deinitable, ImplicitlyCopyable, Movable

__init__A zeroed view, which stands for the input's default.

def __init__(out self)

ofBorrows a string's bytes, with no terminator manufactured.

def of[text_origin: ImmOrigin, //](text: StringSpan[text_origin]) -> BytesView[text_origin]

def of[bytes_origin: ImmOrigin, //](bytes: Span[UInt8, bytes_origin]) -> BytesView[bytes_origin]

Borrows a run of raw bytes.

*× 2The C layout, erased only inside a verb so the borrow lives through the call.

def arguments(self) -> BytesViewArguments

StructRamOptionsThe host memory domains a RAM executor serves, one NUMA node each.2

struct RamOptions[origin: ImmOrigin = ImmUntrackedOrigin]

Parameters

`originImmOrigin`	comptime	The borrowed node ID array, kept alive for as long as the options are used.

Fields

`memory_domains`	`Optional[Pointer[Int32, origin]]`	NUMA node IDs; absent derives every node the process may allocate on.
`memory_domains_count`	`Int`	Entries in `memory_domains`; zero derives.

Conforms to

Copyable, Deinitable, ImplicitlyCopyable, Movable

__init__Derives every memory domain the process may allocate on.

def __init__(out self)

onNames the NUMA nodes explicitly, borrowing the caller's array for the call.

def on[domains_origin: ImmOrigin, //](
    memory_domains: Span[Int32, domains_origin]
) -> RamOptions[domains_origin]

StructNvmeOptionsHow the threads of an NVMe executor move bytes, ustore_nvme_options_t; every zero keeps the default.1

struct NvmeOptions

Fields

`transport`	`Transport`	How bytes move between the files and memory.
`shared_io_channels`	`c_size_t`	I/O channels the threads outside the executor share.
`queue_depth`	`c_size_t`	Submissions a channel holds at once.
`request_buffer_bytes`	`c_size_t`	Bytes of each request buffer of a channel.
`request_buffers_count`	`c_size_t`	Request buffers of each channel.
`stream_buffer_bytes`	`c_size_t`	Bytes of each streaming buffer of a channel.
`stream_buffers_count`	`c_size_t`	Streaming buffers of each channel.

Conforms to

Copyable, Defaultable, Deinitable, ImplicitlyCopyable, Movable

__init__Every engine default.

def __init__(out self)