DenseSave[receiver, path] writes a checkpoint, and the path carries the SafeGraphs extension that picks the layout: .safegraphs for one monolith, .safegraphs.json for a manifest beside its shards. DenseMetadataRead[path] answers the frozen geometry, the metric, quantization, dimensions and connectivity, without loading a shard.
In-memory buffers hold single monoliths only, as UnsignedInteger8 numeric arrays, and the loaders take a ByteArray too:
monolith = DenseSaveBuffer[index, arena];
restored = DenseLoadRamBuffer[monolith];
DenseMetadataReadBuffer[monolith]DenseLoadVramBuffer takes the same options as DenseLoadVram.
DenseLoadRamDenseLoadRam[path] copies a checkpoint, a manifest or a single monolith, into a new, mutable index in host memory.DenseLoadRam[path, OptionsPattern[]]"Executor" runs the call, None meaning the calling thread; "MemoryDomains" lists the NUMA nodes to place shards on, {} deriving every node the process may allocate on; and "ValuesPerKey" bounds the entries sharing one key, which no checkpoint records, 0 leaving it unbounded.
"Executor" | Default: None | |
|---|---|---|
"MemoryDomains" | Default: {} | |
"ValuesPerKey" | Default: 0 |
DenseLoadVramDenseLoadVram[path] copies a checkpoint, a manifest or a single monolith, into a new, mutable index in device memory.DenseLoadVram[path, OptionsPattern[]]"Executor" runs the call, None meaning the calling thread; "Devices" lists the device ordinals to place shards on, {} picking device zero; and "ValuesPerKey" bounds the entries sharing one key, which no checkpoint records, 0 leaving it unbounded.
"Executor" | Default: None | |
|---|---|---|
"Devices" | Default: {} | |
"ValuesPerKey" | Default: 0 |
DenseLoadRamBufferDenseLoadRamBuffer[monolith] copies a checkpoint held in a ByteArray or a byte NumericArray into a new, mutable index in host memory, taking the options of DenseLoadRam.DenseLoadRamBuffer[monolith, OptionsPattern[]]"Executor" | Default: None | |
|---|---|---|
"MemoryDomains" | Default: {} | |
"ValuesPerKey" | Default: 0 |
DenseLoadVramBufferDenseLoadVramBuffer[monolith] copies a checkpoint held in a ByteArray or a byte NumericArray into a new, mutable index in device memory, taking the options of DenseLoadVram.DenseLoadVramBuffer[monolith, OptionsPattern[]]"Executor" | Default: None | |
|---|---|---|
"Devices" | Default: {} | |
"ValuesPerKey" | Default: 0 |
DenseViewRamDenseViewRam[path] serves a checkpoint in place through a memory mapping, as a read-only index.DenseViewRam[path]DenseViewNvmeDenseViewNvme[path] serves a checkpoint in place from NVMe drives as a read-only index, which every later call reads through its executor.DenseViewNvme[path, OptionsPattern[]]"Executor" runs the call, None meaning the calling thread, and "Paths" lists the devices or directories to serve shards from, {} leaving every shard where its manifest points.
"Executor" | Default: None | |
|---|---|---|
"Paths" | Default: {} |
DenseMetadataReadDenseMetadataRead[path] reads a checkpoint's geometry and metric without loading it.DenseMetadataRead[path, OptionsPattern[]]"Executor" runs the call, None meaning the calling thread.
"Executor" | Default: None |
|---|
DenseMetadataReadBufferDenseMetadataReadBuffer[monolith] reads the geometry and metric of a checkpoint held in a ByteArray or a byte NumericArray, taking the option of DenseMetadataRead.DenseMetadataReadBuffer[monolith, OptionsPattern[]]"Executor" | Default: None |
|---|
DenseSaveDenseSave[receiver, path] saves the index under path, as one monolith or as a manifest beside its shards, while readers keep searching.DenseSave[receiver, path]DenseSaveBufferDenseSaveBuffer[receiver, arena] saves the index into arena, as one monolith for an index of one shard or as a manifest leading one part per shard.DenseSaveBuffer[receiver, arena]