Checkpoints ride the SafeGraphs interchange format, and the path extension picks the layout: a .safegraphs monolith, or a .safegraphs.json manifest beside its shards.
try index.on(executor).save(path: "index.safegraphs")
let copied = try DenseIndex.loadRam("index.safegraphs", config: DenseLoadRamConfig(executor: executor))
let mapped = try DenseIndex.viewRam("index.safegraphs")
let described = try DenseIndex.metadataRead("index.safegraphs")
print(described.dimensions, described.metric, described.quantization)loadRam and loadVram copy the checkpoint into a new mutable index; viewRam serves it in place through a memory mapping, and every mutating verb on a viewed index reports readOnly.
Saving demands a mutation-free window, while concurrent readers keep searching throughout.
An in-memory checkpoint is the same bytes reached another way. saveBuffer returns them as a Monolith read in place from an Arena, since only the call knows how many there are, and loadRamBuffer and metadataReadBuffer read them back through a span.
let monolith = try index.on(executor).saveBuffer(arena: arena)
let restored = try monolith.withUnsafeBytes { bytes in
try DenseIndex.loadRamBuffer(bytes.bytes, config: DenseLoadRamBufferConfig(executor: executor))
}DenseMetadataThe geometry and metric a checkpoint froze - everything a reader learns without opening it, and nothing a reader must choose for itself.4struct DenseMetadataA checkpoint records neither the placement nor the bound on entries sharing one key: the first says where it was written rather than where a reader must keep it, and the second bounds a live handle's mutable state rather than the serialized form. Both stay the caller's to choose when loading.
SendabledimensionsThe scalars per stored vector.var dimensions: IntmetricThe distance kind the checkpoint scores with.var metric: DenseMetricquantizationThe scalar type its vectors are stored and quantized into.var quantization: ScalarconnectivityThe connections-per-node bound the graph was built with.var connectivity: IntDenseSaveConfigThe knobs of save.3struct DenseSaveConfigprogressCalled with shards written.var progress: Progress?progressStepsReports the call makes at most, the last at completion; zero derives to ten.var progressSteps: Intinitinit(progress: Progress? = nil, progressSteps: Int = 0)DenseSaveBufferConfigThe knobs of saveBuffer.3struct DenseSaveBufferConfigprogressCalled with bytes written.var progress: Progress?progressStepsReports the call makes at most, the last at completion; zero derives to ten.var progressSteps: Intinitinit(progress: Progress? = nil, progressSteps: Int = 0)DenseLoadRamConfigThe knobs of loadRam.7struct DenseLoadRamConfigexecutorThe executor the read fans out over, one shard per thread; nil reads on the calling thread.var executor: Executor?memoryDomainsThe memory domains to place the parts on, part p loading into shard p modulo their count; empty derives every node the process may allocate on.var memoryDomains: [Int32]cpuClassThe class to dispatch as; nil derives the CPU's own.var cpuClass: CPUClass?valuesPerKeyThe bound on entries sharing one key, which no checkpoint records; zero is unbounded.var valuesPerKey: IntprogressCalled with shards placed.var progress: Progress?progressStepsReports the call makes at most, the last at completion; zero derives to ten.var progressSteps: Intinitinit(executor: Executor? = nil, memoryDomains: [Int32] = [], cpuClass: CPUClass? = nil, valuesPerKey: Int = 0, progress: Progress? = nil, progressSteps: Int = 0)DenseLoadVramConfigThe knobs of loadVram.7struct DenseLoadVramConfigexecutorThe executor the upload runs on, one stream per device; nil uploads from the calling thread.var executor: Executor?devicesThe devices to place the parts on, part p loading onto device p modulo their count; empty derives one free device.var devices: [Int32]gpuClassThe class to dispatch as; nil derives the devices' own.var gpuClass: GPUClass?valuesPerKeyThe bound on entries sharing one key, which no checkpoint records; zero is unbounded.var valuesPerKey: IntprogressCalled with shards placed.var progress: Progress?progressStepsReports the call makes at most, the last at completion; zero derives to ten.var progressSteps: Intinitinit(executor: Executor? = nil, devices: [Int32] = [], gpuClass: GPUClass? = nil, valuesPerKey: Int = 0, progress: Progress? = nil, progressSteps: Int = 0)DenseLoadRamBufferConfigThe knobs of loadRamBuffer.7struct DenseLoadRamBufferConfigexecutorThe executor the copy fans out over; nil copies on the calling thread.var executor: Executor?memoryDomainsThe memory domains to place shards on; the monolith lands whole on the first.var memoryDomains: [Int32]cpuClassThe class to dispatch as; nil derives the CPU's own.var cpuClass: CPUClass?valuesPerKeyThe bound on entries sharing one key, which no checkpoint records; zero is unbounded.var valuesPerKey: IntprogressCalled with bytes placed.var progress: Progress?progressStepsReports the call makes at most, the last at completion; zero derives to ten.var progressSteps: Intinitinit(executor: Executor? = nil, memoryDomains: [Int32] = [], cpuClass: CPUClass? = nil, valuesPerKey: Int = 0, progress: Progress? = nil, progressSteps: Int = 0)DenseLoadVramBufferConfigThe knobs of loadVramBuffer.7struct DenseLoadVramBufferConfigexecutorThe executor the upload runs on, one stream per device; nil uploads from the calling thread.var executor: Executor?devicesThe devices to place shards on; the monolith lands whole on the first.var devices: [Int32]gpuClassThe class to dispatch as; nil derives the devices' own.var gpuClass: GPUClass?valuesPerKeyThe bound on entries sharing one key, which no checkpoint records; zero is unbounded.var valuesPerKey: IntprogressCalled with bytes placed.var progress: Progress?progressStepsReports the call makes at most, the last at completion; zero derives to ten.var progressSteps: Intinitinit(executor: Executor? = nil, devices: [Int32] = [], gpuClass: GPUClass? = nil, valuesPerKey: Int = 0, progress: Progress? = nil, progressSteps: Int = 0)DenseViewNvmeConfigThe knobs of viewNvme.4struct DenseViewNvmeConfigexecutorThe NVMe executor whose rings read the checkpoint's headers; nil reads on the calling thread.var executor: Executor?pathsThe devices the parts are served from - a filesystem path, a transport identifier, or a FUSE mount over object storage; empty serves each part where its manifest points.var paths: [String]cpuClassThe class to dispatch as; nil derives the CPU's own.var cpuClass: CPUClass?initinit(executor: Executor? = nil, paths: [String] = [], cpuClass: CPUClass? = nil)MonolithThe monolith saveBuffer wrote into an arena, read in place.2struct MonolithcountThe bytes in the monolith.var count: Int { get }withUnsafeBytesLends the serialized checkpoint to body, which must not keep the buffer past its return.func withUnsafeBytes<Result, Failure>(
_ body: (UnsafeRawBufferPointer) throws(Failure) -> Result
) throws(Failure) -> Result where Failure : Error