# Reads and Writes

> UStore for Python

### Binary Values

`write`, `set`, `read`, `get`, `contains` and `remove` address binary values by key, while `scan`, `sample` and `measure` walk them, each in the blobs collection its `collection=` names.
A write takes one bytes-like value per key, one value broadcast to every key, or a tape, and borrows every value for the length of the call rather than copying it:

```python
posters = database.collection_create("posters", modality=ustore.Modality.BLOBS)
database.write(keys, [b"a", b"bb", b"ccc"], collection=posters)  # one value per key, read by an accessor
database.write(keys, b"same", collection=posters)       # one value every key repeats
database.write(keys, pyarrow.array([b"a", b"bb", b"ccc"]), collection=posters)  # an Arrow binary array, a u32 tape
database.write(keys, (data, np.array([0, 1, 3, 6], dtype=np.uint64)), collection=posters)  # bytes and offsets, u64
database.write(other_keys, database.read(keys, arena, collection=posters), collection=posters)  # a read, as a tape
```

A tape is an Arrow `binary`, `large_binary`, `string` or `large_string` array, or a `(data, offsets)` tuple of buffers whose offsets are unsigned 32-bit or 64-bit integers, one more than the values, climbing within the data.
It reaches the C ABI as the u32 or u64 tape it already is, so nothing is copied or gathered, and an Arrow null is refused, since `remove` removes keys.
A read answers the tape the engine wrote, with 32-bit offsets while its bytes fit in 4 GiB and 64-bit ones beyond, and `Values` exports it through the Arrow C data interface as `binary` or `large_binary` to match.

`scan(start, wanted, arena)` lists up to `wanted` keys from `start` upwards, and `end=` stops it before a key.
`direction=ScanDirection.DESCENDING` walks down from `start` instead, largest key first, and a `start` of None begins at the first key in the direction:

```python
database.scan(10, 8, arena, collection=posters, end=30)                          # keys in [10, 30)
database.scan(None, 8, arena, collection=posters, direction=ScanDirection.DESCENDING)  # the 8 largest keys
database.scan(30, 8, arena, collection=posters, end=10, direction=ScanDirection.DESCENDING)  # keys in (10, 30]
```

`scan_batch(starts, wanted, arena)` runs one scan per start in one call, and `sample_batch(count, wanted, arena)` draws `count` samples, all from the one collection `collection=` names, every one held to the same `wanted`, which is what the C ABI takes; `scan` and `sample` remain the single forms.
`measure(start, end, arena)` answers a `Measurement` of the bounds on the entries, the bytes of their values and the space they take, without walking the range, which stops before `end` and takes the same `direction=`.
`docs_measure`, `pages_measure` and `graph_measure_vertices` measure the other modalities, as `docs_sample`, `pages_sample` and `graph_sample_vertices` sample them.
`read(keys, arena, offset=16, limit=64)` reads up to 64 bytes of every value from its 16th, where an offset past the end reads empty.
`remove(keys, ends=ends)` removes every key from each of `keys` up to its end in one atomic call, and so do `docs_remove`, `graph_remove_vertices` and `pages_remove`, whose ranges remove whole pages.

`cursor(start=None, *, collection, end=None, direction=ScanDirection.ASCENDING)` opens a walk over the keys of one collection, and its `next(wanted, arena, *, values=False)` pulls the next page into the arena that pull names, with the values of its keys in the same call when asked:

```python
arena = ustore.Arena()
with database.cursor(collection="movies") as cursor:   # reads nothing until its first pull
    while page := cursor.next(4096, arena, values=True):
        for key, poster in zip(page, page.values):      # UUIDs and memoryviews over the arena
            consume(key, poster)
```

A pull resumes past the last key the one before it handed out, so a key written behind the walk never comes and one written ahead of it does.
A page shorter than `wanted` exhausts the walk for good: every later pull answers an empty page, which is falsy, whatever is written since.
A key removed at the head between the listing and the read of one pull reads as an empty value, and a pull that fails or is cancelled leaves the cursor where it was.
The cursor holds its database open, and the transaction or snapshot it reads through, until it closes, so a snapshot may close before the walk ends.
Every pull blocks, on the receiver's executor or the calling thread, so a deferred view opens no cursor, and a cursor serves one pull at a time.
`ustore.keys.scan_keys(view, collection, modality=...)` yields every key through the scans of that modality, `scan`, `docs_scan` or `graph_scan_vertices`, and one arena it keeps for the walk, which the NetworkX and Pandas views build on.

`import ustore`

`from ustore import Values`

## Durability

How durable the writes a call publishes are once it finishes, valued as `ustore_durability_t`.

```python
class Durability(IntEnum)
```

- `BUFFERED` = `0`
- `FLUSHED` = `1`

## BlobsView

The binary values of a collection: one key reads a copy as bytes, a batch a list of them, through one-off arenas.

```python
class BlobsView
```

## ArrowSource

An Arrow stream whose batches a call takes over, or one array, read as a stream of one batch.

```python
type ArrowSource = _ArrowStream | _ArrowArray
```

## IngestOrder

Whether an ingest's keys strictly ascend across its stream, or arrive in any order with the last repeat winning.

```python
type IngestOrder = Literal['sorted', 'unsorted']
```

## Buffer

```python
type Buffer = memoryview | bytes | bytearray
```

## ValueLike

```python
type ValueLike = Buffer
```

## TapeLike

An Arrow `binary`, `large_binary`, `string` or `large_string` array, or a `(data, offsets)` pair of buffers whose offsets are unsigned 32-bit or 64-bit integers, one more than the values, which a write passes without a copy.

```python
type TapeLike = _ArrowArray | tuple[Buffer, Buffer]
```

## TextsLike

One `str` or bytes-like text per task, a tape of them, or one for every task, as markup and names reach pages.

```python
type TextsLike = Sequence[str | Buffer] | str | Buffer | TapeLike
```

## LengthsLike

```python
type LengthsLike = Sequence[int] | int
```

## Values

The values one batch read or cursor pull wrote into an [`Arena`](https://www.unum.cloud/docs/ustore/python/batches-and-arenas.md#Arena), viewing it until it rewinds, and a tape a write takes back.

```python
class Values
```

## __arrow_c_array__

The schema and array PyCapsules of the batch as Arrow `binary`, or `large_binary` once its bytes outgrow 32-bit offsets: the presence bits, offsets and bytes the read wrote, exported in place.

```python
def __arrow_c_array__(
    requested_schema: object | None = None,
) -> tuple[object, object]
```

## __dlpack__

A DLPack capsule over the memory the buffer protocol exports, pinning the arena until its consumer deletes it, so `numpy.from_dlpack` reads it in place.

```python
def __dlpack__(
    *,
    stream: object | None = None,
    max_version: tuple[int, int] | None = None,
    dl_device: tuple[int, int] | None = None,
    copy: bool | None = None,
) -> object
```

## __dlpack_device__

The DLPack device of the arena: (1, 0) for host pages, (2, ordinal) for NVIDIA, (10, ordinal) for AMD.

```python
def __dlpack_device__() -> tuple[int, int]
```

## lengths

The length of every value, derived from the offsets, with None where the key was absent.

```python
lengths: tuple[int | None, ...]
```