# Documents

> UStore for Rust

Documents are JSON text, which the engine parses as it writes them.
`Collection::docs` parses what it reads through JsonTape: `get` hands out a `jsontape::Json` parsed out of the one-off arena it read through, `get_json` hands out the text itself as a `String`, and `set` writes a `jsontape::Json`.
With the `serde` feature, `get_as` deserializes a document into any `DeserializeOwned` type.
`ustore::Json` names JSON text in the batch verbs below, so JsonTape's types are always spelled with their `jsontape::` prefix.

```rust
let mut movies = database.collection("movies")?;
movies.docs().set(avatar, &jsontape::parse(br#"{"title":"Avatar","year":2009}"#)?)?;
let title = movies.docs().get(avatar)?.and_then(|movie| movie["title"].as_str().map(str::to_owned));
let text: Option<String> = movies.docs().get_json(avatar)?;
let fields: Option<BTreeMap<String, jsontape::Json>> = movies.docs().get_as(avatar)?;
```

The batch verbs carry documents as text both ways and parse nothing on the Rust side.
A field is a top-level key or an RFC 6901 pointer, which crosses as the bytes of a `&str`, so a `"/user/name"` literal crosses as it stands, and an empty field addresses the whole document.
`docs_write` writes whole documents or fields inside them, and `docs_read` reads either, as `ustore_docs_write` and `ustore_docs_read` do.

```rust
use ustore::{Column, Document, DocumentEntry, FieldAddress, FieldEntry, Json, KeyedBy, Modification, Text};

let documents = [DocumentEntry::<Json>::new(ada, r#"{"name":"Ada","born":1815}"#)];
database.head().docs_write(people, &documents, Modification::Upsert, Durability::Buffered)?;

let born = [DocumentEntry::<i64>::new(Key::from(5u64), &1815)];
database.head().docs_write(people, &born, Modification::Upsert, Durability::Buffered)?;

let keyed = [Document::new(r#"{"_id":2,"name":"Grace"}"#)];
database.head().docs_write(people, KeyedBy("_id", &keyed), Modification::Insert, Durability::Buffered)?;

let died = [FieldEntry::<i64>::new(ada, "died", &1852)];
database.head().docs_write(people, &died, Modification::Upsert, Durability::Buffered)?;
let family = [FieldEntry::<Json>::new(ada, "/family", r#"{"father":"Byron"}"#)];
database.head().docs_write(people, &family, Modification::Upsert, Durability::Buffered)?;
let unborn = [DocumentEntry::<Json>::new(ada, r#"{"born":null}"#)];
database.head().docs_write(people, &unborn, Modification::Merge, Durability::Buffered)?;

let whole = database.head().docs_read(people, [ada, grace], Json, &mut arena)?;
let names = database.head().docs_read(people, &[FieldAddress::new(ada, "name")], Text, &mut arena)?;
```

A write carries one format for the whole batch, named by the type of its entries: a scalar from `f64` down to `u8` or `bool`, `Text` for a JSON string, or `Json` for any JSON value.
The same formats write a whole document, which is then one number, one boolean or one string rather than an object.
`KeyedBy` writes documents that carry their keys under a field of their own: a non-negative integer, spelled as `Key::from` spells it, or a UUID in canonical text.
Mixing two formats in one batch does not compile, and every entry carries a value: `On::remove` removes whole documents, and a merge carrying a JSON `null` removes a field.
`Modification` chooses between an upsert, an update, an insert, an RFC 6902 patch and an RFC 7386 merge.
Reads hand back the same `Values` a binary read does, of keys for whole documents or of `FieldAddress` values for fields, exported in the format named before the arena, `Json` or printed `Text`, with a missing document or field reported as `None`.

Every document verb also takes owned inputs, which a deferred view needs.
Whole documents go as keys beside a `stringtape::CharsTape` of JSON texts, or as the tape alone inside a `KeyedBy`, and field writes as keys and a `CharsTape` of paths beside a tape of JSON values.
A field read takes keys beside a `CharsTape` of paths, and a gather a `Vec<Column<'static>>`, whose `Column::new` takes a `&str` or a `String`.

```rust
use ustore::stringtape::CharsTape;

let deferred = database.head().deferred(&executor);
let texts: CharsTape<u64> = [r#"{"name":"Ada"}"#, r#"{"name":"Grace"}"#].into_iter().collect();
deferred.docs_write(people, (vec![ada, grace], texts), Modification::Upsert, Durability::Buffered)?.wait()?;

let paths: CharsTape<u32> = ["name", "name"].into_iter().collect();
let names = deferred.docs_read(people, (vec![ada, grace], paths), Text, &mut arena)?.await?;
assert_eq!(names.get(1), Some(b"Grace".as_slice()));
```

A gist lists the unique paths to the leaves of a batch of documents as `&str` pointers, and a gather converts fields into typed columns, a row per key:

```rust
let paths = database.head().docs_gist(people, [ada, grace], &mut arena)?;

let columns = [Column::new::<u64>("born"), Column::new::<Text>("name")];
let gathered = database.head().docs_gather(people, &[ada, grace], &columns, &mut other_arena)?;
let born: &[u64] = gathered.scalars::<u64>(0)?;
let valid = gathered.validities(0)?.unwrap().get(1);
let name = gathered.strings(1)?.get(0);
```

`Columns` borrow their arena and hold the columns they were asked for, borrowed or owned, and hand out scalar columns as slices straight out of the arena, and string columns as a `StringColumn` each, whose `Text` strings come without the null terminator the arena keeps after them.
Beside the scalars a gather converts into, `Half` is a column of IEEE 754 binary16 bit patterns the caller reinterprets and `UUID` a column of sixteen bytes parsed from canonical text; neither is a format a write carries, and a value that will not fit either leaves its cell zero and sets the collision bit of its row.
Every column carries three bitmaps: validity for a value found and converted, conversions for a value whose meaning changed on the way, such as text parsed into a number, and collisions for a value of a type that would not convert.
Asking for a column in a type other than the one it was gathered in is refused with `Status::InvalidArgument`.
Paths from one arena address the fields a gather reads through another.

`use ustore::Docs;`

`use ustore::Binary;`

`use ustore::Bitmap;`

`use ustore::Column;`

`use ustore::ColumnFormat;`

`use ustore::Columns;`

`use ustore::DocsGatherConfig;`

`use ustore::Document;`

`use ustore::DocumentEntry;`

`use ustore::FieldAddress;`

`use ustore::FieldEntry;`

`use ustore::FieldFormat;`

`use ustore::FieldType;`

`use ustore::Half;`

`use ustore::Json;`

`use ustore::KeyedBy;`

`use ustore::Modification;`

`use ustore::Paths;`

`use ustore::ReadFormat;`

`use ustore::Scalar;`

`use ustore::StringColumn;`

`use ustore::Text;`

`use ustore::WritableScalar;`

`use ustore::docs::DocsWrite;`

`use ustore::docs::DocsRemove;`

`use ustore::docs::DocsRemoveRange;`

`use ustore::docs::DocsRead;`

`use ustore::docs::DocsGist;`

`use ustore::docs::DocsGather;`

## Docs

The JSON documents of one collection, parsed through JsonTape.

```rust
pub struct Docs<'collection, 'view, Parent = Database>
```

Every verb consumes the view; [`Collection::docs`](https://www.unum.cloud/docs/ustore/rust/collections.md#docs) hands out the next one.

Implements: `Debug`

## get

The document under [`key`](#key), parsed out of the one-off arena it was read into, or [`None`](https://www.unum.cloud/docs/ustore/rust/transactions.md#None) for a key without one.

```rust
pub fn get(self, key: Key) -> Result<Option<Json>>
```

## get_json

The document under [`key`](#key) as the JSON text the engine holds, or [`None`](https://www.unum.cloud/docs/ustore/rust/transactions.md#None) for a key without one.

```rust
pub fn get_json(self, key: Key) -> Result<Option<String>>
```

## set

Stores `document` under [`key`](#key), creating the collection to hold documents when it is missing.

```rust
pub fn set(self, key: Key, document: &Json) -> Result<()>
```

## remove

Removes the document under [`key`](#key), whether or not the collection holds one.

```rust
pub fn remove(self, key: Key) -> Result<()>
```

## Binary

A gathered column of printed text without null terminators.

```rust
pub struct Binary;
```

Implements: `Clone`, `Copy`, `StructuralPartialEq`, `PartialEq`, `Eq`, `Debug`, `Hash`, `[ColumnFormat](#ColumnFormat)`

## Bitmap

One bit per row of a batch, least significant bit first.

```rust
pub struct Bitmap<'arena>
```

Implements: `Clone`, `Copy`, `Debug`

## len

How many rows the bitmap covers.

```rust
pub fn len(&self) -> usize
```

## is_empty

Whether the bitmap covers no rows.

```rust
pub fn is_empty(&self) -> bool
```

## get

Whether the bit of `row` is set, which a row past the end never is.

```rust
pub fn get(&self, row: usize) -> bool
```

## as_bytes

The bytes of the bitmap, as Arrow reads a validity buffer.

```rust
pub fn as_bytes(&self) -> &'arena [u8]
```

## Column

A field to gather from every document, borrowed or owned, and the type to convert it into.

```rust
pub struct Column<'field>
```

Implements: `Clone`, `Debug`

## new

Gathers `field`, a `&str` or an owned [`String`](#String), converted into `Format`.

```rust
pub fn new<Format: ColumnFormat>(field: impl Into<Cow<'field, str>>) -> Self
```

## ColumnFormat

The format a gathered column converts its values into.

```rust
pub trait ColumnFormat: Format
```

Implementors: - `impl<Value: [Scalar](#Scalar)> [ColumnFormat](#ColumnFormat) for Value`
- `impl [ColumnFormat](#ColumnFormat) for [Text](https://www.unum.cloud/docs/ustore/rust/platform-notes.md#Text)`
- `impl [ColumnFormat](#ColumnFormat) for [Binary](#Binary)`

## Columns

The columns one gather exported, a row per document and a column per requested field, holding the requests that name the format of each.

```rust
pub struct Columns<'arena, Requested>
```

A value is valid when it was found and converted, converted when the conversion changed its meaning,
such as text parsed into a number, and colliding when a value of an inconvertible type sits there.

Implements: `Clone`, `Copy`, `Debug`

## rows

How many documents the gather read.

```rust
pub fn rows(&self) -> usize
```

## columns

How many fields the gather read.

```rust
pub fn columns(&self) -> usize
```

## validities

Which rows of `column` hold a value, or [`None`](https://www.unum.cloud/docs/ustore/rust/transactions.md#None) past the last column.

```rust
pub fn validities(&self, column: usize) -> Result<Option<Bitmap<'arena>>>
```

## conversions

Which rows of `column` changed their meaning on conversion, or [`None`](https://www.unum.cloud/docs/ustore/rust/transactions.md#None) past the last column.

```rust
pub fn conversions(&self, column: usize) -> Result<Option<Bitmap<'arena>>>
```

## collisions

Which rows of `column` hold a value that would not convert, or [`None`](https://www.unum.cloud/docs/ustore/rust/transactions.md#None) past the last column.

```rust
pub fn collisions(&self, column: usize) -> Result<Option<Bitmap<'arena>>>
```

## scalars

The values of a scalar column, one per row, with zero or a partial conversion in invalid rows.

```rust
pub fn scalars<Value: Scalar>(&self, column: usize) -> Result<&'arena [Value]>
```

## strings

The values of a column gathered as [`Text`](https://www.unum.cloud/docs/ustore/rust/platform-notes.md#Text) or [`Binary`](#Binary).

```rust
pub fn strings(&self, column: usize) -> Result<StringColumn<'arena>>
```

## to_arrow

Every column as one Arrow array beside its schema, in request order, or none for a gather that reached no engine.

```rust
pub fn to_arrow(&self) -> Result<Vec<ArrowExport<'arena>>>
```

Scalars and [`Binary`](#Binary) columns export in place, and [`Text`](https://www.unum.cloud/docs/ustore/rust/platform-notes.md#Text) ones copied without their
terminators. The columns of a device arena stay on the device, as
[`ArrowDeviceArray::device_type`](https://www.unum.cloud/docs/ustore/rust/reads-and-writes.md#ArrowDeviceArray) says. Each export borrows the arena, so the call that
would rewind it does not compile while one lives:

```
use ustore::{Arena, Column, RAMOptions, Database, Key, Modality, Text};

let database = Database::init_ram(&#x26;RAMOptions::default())?;
let mut arena = Arena::new()?;
let people = database.collection_create("people", Modality::Docs)?;
let columns = [Column::new::&#x3C;Text>("name")];
let ada = Key::from(1u64);
let exported = database.head().docs_gather(people, &#x26;[ada], &#x26;columns, &#x26;mut arena)?.to_arrow()?;
database.head().docs_gist(people, [ada], &#x26;mut arena)?;
drop(exported);
```

Calls `ustore_docs_column_view_to_arrow`.

## DocsGatherConfig

The typed columns a gather reads; see [`On::docs_gather`](https://www.unum.cloud/docs/ustore/rust/batches-and-arenas.md#docs_gather).

```rust
pub struct DocsGatherConfig<Requested>
```

- `columns` `Requested`: The fields and the format each lands in, borrowed or owned.

Implements: `Clone`, `Copy`, `Debug`, `From<Requested>`

## Document

A document carrying its own key under an id field, which [`KeyedBy`](#KeyedBy) names, laid out so a slice of them crosses through an accessor.

```rust
pub struct Document<'text>
```

Implements: `Clone`, `Copy`, `Debug`

## new

Borrows the JSON text of a document.

```rust
pub fn new(text: &'text str) -> Self
```

## DocumentEntry

A document and the key it is stored under, laid out so a slice of them crosses as it stands: the keys as one strided run, and the documents through an accessor over the same entries.

```rust
pub struct DocumentEntry<'value, Format: FieldFormat>
```

A whole document is written in one format the batch shares: [`Json`](https://www.unum.cloud/docs/ustore/rust/platform-notes.md#Json) for the text the engine parses,
[`Text`](https://www.unum.cloud/docs/ustore/rust/platform-notes.md#Text) for a document that is one JSON string, or any [`WritableScalar`](#WritableScalar) for a document that is
one number or one boolean.

Implements: `Clone`, `Copy`, `Debug`

## new

Pairs a key with the document to store under it.

```rust
pub fn new(key: Key, value: &'value Format::Value) -> Self
```

## key

The key this entry stores under.

```rust
pub fn key(&self) -> Key
```

## FieldAddress

A field of one document to read, laid out so a slice of them crosses as it stands.

```rust
pub struct FieldAddress<'field>
```

Implements: `Clone`, `Copy`, `Debug`

## new

Addresses `field` of the document under [`key`](#key).

```rust
pub fn new(key: Key, field: &'field str) -> Self
```

## FieldEntry

A field of one document and the value written into it, laid out so a slice of them crosses as it stands.

```rust
pub struct FieldEntry<'value, Format: FieldFormat>
```

Implements: `Clone`, `Copy`, `Debug`

## new

Pairs a field with the value to write into it.

```rust
pub fn new(key: Key, field: &'value str, value: &'value Format::Value) -> Self
```

## FieldFormat

The format of the values one field write carries, which the whole batch shares.

```rust
pub trait FieldFormat: Format
```

Implementors: - `impl<Value: [WritableScalar](#WritableScalar)> [FieldFormat](#FieldFormat) for Value`
- `impl [FieldFormat](#FieldFormat) for [Json](https://www.unum.cloud/docs/ustore/rust/platform-notes.md#Json)`
- `impl [FieldFormat](#FieldFormat) for [Text](https://www.unum.cloud/docs/ustore/rust/platform-notes.md#Text)`

## Value

What a field of this format is written from.

```rust
type Value: ?Sized;
```

## *

The bytes the ABI reads for [`value`](https://www.unum.cloud/docs/ustore/rust/scans.md#value).

```rust
fn bytes(value: &Self::Value) -> &[u8]
```

## FieldType

The type a field crosses as, which every format above names one of.

```rust
pub enum FieldType
```

- `Json` = `0`: JSON text.
- `Null` = `10`: The JSON null.
- `Bool` = `11`: A boolean.
- `UUID` = `12`: A 128-bit UUID.
- `F64` = `42`: A 64-bit float.
- `F32` = `41`: A 32-bit float.
- `F16` = `40`: A 16-bit float.
- `I64` = `23`: A signed 64-bit integer.
- `I32` = `22`: A signed 32-bit integer.
- `I16` = `21`: A signed 16-bit integer.
- `I8` = `20`: A signed 8-bit integer.
- `U64` = `33`: An unsigned 64-bit integer.
- `U32` = `32`: An unsigned 32-bit integer.
- `U16` = `31`: An unsigned 16-bit integer.
- `U8` = `30`: An unsigned 8-bit integer.
- `Binary` = `50`: The text of a value without a null terminator.
- `String` = `51`: A value printed as text.

Implements: `Clone`, `Copy`, `StructuralPartialEq`, `PartialEq`, `Eq`, `Debug`, `Default`, `Hash`, `Display`, `FromStr`

## Half

A gathered half-precision float, held as the two bytes of its IEEE 754 binary16 encoding.

```rust
pub struct Half(pub u16);
```

The crate reinterprets nothing: a value whose magnitude will not fit sets the collision bit of its
row and leaves the cell zero.

- `0` `u16`

Implements: `[ColumnFormat](#ColumnFormat)`, `Clone`, `Copy`, `StructuralPartialEq`, `PartialEq`, `Eq`, `Debug`, `Default`, `Hash`, `[Scalar](#Scalar)`

## Json

JSON text: a whole value written into a field, or a field read back as the JSON it holds.

```rust
pub struct Json;
```

Implements: `Clone`, `Copy`, `StructuralPartialEq`, `PartialEq`, `Eq`, `Debug`, `Hash`, `[FieldFormat](#FieldFormat)`, `[ReadFormat](#ReadFormat)`

## KeyedBy

Documents carrying their keys under one field of their own, as a docs write takes them: borrowed [`Document`](#Document) values, or an owned `stringtape::CharsTape` of JSON texts, which a deferred write needs beside an owned or `'static` field.

```rust
pub struct KeyedBy<Field, Texts>(pub Field, pub Texts);
```

- `0` `Field`: The field holding the key of every document, such as "_id": a non-negative integer, spelled as
[`Key::from`](https://www.unum.cloud/docs/ustore/rust/collections.md#Key) spells it, or a UUID in canonical text.
- `1` `Texts`: The documents.

Implements: `Clone`, `Debug`

## Modification

How a write combines what it carries with what the document already holds.

```rust
pub enum Modification
```

- `Upsert` = `0`: Replaces the document or the field, creating it when missing.
- `Update` = `1`: Replaces the document or the field, which must exist.
- `Insert` = `2`: Creates the document or the field, which must not exist.
- `Patch` = `3`: Applies the carried RFC 6902 patch.
- `Merge` = `4`: Applies the carried RFC 7386 merge patch.

Implements: `Clone`, `Copy`, `StructuralPartialEq`, `PartialEq`, `Eq`, `Debug`, `Default`, `Hash`, `Display`, `FromStr`

## Paths

The unique paths to the leaves of the documents one gist inspected, in the order they were found.

```rust
pub struct Paths<'arena>
```

Implements: `Clone`, `Copy`, `Debug`

## len

How many unique paths the gist found.

```rust
pub fn len(&self) -> usize
```

## is_empty

Whether the gist found no leaves at all.

```rust
pub fn is_empty(&self) -> bool
```

## get

The JSON Pointer at `index`, ready to address a field, or [`None`](https://www.unum.cloud/docs/ustore/rust/transactions.md#None) past the end.

```rust
pub fn get(&self, index: usize) -> Option<&'arena str>
```

## iter

Every path in the order the gist found it.

```rust
pub fn iter(&self) -> impl Iterator<Item = &'arena str> + 'arena
```

## ReadFormat

The format a field read exports its values in.

```rust
pub trait ReadFormat: Format
```

Implementors: - `impl [ReadFormat](#ReadFormat) for [Json](https://www.unum.cloud/docs/ustore/rust/platform-notes.md#Json)`
- `impl [ReadFormat](#ReadFormat) for [Text](https://www.unum.cloud/docs/ustore/rust/platform-notes.md#Text)`

## Scalar

A fixed-width value a gathered column converts into.

```rust
pub trait Scalar: Format + Copy
```

Implementors: - `impl [Scalar](#Scalar) for f64`
- `impl [Scalar](#Scalar) for f32`
- `impl [Scalar](#Scalar) for [Half](#Half)`
- `impl [Scalar](#Scalar) for i64`
- `impl [Scalar](#Scalar) for i32`
- `impl [Scalar](#Scalar) for i16`
- `impl [Scalar](#Scalar) for i8`
- `impl [Scalar](#Scalar) for u64`
- `impl [Scalar](#Scalar) for u32`
- `impl [Scalar](#Scalar) for u16`
- `impl [Scalar](#Scalar) for u8`
- `impl [Scalar](#Scalar) for bool`
- `impl [Scalar](#Scalar) for [UUID](#UUID)`

## StringColumn

A gathered column of strings, whose value in a row is absent unless the row is valid.

```rust
pub struct StringColumn<'arena>
```

Implements: `Clone`, `Copy`, `Debug`, `IntoIterator`

## len

How many rows the column holds.

```rust
pub fn len(&self) -> usize
```

## is_empty

Whether the column holds no rows.

```rust
pub fn is_empty(&self) -> bool
```

## get

The string of `row` without its terminator, or [`None`](https://www.unum.cloud/docs/ustore/rust/transactions.md#None) for a row whose field was missing or would not convert.

```rust
pub fn get(&self, row: usize) -> Option<&'arena [u8]>
```

## iter

Every row in order.

```rust
pub fn iter(&self) -> Rows<StringColumn<'arena>, Option<&'arena [u8]>>
```

## Text

A string: a field written as a JSON string, or a value read and gathered as printed text.

```rust
pub struct Text;
```

Implements: `Clone`, `Copy`, `StructuralPartialEq`, `PartialEq`, `Eq`, `Debug`, `Hash`, `[FieldFormat](#FieldFormat)`, `[ReadFormat](#ReadFormat)`, `[ColumnFormat](#ColumnFormat)`

## WritableScalar

A scalar a write carries into a whole document or into one of its fields.

```rust
pub trait WritableScalar: Scalar
```

[`Half`](#Half) and [`UUID`](#UUID) are gathered but never written: the engine builds a document from the bytes
of a scalar and has no JSON spelling for either.

Implementors: - `impl [WritableScalar](#WritableScalar) for f64`
- `impl [WritableScalar](#WritableScalar) for f32`
- `impl [WritableScalar](#WritableScalar) for i64`
- `impl [WritableScalar](#WritableScalar) for i32`
- `impl [WritableScalar](#WritableScalar) for i16`
- `impl [WritableScalar](#WritableScalar) for i8`
- `impl [WritableScalar](#WritableScalar) for u64`
- `impl [WritableScalar](#WritableScalar) for u32`
- `impl [WritableScalar](#WritableScalar) for u16`
- `impl [WritableScalar](#WritableScalar) for u8`
- `impl [WritableScalar](#WritableScalar) for bool`

## DocsWrite

A write of whole documents or of fields inside them; see [`On::docs_write`](https://www.unum.cloud/docs/ustore/rust/batches-and-arenas.md#docs_write).

```rust
pub struct DocsWrite<'collections, Documents>
```

Implements: `Debug`

## DocsRemove

A removal of a batch of whole documents; see [`On::docs_remove`](https://www.unum.cloud/docs/ustore/rust/batches-and-arenas.md#docs_remove).

```rust
pub struct DocsRemove<'collections, Keys>
```

Implements: `Debug`

## DocsRemoveRange

A removal of one range of documents; see [`On::docs_remove_range`](https://www.unum.cloud/docs/ustore/rust/batches-and-arenas.md#docs_remove_range).

```rust
pub struct DocsRemoveRange
```

Implements: `Debug`

## DocsRead

A read of whole documents or of fields inside them; see [`On::docs_read`](https://www.unum.cloud/docs/ustore/rust/batches-and-arenas.md#docs_read).

```rust
pub struct DocsRead<'arena, 'collections, Targets>
```

Implements: `Debug`

## DocsGist

A listing of the paths to the leaves of documents; see [`On::docs_gist`](https://www.unum.cloud/docs/ustore/rust/batches-and-arenas.md#docs_gist).

```rust
pub struct DocsGist<'arena, 'collections, Keys>
```

Implements: `Debug`

## DocsGather

A conversion of fields into typed columns; see [`On::docs_gather`](https://www.unum.cloud/docs/ustore/rust/batches-and-arenas.md#docs_gather).

```rust
pub struct DocsGather<'arena, 'collections, Keys, Requested>
```

Implements: `Debug`