Types

The C standard doesn't specify the signedness of char. On x86 char is signed by default while on Arm it is unsigned by default. That's why we don't define sz_char_t and generally use explicit sz_i8_t and sz_u8_t.

Microsoft Visual C++ (MSVC) typically follows the LLP64 data model on 64-bit platforms, where integers, pointers, and longs differ in size.

GCC and Clang on 64-bit Unix-like systems typically follow the LP64 model instead:

Positive values are reserved for results with a caveat.

Statuses fall into four categories, with no interruptions to report. Resources: sz_bad_alloc_k, sz_missing_gpu_k, sz_device_code_mismatch_k, sz_missing_kernel_k and sz_missing_library_k. Input: sz_invalid_utf8_k, sz_contains_duplicates_k, sz_unexpected_dimensions_k, sz_device_memory_mismatch_k, sz_authentication_failed_k and sz_invalid_syntax_k. State: sz_overflow_risk_k. Defects: sz_status_unknown_k. Any verb that allocates, binds a device or touches a file may also return a Resources status. A verb's own block lists only what its contract adds.

The underlying numeric type holds 4 bytes, with over 4 billion possible states, but:

• UTF-8 in its largest 4-byte form has only 3 + 6 + 6 + 6 = 21 bits of usable space, for 2 million states.
• Unicode, in turn, has only 1'114'112 possible code points from U+0000 to U+10FFFF.
• Of those, in Unicode 16, only 155'063 are assigned characters, just over 17 bits' worth.

That's 0.004% of the 32-bit space, so sparse data-structures suit UTF-8 oriented algorithms.

It's a Plain-Old Datatype (POD) type, unlike the std::string_view.

It can be filled and probed like this:

#include <stringzilla/types.h>
int main() {
    char const *alphabet = "abcdefghijklmnopqrstuvwxyz";
    sz_byteset_t byteset;
    sz_byteset_init(&byteset);
    for (sz_size_t i = 0; i < 26; ++i)
        sz_byteset_add(&byteset, alphabet[i]);
    return sz_byteset_contains(&byteset, 'a') && !sz_byteset_contains(&byteset, 'A') ? 0 : 1;
}

It can help view the contents as 8-bit, 16-bit, 32-bit, or 64-bit integers, as well as 1x XMM register.

It can help view the contents as 8-bit, 16-bit, 32-bit, or 64-bit integers, as well as 2x XMM registers or 1x YMM register.

It can help view the contents as 8-bit, 16-bit, 32-bit, or 64-bit integers, as well as 4x XMM registers or 2x YMM registers or 1x ZMM register.

Kernels call it once per block of up to 64 indices, and it must accept any count.

Tapes are what Apache Arrow, the Python Strings class and GPU kernels read directly. The accessor covers every other shape, from arrays of views to the nodes of a parsed tree, at one indirect call per block of strings. A slice of a tape keeps its parent's offsets, so its first offset may be above zero.

Its payload and offsets are two blocks of exactly the sizes they need, so both sizes follow from the offsets, which the host can always read. A zeroed span is an empty tape holding no blocks.

Consider sz_byteset_add_u8.

Consider sz_byteset_contains_u8.

Lets an ascending-only byte-compress, like vpcompressb, pack lanes in descending order.

bits must hold more than n set bits. One tested home for the per-ISA SIMD "n-th lane" locate.

Similar to the _mm_blend_epi16 intrinsic on x86.

Mirrors CPython's ADJUST_INDICES: negative indices count from the end, end is clamped into the string, and a window with start past end, out of range or inverted, is "empty" - exactly when str.find("") and str.rfind("") return -1.

Used for Pythonic-style slicing.

Assumes a non-zero divisor and no overflow on x + divisor.

There is a well known SWAR sequence for that known to chess programmers, willing to flip a bit-matrix of pieces along the main A1-H8 diagonal.

A step of one over a tape yields a tape over the same bytes and offsets, leaving views untouched. Every other selection fills views and points the accessor of slice at them.

The default often comes from locale.h, stddef.h, stdio.h, stdlib.h, string.h, time.h, or wchar.h.

A branchless approach is well known for signed integers, but it doesn't apply to unsigned ones. The simple branching baseline comes first, then the shift-only form for signed integers, then two forms for any integers, with and without multiplication: