Scalar Types

Used for Hamming distance and Jaccard similarity via popcount. Dimension count must be a multiple of 8, the values per byte.

Range per element: [−8, +7]. Elements sign-extended to i8 for arithmetic. Dimension count must be a multiple of 2, the values per byte.

Range per element: [0, 15]. Elements zero-extended to u8 for arithmetic. Dimension count must be a multiple of 2, the values per byte.

Range: ±448, no infinities (all-ones exponent → NaN at 0x7F/0xFF). 114 of 254 finite values, or 44.9%, fall in [−1, +1].

Range: ±57 344, supports infinities at 0x7C/0xFC. 122 of 248 finite values (49.2%) fall in [−1, +1].

Stored as 0b00SEEMMM with 2 bits of padding. Range: ±7.5, no infinities or NaN. 64 total codes: 48 normal, 14 subnormal (exp=0, mant ≠ 0), 2 zeros (±0). 18 of 64 values (28.1%) fall in [−1, +1]. Subnormal values span [±0.125, ±0.875]. Losslessly promotable to E4M3 by rebiasing exponent +6 (normals) or normalizing (subnormals).

Stored as 0b00SEEEMM with 2 bits of padding. Range: ±28, no infinities or NaN. 64 total codes: 56 normal, 6 subnormal (exp=0, mant ≠ 0), 2 zeros (±0). 26 of 64 values (40.6%) fall in [−1, +1]. Subnormal values span [±0.0625, ±0.1875]. Losslessly promotable to E5M2 by rebiasing exponent +12 (normals) or normalizing (subnormals).

OCP MX v1.0 sub-format: sign(1) + exponent(2) + mantissa(1), bias=1. Range: ±6.0, no Inf or NaN. 16 total codes: 8 magnitudes {0, 0.5, 1.0, 1.5, 2.0, 3.0, 4.0, 6.0} × 2 signs (two zeros). Used as the element type of MXFP4 (block=32, UE8M0 scales) and NVFP4 (block=16, UE4M3 scales). Dimension count must be a multiple of 2, the values per byte.

Encodes 2^(v - 127) for v ∈ [1, 254]; v = 0 is zero; v = 255 is the NaN-block sentinel. Used as the per-block scale byte for MXFP4, MXFP6, MXFP8, MXINT8.

Range: [0, +448]. Used as the per-block scale byte for NVFP4 (block=16) alongside an f32 "tensor scale".

Range: [−128, +127].

Range: [0, 255].

Range: [−32 768, +32 767].

Range: [0, 65 535].

Range: [−2³¹, +2³¹−1].

Range: [0, 2³²−1].

Range: [−2⁶³, +2⁶³−1].

Range: [0, 2⁶⁴−1].

sign(1) + exponent(8) + mantissa(23), bias=127.

sign(1) + exponent(11) + mantissa(52), bias=1023.

Includes complex type descriptors which in C code would use the real counterparts, but the independent flags contain metadata to be passed between programming language interfaces.

Block-scaled formats are composites, element_dtype | scale_dtype: two bits set, so they never collide with each other or with a singleton. The scale dtype fixes the block, 16 for UE4M3 in NVFP4 and 32 for UE8M0 in the MX family.

Elements are grouped in fixed-size contiguous blocks; each block has its own scale_dtype scale byte stored in a separate scales buffer. An optional tensor_scale_dtype scalar multiplier applies to the whole tensor (NVFP4's per-tensor FP32 factor; absent for the MX family).

Plain (non-block-scaled) buffers are described by scale_dtype = nk_dtype_unknown_k, tensor_scale_dtype = nk_dtype_unknown_k, and block_size = 0 — use the nk_plain() factory.

Clang and GCC bring the float16_t symbol when you compile for AArch64. MSVC lacks it, and it's vld1_f16-like intrinsics are in reality macros, that cast to 16-bit integers internally, instead of using floats. Some of those are defined as aliases, so we use #define preprocessor directives instead of typedef to avoid errors.

Block-scaled composite POD types — one value is one whole block.

Each composite combines a packed element buffer with its per-block scale byte in a single POD struct. The struct IS the value: sizeof(nk_nvfp4_t) == 9 means one NVFP4 value, not one element. Storage in containers strides by sizeof(struct); logical element count per value is reported by nk_dimensions_per_value(composite_dtype) (16 for NVFP4, 32 for MX). Dimension count must be a multiple of that block size.

The NVFP4 per-tensor f32 tensor scale multiplier is not part of the block value — it lives on the enclosing tensor and is passed explicitly to encode/decode helpers.

Kernel outputs widened to f32c.

Kernel outputs widened to f32c.

TODO: On GCC and Clang we use __transparent_union__ attribute to allow implicit conversions between the different vector types when passing them as function arguments. The most important side-effect of this is that the argument of such type is passed to functions using the calling convention of the first member of the union, which in our case is a register-based calling convention for SIMD types.

Bridges different type systems (Python, JavaScript, etc.) where scalars arrive as f64 but need to be passed to kernels as typed pointers. Callers fill the appropriate union member based on the target dtype, then pass the union address as void const *.

Uses mask = -sign; ordered = value ^ mask — the constant offset cancels in subtraction. Returns negative if a < b, 0 if equal, positive if a > b.

Based on xfastsinf_u3500 in the SLEEF library.

Based on xfastcosf_u3500 in the SLEEF library.

Based on xatanf in the SLEEF library.

Based on xatan in the SLEEF library.

Based on xatan2f in the SLEEF library.

Based on xatan2 in the SLEEF library.

Based on xsin in the SLEEF library.

Based on xcos in the SLEEF library.

A block-scaled dtype is one of the composites, like nk_mxfp8e4m3_k or nk_nvfp4_k, whose element dtype, scale dtype and block size nk_block_scaled_format_of_dtype derives:

Codes and scales are contiguous runs. A destination NVFP4 tensor_scale holding a non-zero value is applied as is; holding zero, it receives the tensor scale that the kernel derives from the source data it encodes.

E4M3 has 1 sign bit, 4 exponent bits biased by 7, and 3 mantissa bits. It spans [-448, +448] with no infinities and only two NaN encodings, 0x7F and 0xFF. Subnormals decode to the value (-1)ˢ × mantissa × 2⁻⁹, which is mantissa / 512.

E4M3 has 1 sign bit, 4 exponent bits biased by 7, and 3 mantissa bits. It spans [-448, +448] with no infinities and only two NaN encodings. Values with |x| < 2⁻⁶ are encoded as subnormals.

E5M2 has 1 sign bit, 5 exponent bits biased by 15, and 2 mantissa bits. Its range is ±57344, and it keeps IEEE 754 infinities and NaNs. Values with |x| < 2⁻¹⁴ are encoded as subnormals.

E2M3FN has 1 sign bit, 2 exponent bits biased by 1, and 3 mantissa bits. It spans [-7.5, +7.5] with no infinities or NaNs, and saturates to the maximum on overflow. Values with |x| < 0.5 are encoded as subnormals.

E3M2FN has 1 sign bit, 3 exponent bits biased by 3, and 2 mantissa bits. It spans [-28, +28] with no infinities or NaNs, and saturates to the maximum on overflow. Values with |x| < 0.25 are encoded as subnormals.

A zero byte decodes to 0, 0xFF to NaN as the block-NaN sentinel, and any other v to 2^(v - 127).

Rounds to the nearest power of two, ties to even in log2 space, per the OCP MX, NVIDIA and AMD convention. Plain ceil or floor biases the block's dynamic range, as floor underestimates it, so the split sits at the geometric midpoint √2 × 2^e. NaN maps to 0xFF as the block-NaN sentinel, zero and subnormals map to 0x00, and overflow saturates to 0xFE.

Takes the absolute value first, as the sign bit is not representable, then rounds to nearest even like the underlying E4M3 encoder, matching the NVFP4 and OCP scale convention. NaN maps to the E4M3 NaN code 0x7F.

tensor_scale is the per-tensor multiplier.

Returns nk_plain(dtype) when dtype is not a composite.

Floats get their unit roundoff with two bits of slack for matrix units that truncate; integer sums are exact. Kernel families derive their bounds from it, like nk_dot_error_bound.

For sub-byte types multiple dimensions share a single byte container. For byte-or-larger types this is always 1.