C · Dot Products

Dot Products

#include <numkong/dot.h> 8.0.0-rc.1 · b1f866ab

Dot products are one of the broadest parts of the native SDK. They include real, complex, packed-binary, mini-float, and mixed-precision forms.

nk_f32c_t a[384];
nk_f32c_t b[384];
nk_f64c_t out = {0, 0};         // widened f32c → f64c output

nk_dot_f32c_best(a, b, 384, &out, capabilities, NULL);   // complex inner product
nk_vdot_f32c_best(a, b, 384, &out, capabilities, NULL);  // conjugated variant, like numpy.vdot

For quantized retrieval pipelines, the storage format often matters more than the nominal math family. The native SDK lets you keep the compact representation and still get a widened output.

API

Functions

nk_dot_*_best× 13Dot product computing the sum of elementwise products between two vectors.
dot.h:162 ↗
nk_status_t nk_dot_f32_best(
    nk_f32_t const *a,
    nk_f32_t const *b,
    nk_size_t n,
    nk_f64_t *result,
    nk_capability_t capabilities,
    nk_stream_t stream);
Symbola, bresultBackends
nk_dot_f32_bestnk_f32_tnk_f64_t9
nk_dot_f64_bestnk_f64_tnk_f64_t9
nk_dot_f16_bestnk_f16_tnk_f32_t12
nk_dot_bf16_bestnk_bf16_tnk_f32_t13
nk_dot_i8_bestnk_i8_tnk_i32_t13
nk_dot_u8_bestnk_u8_tnk_u32_t13
nk_dot_i4_bestnk_i4x2_tnk_i32_t6
nk_dot_u4_bestnk_u4x2_tnk_u32_t6
nk_dot_e4m3_bestnk_e4m3_tnk_f32_t13
nk_dot_e5m2_bestnk_e5m2_tnk_f32_t13
nk_dot_e2m3_bestnk_e2m3_tnk_f32_t11
nk_dot_e2m1_bestnk_e2m1x2_tnk_f32_t8
nk_dot_e3m2_bestnk_e3m2_tnk_f32_t9

Backends

  • nk_dot_f32serialneonsvehaswellskylakervvv128relaxedpowervsxloongsonasx
  • nk_dot_f64serialneonsvehaswellskylakervvv128relaxedpowervsxloongsonasx
  • nk_dot_f16serialneonneonfhmsvehalfhaswellskylakediamondrvvrvvhalfv128relaxedpowervsxloongsonasx
  • nk_dot_bf16serialneonneonbfdotsvebfdothaswellskylakegenoarvvrvvbf16v128v128relaxedpowervsxloongsonasx
  • nk_dot_i8serialneonsdotsvesdothaswellskylakeicelakealdersierrarvvv128v128relaxedpowervsxloongsonasx
  • nk_dot_u8serialneonsdotsvesdothaswellskylakeicelakealdersierrarvvv128v128relaxedpowervsxloongsonasx
  • nk_dot_i4serialneonsdothaswellicelakervvv128relaxed
  • nk_dot_u4serialneonsdothaswellicelakervvv128relaxed
  • nk_dot_e4m3serialneonneonfhmneonbfdotneonfp8haswellskylakeicelakediamondrvvrvvhalfrvvbf16v128relaxed
  • nk_dot_e5m2serialneonneonfhmneonbfdotneonfp8haswellskylakegenoadiamondrvvrvvhalfrvvbf16v128relaxed
  • nk_dot_e2m3serialneonneonsdotneonfp8haswellskylakeicelakealdersierrarvvv128relaxed
  • nk_dot_e2m1serialneonsdothaswellskylakealdersierrarvvv128relaxed
  • nk_dot_e3m2serialneonneonsdotneonfp8haswellskylakeicelakervvv128relaxed

Parameters

ainThe first vector.
binThe second vector.
ninCounts dimensions, a multiple of the values per byte.
resultoutThe output dot product value.
capabilitiesinOne device's capabilities, like nk_cpu_capabilities_enabled reports.
streaminNull on the CPU, or the GPU stream of that device to queue on.

Notes

  • The output value can be negative.
  • The output value is zero if and only if the two vectors are orthogonal.
  • Defined for floating-point, integer, and binary data types.

Statuses

nk_dot_u1_bestDot product of two binary vectors, the number of bit positions both set.
dot.h:199 ↗
nk_status_t nk_dot_u1_best(
    nk_u1x8_t const *a,
    nk_u1x8_t const *b,
    nk_size_t n_bits,
    nk_u32_t *result,
    nk_capability_t capabilities,
    nk_stream_t stream);

Backends

  • nk_dot_u1serialneonhaswellicelakervvrvvbbv128powervsx

Parameters

ainThe first vector.
binThe second vector.
n_bitsinCounts dimensions in bits, a multiple of the bits per byte.
resultoutThe output dot product value.
capabilitiesinOne device's capabilities, like nk_cpu_capabilities_enabled reports.
streaminNull on the CPU, or the GPU stream of that device to queue on.

Statuses

nk_dot_*_best× 4Complex dot product computing the sum of elementwise products between two complex vectors.
dot.h:231 ↗
nk_status_t nk_dot_f32c_best(
    nk_f32c_t const *a_pairs,
    nk_f32c_t const *b_pairs,
    nk_size_t count_pairs,
    nk_f64c_t *result,
    nk_capability_t capabilities,
    nk_stream_t stream);
Symbola_pairs, b_pairsresultBackends
nk_dot_f32c_bestnk_f32c_tnk_f64c_t7
nk_dot_f64c_bestnk_f64c_tnk_f64c_t7
nk_dot_f16c_bestnk_f16c_tnk_f32c_t5
nk_dot_bf16c_bestnk_bf16c_tnk_f32c_t4

Backends

  • nk_dot_f32cserialneonsvehaswellskylakervvv128relaxed
  • nk_dot_f64cserialneonsvehaswellskylakervvv128relaxed
  • nk_dot_f16cserialneonneonfhmsvehalfhaswell
  • nk_dot_bf16cserialneonbfdothaswellgenoa

Parameters

a_pairsinThe first complex vector.
b_pairsinThe second complex vector.
count_pairsinThe number of complex pairs in the vectors.
resultoutThe output complex value as {real, imag}.
capabilitiesinOne device's capabilities, like nk_cpu_capabilities_enabled reports.
streaminNull on the CPU, or the GPU stream of that device to queue on.

Statuses

nk_vdot_*_best× 4Complex conjugate dot product between two complex vectors.
dot.h:256 ↗
nk_status_t nk_vdot_f32c_best(
    nk_f32c_t const *a_pairs,
    nk_f32c_t const *b_pairs,
    nk_size_t count_pairs,
    nk_f64c_t *result,
    nk_capability_t capabilities,
    nk_stream_t stream);
Symbola_pairs, b_pairsresultBackends
nk_vdot_f32c_bestnk_f32c_tnk_f64c_t7
nk_vdot_f64c_bestnk_f64c_tnk_f64c_t7
nk_vdot_f16c_bestnk_f16c_tnk_f32c_t5
nk_vdot_bf16c_bestnk_bf16c_tnk_f32c_t4

Backends

  • nk_vdot_f32cserialneonsvehaswellskylakervvv128relaxed
  • nk_vdot_f64cserialneonsvehaswellskylakervvv128relaxed
  • nk_vdot_f16cserialneonneonfhmsvehalfhaswell
  • nk_vdot_bf16cserialneonbfdothaswellgenoa

Parameters

a_pairsinThe first complex vector.
b_pairsinThe second complex vector.
count_pairsinThe number of complex pairs in the vectors.
resultoutThe output complex value as {real, imag}.
capabilitiesinOne device's capabilities, like nk_cpu_capabilities_enabled reports.
streaminNull on the CPU, or the GPU stream of that device to queue on.

Statuses

nk_dot_output_dtypeReturns the output dtype for dot products.
dot.h:929 ↗
nk_dtype_t nk_dot_output_dtype(nk_dtype_t dtype);
nk_dot_error_boundReturns the error bound of dot products, per nk_accumulation_error_bound of their output.
dot.h:960 ↗
nk_f64_t nk_dot_error_bound(nk_dtype_t dtype);
nk_dot_find_kernelFinds the dot product kernel of kind for dtype, from the best of capabilities.
dot.h:976 ↗
nk_status_t nk_dot_find_kernel(
    nk_kernel_kind_t kind,
    nk_dtype_t dtype,
    nk_capability_t capabilities,
    nk_kernel_punned_t *kernel,
    nk_capability_t *capability);

Parameters

kindinThe verb whose kernel to find.
dtypeinThe input type the kernel reads.
capabilitiesinThe mask to pick the best capability from.
kerneloutThe kernel, or null when none of capabilities has it.
capabilityoutThe capability the kernel belongs to, or zero.

Statuses