Python · Set Similarity

Set Similarity

import numkong 8.0.0-rc.1 · b1f866ab

Packed-binary metrics operate on packed bits. That is why the right NumPy equivalent uses np.packbits, not bool arrays fed to scalar Python code. A uint1 Tensor counts bits in its shape, while a raw byte array passed with dtype="uint1" holds 8 bits per byte.

import numpy as np
import numkong as nk

a_bits = np.random.randint(0, 2, size=256, dtype=np.uint8)
b_bits = np.random.randint(0, 2, size=256, dtype=np.uint8)
a, b = np.packbits(a_bits), np.packbits(b_bits)  # 32 bytes each

hamming = nk.hamming(a, b, dtype="uint1")
jaccard = nk.jaccard(a, b, dtype="uint1")

a_tensor = nk.astype(a_bits, "uint1")  # shape (256,), 32 bytes of storage

Integer set Jaccard works on sorted ascending arrays of integer identifiers. Both inputs must be sorted in ascending order for correct results.

set_a = np.array([1, 3, 5, 7, 9], dtype=np.uint32)  # must be sorted ascending
set_b = np.array([3, 5, 8, 9, 10], dtype=np.uint32)  # must be sorted ascending
jaccard_sets = nk.jaccard(set_a, set_b) # |A ∩ B| / |A ∪ B|
assert 0.0 < jaccard_sets < 1.0, "|A ∩ B| / |A ∪ B| should be in (0, 1)"

API

Functions

hammingCompute the Hamming distance between two binary vectors.
__init__.pyi:1097 ↗
def hamming(
    a: _BufferType,
    b: _BufferType,
    /,
    dtype: Literal['uint1', 'int4', 'uint4', 'int64', 'int32', 'int16', 'int8', 'uint64', 'uint32', 'uint16', 'uint8'] | _MiniFloatType | None = None,
    *,
    out: _BufferType | None = None,
    out_dtype: _FloatTypeName | _MiniFloatType | None = None,
    capabilities: Capability | int | None = ...,
    stream: int | None = ...,
) -> float | Tensor | None

Similar to scipy.spatial.distance.hamming. Two vectors give a scalar. Matrices pair row i of a with row i of b, a single row pairing with every row of the other, into a new Tensor of one result per pair, or into out, returning None.

Parameters

a_BufferTypeFirst binary vector or matrix.
b_BufferTypeSecond binary vector or matrix, of the dtype of a.
dtypeLiteral['uint1', 'int4', 'uint4', 'int64', 'int32', 'int16', 'int8', 'uint64', 'uint32', 'uint16', 'uint8'] | _MiniFloatType | NoneInput dtype to read both buffers as, overriding the one they report.
out_BufferType | NoneOutput vector for the results.
out_dtype_FloatTypeName | _MiniFloatType | NoneResult dtype, by default that of out, else float64.
capabilitiesCapability | int | NoneCapabilities to run within: every CPU one by default, or for packed operands the mask that packed them.
streamint | NoneA GPU stream pointer, left None on the CPU.
jaccardCompute the Jaccard distance, the bitwise Tanimoto metric, between two binary vectors.
__init__.pyi:1122 ↗
def jaccard(
    a: _BufferType,
    b: _BufferType,
    /,
    dtype: Literal['uint1', 'int4', 'uint4', 'int64', 'int32', 'int16', 'int8', 'uint64', 'uint32', 'uint16', 'uint8'] | _MiniFloatType | None = None,
    *,
    out: _BufferType | None = None,
    out_dtype: _FloatTypeName | _MiniFloatType | None = None,
    capabilities: Capability | int | None = ...,
    stream: int | None = ...,
) -> float | Tensor | None

Similar to scipy.spatial.distance.jaccard. Two vectors give a scalar. Matrices pair row i of a with row i of b, a single row pairing with every row of the other, into a new Tensor of one result per pair, or into out, returning None.

Parameters

a_BufferTypeFirst binary vector or matrix.
b_BufferTypeSecond binary vector or matrix, of the dtype of a.
dtypeLiteral['uint1', 'int4', 'uint4', 'int64', 'int32', 'int16', 'int8', 'uint64', 'uint32', 'uint16', 'uint8'] | _MiniFloatType | NoneInput dtype to read both buffers as, overriding the one they report.
out_BufferType | NoneOutput vector for the results.
out_dtype_FloatTypeName | _MiniFloatType | NoneResult dtype, by default that of out, else float64.
capabilitiesCapability | int | NoneCapabilities to run within: every CPU one by default, or for packed operands the mask that packed them.
streamint | NoneA GPU stream pointer, left None on the CPU.