Python · Runtime Dispatch

Runtime Dispatch

import numkong 8.0.0-rc.1 · b1f866ab

API

Types

CapabilityCPU SIMD capabilities, one bit each, as the capabilities_* functions report and set them.
__init__.pyi:689 ↗
class Capability(IntFlag)

Values

SERIALSerial code without SIMD, available on every CPU.
NEONArm NEON, the AArch64 Advanced SIMD baseline.
NEONHALFNEON with half-precision arithmetic, FEAT_FP16.
NEONBFDOTNEON with BFloat16 dot products, FEAT_BF16.
NEONFHMNEON with half-precision multiply-accumulate into single precision, FEAT_FHM.
NEONSDOTNEON with 8-bit integer dot products, FEAT_DotProd.
NEONFP8NEON with FP8 dot products, FEAT_FP8DOT4.
SVEArm Scalable Vector Extension.
SVEHALFSVE with half-precision arithmetic.
SVESDOTSVE with 8-bit integer matrix multiplies, FEAT_I8MM.
SVEBFDOTSVE with BFloat16 dot products.
SVE2Arm SVE2.
SMEArm Scalable Matrix Extension.
SMEF64SME with double-precision outer products, FEAT_SME_F64F64.
SMEBI32SME with binary outer products into 32-bit integers, SME_BI32I32.
HASWELLx86 AVX2, FMA, F16C, BMI1 and BMI2, as on Intel Haswell.
ALDERHASWELL plus AVX-VNNI, as on Intel Alder Lake.
SIERRAALDER plus AVX-VNNI-INT8, as on Intel Sierra Forest.
SKYLAKEHASWELL plus AVX-512 F, DQ, BW and VL, as on Intel Skylake-X.
ICELAKESKYLAKE plus AVX-512 VNNI, IFMA, BITALG, VBMI, VBMI2 and VPOPCNTDQ, as on Ice Lake.
GENOASKYLAKE plus AVX-512 BF16 and VNNI, as on AMD Genoa.
TURINGENOA plus AVX-512 VBMI2 and VP2INTERSECT, as on AMD Turin.
SAPPHIRESKYLAKE plus AVX-512 FP16, as on Intel Sapphire Rapids.
DIAMONDSAPPHIRE plus AVX10.2, as on Intel Diamond Rapids.
SAPPHIREAMXSAPPHIRE and GENOA plus AMX tiles with BF16 and INT8, as on Intel Sapphire Rapids.
GRANITEAMXSAPPHIREAMX plus AMX-FP16, as on Intel Granite Rapids.
DIAMONDAMXSAPPHIREAMX plus AMX-FP8 and AMX-AVX512, as on Intel Diamond Rapids.
RVVRISC-V Vector extension.
RVVBF16RVV with BFloat16 widening multiply-accumulate, Zvfbfwma.
RVVHALFRVV with half-precision arithmetic, Zvfh.
RVVBBRVV with basic bit manipulation, Zvbb.
V128WebAssembly 128-bit SIMD.
V128RELAXEDWebAssembly Relaxed SIMD.
POWERVSXIBM Power VSX at ISA 3.0, as on POWER9.
LOONGSONASXLoongArch LASX 256-bit SIMD.

Functions

cpu_capabilities_detectedWhat this CPU can execute.
__init__.pyi:772 ↗
def cpu_capabilities_detected() -> Capability
cpu_capabilities_compiledWhat this binary holds CPU kernels for.
__init__.pyi:776 ↗
def cpu_capabilities_compiled() -> Capability
cpu_capabilities_enabledWhat CPU kernels run within: both of the above, settled as the library loads.
__init__.pyi:780 ↗
def cpu_capabilities_enabled() -> Capability
cpu_configure_threadPrepare the calling thread for the CPU kernels of capabilities.
__init__.pyi:784 ↗
def cpu_configure_thread(capabilities: Capability, /) -> None
cuda_count_devicesHow many CUDA devices this process sees, zero without the runtime or its kernels.
__init__.pyi:788 ↗
def cuda_count_devices() -> int
cuda_capabilities_detectedWhat CUDA device ordinal can execute, raising ValueError past the last one.
__init__.pyi:792 ↗
def cuda_capabilities_detected(ordinal: int, /) -> Capability
cuda_capabilities_compiledWhat this binary holds CUDA kernels for.
__init__.pyi:796 ↗
def cuda_capabilities_compiled() -> Capability
cuda_capabilities_enabledThe mask CUDA device ordinal dispatches with, raising ValueError past the last one.
__init__.pyi:800 ↗
def cuda_capabilities_enabled(ordinal: int, /) -> Capability
stream_init_cudaA new cudaStream_t on CUDA device ordinal as an integer, raising ValueError past the last one.
__init__.pyi:804 ↗
def stream_init_cuda(ordinal: int, /) -> int
stream_free_cudaFree a stream stream_init_cuda made, once synchronize joined its work.
__init__.pyi:808 ↗
def stream_free_cuda(stream: int, /) -> None
rocm_count_devicesHow many ROCm devices this process sees, zero without the runtime or its kernels.
__init__.pyi:812 ↗
def rocm_count_devices() -> int
rocm_capabilities_detectedWhat ROCm device ordinal can execute, raising ValueError past the last one.
__init__.pyi:816 ↗
def rocm_capabilities_detected(ordinal: int, /) -> Capability
rocm_capabilities_compiledWhat this binary holds ROCm kernels for.
__init__.pyi:820 ↗
def rocm_capabilities_compiled() -> Capability
rocm_capabilities_enabledThe mask ROCm device ordinal dispatches with, raising ValueError past the last one.
__init__.pyi:824 ↗
def rocm_capabilities_enabled(ordinal: int, /) -> Capability
stream_init_rocmA new hipStream_t on ROCm device ordinal as an integer, raising ValueError past the last one.
__init__.pyi:828 ↗
def stream_init_rocm(ordinal: int, /) -> int
stream_free_rocmFree a stream stream_init_rocm made, once synchronize joined its work.
__init__.pyi:832 ↗
def stream_free_rocm(stream: int, /) -> None
metal_count_devicesHow many Metal devices this process sees, zero without the runtime or its kernels.
__init__.pyi:836 ↗
def metal_count_devices() -> int
metal_capabilities_detectedWhat Metal device ordinal can execute, raising ValueError past the last one.
__init__.pyi:840 ↗
def metal_capabilities_detected(ordinal: int, /) -> Capability
metal_capabilities_compiledWhat this binary holds Metal kernels for.
__init__.pyi:844 ↗
def metal_capabilities_compiled() -> Capability
metal_capabilities_enabledThe mask Metal device ordinal dispatches with, raising ValueError past the last one.
__init__.pyi:848 ↗
def metal_capabilities_enabled(ordinal: int, /) -> Capability
stream_init_metalA new id<MTLCommandQueue> on Metal device ordinal as an integer, raising ValueError past the last one.
__init__.pyi:852 ↗
def stream_init_metal(ordinal: int, /) -> int
stream_free_metalFree a stream stream_init_metal made, once synchronize joined its work.
__init__.pyi:856 ↗
def stream_free_metal(stream: int, /) -> None
synchronizeWait for everything queued on stream of the device capabilities describes, None for its default.
__init__.pyi:860 ↗
def synchronize(
    capabilities: Capability | int,
    /,
    stream: int | None = None,
) -> None