Rust · Hashing and Checksums

Hashing and Checksums

use stringzilla::Aes256CtrKey;use stringzilla::Aes256GcmKey;use stringzilla::Aes256GcmEncryptor;use stringzilla::Aes256GcmDecryptor;use stringzilla::Hasher;use stringzilla::Sha256;use stringzilla::BuildSzHasher;use stringzilla::*; 6.0.0-rc.1 · 91a27bdd

These free functions cover a byte checksum, seeded and unseeded 64-bit hashes, and a multi-seed hash that fills an output slice in one pass.

pub fn bytesum<T: AsRef<[u8]>>(text: T) -> u64;
pub fn hash<T: AsRef<[u8]>>(text: T) -> u64;
pub fn hash_with_seed<T: AsRef<[u8]>>(text: T, seed: u64) -> u64;
pub fn hash_multiseed_into<T: AsRef<[u8]>>(text: T, seeds: &[u64], out: &mut [u64]) -> Result<(), Status>;

bytesum is an order-insensitive byte sum; hash / hash_with_seed are order-sensitive AES-based 64-bit hashes. hash_multiseed_into hashes one input under many seeds in a single pass, handy for MinHash, Count-Min sketches, and Bloom/cuckoo filters; it returns Status::UnexpectedDimensions if out.len() != seeds.len():

use stringzilla as sz;

assert_eq!(sz::bytesum("hi"), 209);
assert_ne!(sz::hash("Hello"), sz::hash("World"));
assert_eq!(sz::hash_with_seed("Hello", 42), sz::hash_with_seed("Hello", 42));

let seeds = [1u64, 2, 3, 4];
let mut out = [0u64; 4];
sz::hash_multiseed_into("token", &seeds, &mut out).unwrap();

These are also available as StringZillableUnary methods on any AsRef<[u8]>:

use stringzilla::StringZillableUnary;

assert_eq!(b"Hello".sz_bytesum(), 500);
assert_ne!(b"Hello".sz_hash(), b"World".sz_hash());

§ Incremental Hashing

Hasher is an incremental AES-based hasher implementing core::hash::Hasher:

use stringzilla::Hasher;

let mut hasher = Hasher::new(123);
hasher.update(b"Hello, ").update(b"world!");
assert_eq!(hasher.digest(), Hasher::new(123).update(b"Hello, world!").digest());

Hasher API: new(seed), update(&mut self, &[u8]) -> &mut Self, digest(&self) -> u64, plus the full core::hash::Hasher trait.

With the std feature, BuildSzHasher plugs the AES-based hash into the standard HashMap and HashSet as their hasher, replacing the default SipHash. For string keys this swaps a scalar hash for StringZilla’s vectorized one while keeping the entire collection API unchanged. The seed defaults to 0 and is deterministic across runs, so pass a random seed through with_seed when you want per-process resistance to hash-flooding:

use std::collections::{HashMap, HashSet};
use stringzilla::BuildSzHasher;

let mut counts: HashMap<&str, i32, BuildSzHasher> = HashMap::with_hasher(BuildSzHasher::with_seed(0));
for word in ["apple", "banana", "apple"] { *counts.entry(word).or_insert(0) += 1; }
assert_eq!(counts["apple"], 2);

let mut seen: HashSet<&str, BuildSzHasher> = HashSet::with_hasher(BuildSzHasher::with_seed(42));
seen.insert("apple");
seen.insert("banana");
assert!(seen.contains("apple"));
assert_eq!(seen.len(), 2);

§ SHA-256 and HMAC

hmac_sha256 computes a keyed HMAC-SHA-256 over a message, returning a 32-byte tag. hmac_sha256_multistate authenticates many messages under one key at once — a batch of tokens or webhook bodies sharing a secret — running both the message pass and HMAC’s outer wrap through the lane-parallel kernels. It allocates nothing, so it works under no_std: states is scratch of one state per message, reused for both passes, and only tags is written for keeps.

pub fn hmac_sha256(key: &[u8], message: &[u8]) -> Sha256Digest;
pub fn hmac_sha256_multistate<Element: AsRef<[u8]>>(
    key: &[u8],
    messages: &[Element],
    states: &mut [Sha256],
    tags: &mut [Sha256Digest],
) -> Result<(), Status>;

Sha256 supports one-shot and incremental hashing, returning a 32-byte digest:

use stringzilla::{Sha256, hmac_sha256};

let digest = Sha256::hash(b"Hello, world!");
assert_eq!(digest.len(), 32);

let mut hasher = Sha256::new();
hasher.update(b"Hello, ").update(b"world!");
assert_eq!(hasher.digest(), digest);

let mac = hmac_sha256(b"secret_key", b"important message");
assert_eq!(mac.len(), 32);

Sha256 API: new(), update(&mut self, &[u8]) -> &mut Self, digest(&self) -> Sha256Digest, hash(&[u8]) -> Sha256Digest, where Sha256Digest is [u8; SHA256_DIGEST_LENGTH].

Digesting one message is a serial dependency chain, but independent messages compress in parallel lanes — sixteen at a time on AVX-512, eight on AVX2. sha256_multistate_update advances one hasher per message, taking anything that dereferences to bytes, so a Vec<String> needs no intermediate slice of slices. Use sha256_multistate_update_by(&mut states, &records, |record| record.payload.as_bytes()) when each message is a field of a larger element.

Lanes advance in lockstep within a group, so a group costs as much as its longest message. Every length is handled correctly, but throughput is best when messages of similar length share a group, which argsort arranges.

use stringzilla as sz;

let texts: Vec<String> = vec!["alpha".into(), "beta".into()];
let mut states = vec![sz::Sha256::new(); texts.len()];
let mut digests = vec![[0u8; 32]; texts.len()];

sz::sha256_multistate_update(&mut states, &texts).unwrap();
sz::sha256_multistate_digest(&states, &mut digests).unwrap();
assert_eq!(digests[0], sz::Sha256::hash(b"alpha"));

API

Types

Sha256DigestOne SHA256 digest, as produced by Sha256::digest and sha256_multistate_digest.
hash.rs:34 ↗
pub type Sha256Digest = [u8; 32];
StructAes256CtrKeyExpanded AES-256 round-key schedule for counter mode, the construction named AES-256-CTR.3
cipher.rs:35 ↗
pub struct Aes256CtrKey

Counter mode is unauthenticated and seekable: the keystream at a byte offset depends on nothing but the key, the nonce and that offset, so a caller may decrypt the middle of a file without touching its start. Reusing a nonce under one key exposes the exclusive-or of two plaintexts.

Mirrors sz_aes256_key_t, and carries none of the Galois hash powers counter mode never reads.

Implements

Drop, Debug
newExpands a 32-byte secret into the AES-256 round-key schedule.
cipher.rs:100 ↗
pub fn new(secret: &[u8; 32]) -> Self
xor_intoExclusive-ors text against the keystream starting at byte_offset, writing output.
cipher.rs:118 ↗
pub fn xor_into(
    &self,
    nonce: &[u8; 12],
    byte_offset: u64,
    text: &[u8],
    output: &mut [u8],
) -> Result<(), Status>

Counter mode is its own inverse, so this call both encrypts and decrypts.

Returns Status::UnexpectedDimensions if output is not as long as text.

xor_in_placeExclusive-ors text against the keystream starting at byte_offset, in place.
cipher.rs:145 ↗
pub fn xor_in_place(&self, nonce: &[u8; 12], byte_offset: u64, text: &mut [u8])

Counter mode is its own inverse, so this call both encrypts and decrypts.

StructAes256GcmKeyExpanded AES-256 schedule plus the Galois hash subkey powers, the construction named AES-256-GCM.5
cipher.rs:48 ↗
pub struct Aes256GcmKey

Authentication costs the ability to seek, so this mode transforms whole messages or bounded chunks in order. Reusing a nonce under one key exposes the hash subkey, from which every message under that key can be forged.

Mirrors sz_aes256_gcm_key_t.

Implements

Drop, Debug
newExpands a 32-byte secret into a schedule and the Galois hash subkey powers.
cipher.rs:192 ↗
pub fn new(secret: &[u8; 32]) -> Self
encrypt_intoEncrypts text into output, returning the tag over the ciphertext and associated.
cipher.rs:210 ↗
pub fn encrypt_into(
    &self,
    nonce: &[u8; 12],
    associated: &[u8],
    text: &[u8],
    output: &mut [u8],
) -> Result<[u8; 16], Status>

The associated bytes are authenticated but not encrypted, such as a routing header.

Returns Status::UnexpectedDimensions if output is not as long as text.

encrypt_in_placeEncrypts text in place, returning the tag over the ciphertext and associated.
cipher.rs:240 ↗
pub fn encrypt_in_place(&self, nonce: &[u8; 12], associated: &[u8], text: &mut [u8]) -> [u8; 16]
decrypt_intoVerifies tag and decrypts text into output, which holds the plaintext only on success.
cipher.rs:274 ↗
pub fn decrypt_into(
    &self,
    nonce: &[u8; 12],
    associated: &[u8],
    text: &[u8],
    output: &mut [u8],
    tag: &[u8; 16],
) -> Result<(), Status>

A rejected tag, Status::AuthenticationFailed, leaves output zeroed, so a caller who drops the error still cannot read forged plaintext. The comparison takes the same time wherever the tag first differs.

Returns Status::UnexpectedDimensions, writing nothing, if output is not as long as text.

decrypt_in_placeVerifies tag and decrypts text in place, which holds the plaintext only on success.
cipher.rs:305 ↗
pub fn decrypt_in_place(
    &self,
    nonce: &[u8; 12],
    associated: &[u8],
    text: &mut [u8],
    tag: &[u8; 16],
) -> Result<(), Status>

A rejected tag, Status::AuthenticationFailed, zeroes text, destroying the ciphertext it arrived with.

StructAes256GcmEncryptorSeals a message delivered in chunks under Galois/counter mode, or GCM, accumulating its tag as it goes.5
cipher.rs:81 ↗
pub struct Aes256GcmEncryptor

A chunk boundary is invisible to the result, so any chunking produces the ciphertext and tag a single Aes256GcmKey::encrypt_into would.

Mirrors sz_aes256_gcm_encryptor_t, whose address every call below passes.

Implements

Drop
newBegins a chunked encryption under key and nonce, copying key in so this cannot outlive it.
cipher.rs:399 ↗
pub fn new(key: &Aes256GcmKey, nonce: &[u8; 12]) -> Self
*× 2Absorbs associated data, which is authenticated but not encrypted.
cipher.rs:418 ↗
pub fn associate(&mut self, associated: &[u8]) -> &mut Self

All of it must be absorbed before the first chunk of the message.

encrypt_intoEncrypts one chunk into output and folds its ciphertext into the running tag.
cipher.rs:436 ↗
pub fn encrypt_into(&mut self, text: &[u8], output: &mut [u8]) -> Result<&mut Self, Status>

Returns Status::UnexpectedDimensions, absorbing nothing, if output is not as long as text.

encrypt_in_placeEncrypts one chunk in place and folds its ciphertext into the running tag.
cipher.rs:455 ↗
pub fn encrypt_in_place(&mut self, text: &mut [u8]) -> &mut Self
digestReturns the tag over everything encrypted so far, leaving the encryptor ready for more.
cipher.rs:473 ↗
pub fn digest(&self) -> [u8; 16]
StructAes256GcmDecryptorOpens a message delivered in chunks under Galois/counter mode, or GCM, accumulating its tag as it goes.4
cipher.rs:94 ↗
pub struct Aes256GcmDecryptor

Every emitted byte leaves this type unauthenticated, which is why the methods that emit them say so in their names. Only Aes256GcmDecryptor::verify tells the caller whether they were genuine.

Mirrors sz_aes256_gcm_decryptor_t, whose address every call below passes.

Implements

Drop
newBegins a chunked decryption under key and nonce, copying key in so this cannot outlive it.
cipher.rs:491 ↗
pub fn new(key: &Aes256GcmKey, nonce: &[u8; 12]) -> Self
decrypt_unverified_intoDecrypts one chunk into output and folds its ciphertext into the running tag.
cipher.rs:534 ↗
pub fn decrypt_unverified_into(
    &mut self,
    text: &[u8],
    output: &mut [u8],
) -> Result<&mut Self, Status>

The emitted bytes are not authenticated. Nothing has yet checked that this ciphertext is genuine, so the caller owns that risk: buffer the output until Self::verify succeeds and discard it if it does not. A streaming decryption cannot avoid this, which is why the name says so. Callers who can hold the whole message should use Aes256GcmKey::decrypt_into, which emits nothing until the tag has matched.

Returns Status::UnexpectedDimensions, absorbing nothing, if output is not as long as text.

decrypt_unverified_in_placeDecrypts one chunk in place and folds its ciphertext into the running tag.
cipher.rs:557 ↗
pub fn decrypt_unverified_in_place(&mut self, text: &mut [u8]) -> &mut Self

The emitted bytes are not authenticated, exactly as in Self::decrypt_unverified_into, and in place they also overwrite the ciphertext a failed Self::verify would have to discard.

verifyChecks tag against everything absorbed so far, in time independent of where it differs, returning Status::AuthenticationFailed on a mismatch.
cipher.rs:577 ↗
pub fn verify(&self, tag: &[u8; 16]) -> Result<(), Status>

Until this returns Ok, every byte the decryptor emitted is an attacker's to choose.

StructHasherIncremental hasher state for StringZilla's 64-bit hash.3
hash.rs:19 ↗
pub struct Hasher

Use Hasher::new(seed) to construct, then call update(&mut self, data) zero or more times, and finally call digest(&self) to read the current hash value without consuming the state.

Implements

Debug, Clone, Copy, PartialEq, Default, Hasher
newCreates a new hasher initialized with seed.
hash.rs:72 ↗
pub fn new(seed: u64) -> Self
*× 2Updates the hasher with more data.
hash.rs:93 ↗
pub fn update(&mut self, data: &[u8]) -> &mut Self
digestReturns the current hash value without consuming the state.
hash.rs:108 ↗
pub fn digest(&self) -> u64
StructSha256Incremental SHA256 hasher state for cryptographic hashing.3
hash.rs:64 ↗
pub struct Sha256(_);

Examples

One-shot hashing:

use stringzilla::Sha256;
let digest = Sha256::hash(b"Hello, world!");
assert_eq!(digest.len(), 32); // 256 bits = 32 bytes

Incremental hashing:

use stringzilla::Sha256;
let mut hasher = Sha256::new();
hasher.update(b"Hello, ");
hasher.update(b"world!");
let digest = hasher.digest();
assert_eq!(digest, Sha256::hash(b"Hello, world!"));

Implements

Debug, Clone, Copy, Default
newCreates a new SHA256 hasher with the initial state.
hash.rs:138 ↗
pub fn new() -> Self
digestReturns the current SHA256 digest, leaving the hasher able to accept more data.
hash.rs:167 ↗
pub fn digest(&self) -> Sha256Digest
hashConvenience method to hash data in one call.
hash.rs:182 ↗
pub fn hash(data: &[u8]) -> Sha256Digest
StructBuildSzHasherBuildHasher for constructing Hasher instances, enabling use with HashMap/HashSet.1
hash.rs:505 ↗
pub struct BuildSzHasher

By default uses seed 0 for deterministic hashing across runs and platforms. If you need DOS-resistant randomized seeding, consider wrapping this in your application with a per-process random seed.

Fields

seedu64The seed every built hasher starts from.

Implements

Debug, Clone, Copy, Default, BuildHasher
with_seedBuilds a hasher factory with the given seed.
hash.rs:514 ↗
pub const fn with_seed(seed: u64) -> Self

Functions

sha256_multistate_updateAdvances many independent SHA256 states at once, one message per lane.
hash.rs:228 ↗
pub fn sha256_multistate_update<Element: AsRef<[u8]>>(
    states: &mut [Sha256],
    chunks: &[Element],
) -> Result<(), Status>

Hashing a single message is a serial dependency chain, so no instruction set can speed it up. Independent messages do compress in parallel lanes: sixteen at a time on AVX-512, eight on AVX2. Feed at least a few kilobytes per lane per call, since shorter chunks never reach the lane-parallel path.

Borrows the caller's storage throughout and allocates nothing. Accepts anything that dereferences to bytes, so a Vec<String> or Vec<Vec<u8>> needs no intermediate slice-of-slices. See sha256_multistate_update_by for messages that do not sit in one contiguous slice.

Errors

Returns Status::UnexpectedDimensions if chunks does not have exactly one entry per lane.

Examples

use stringzilla as sz;

let mut states = vec![sz::Sha256::new(); 2];
let heads: Vec<String> = vec!["Hello, ".into(), "Goodbye, ".into()];
let tails: Vec<String> = vec!["world!".into(), "world!".into()];
sz::sha256_multistate_update(&mut states, &heads).unwrap();
sz::sha256_multistate_update(&mut states, &tails).unwrap();

let mut digests = vec![[0u8; 32]; 2];
sz::sha256_multistate_digest(&states, &mut digests).unwrap();
assert_eq!(digests[0], sz::Sha256::hash(b"Hello, world!"));
assert_eq!(digests[1], sz::Sha256::hash(b"Goodbye, world!"));
sha256_multistate_update_byAdvances many independent SHA256 states at once, taking each lane's next chunk as a byte-slice key of the matching element of chunks.
hash.rs:256 ↗
pub fn sha256_multistate_update_by<Element, Key>(
    states: &mut [Sha256],
    chunks: &[Element],
    key: Key,
) -> Result<(), Status>
where
    Key: Fn(&Element) -> &[u8],

key runs inside the C call, where a panic cannot unwind and aborts the process.

Errors

Returns Status::UnexpectedDimensions if chunks does not have exactly one entry per lane.

Examples

use stringzilla as sz;

struct Record { payload: &'static str }
let records = [Record { payload: "alpha" }, Record { payload: "beta" }];

let mut states = vec![sz::Sha256::new(); 2];
sz::sha256_multistate_update_by(&mut states, &records, |record| record.payload.as_bytes()).unwrap();

let mut digests = vec![[0u8; 32]; 2];
sz::sha256_multistate_digest(&states, &mut digests).unwrap();
assert_eq!(digests[0], sz::Sha256::hash(b"alpha"));
sha256_multistate_digestWrites each lane's digest into caller-provided storage, leaving every lane able to accept more data.
hash.rs:287 ↗
pub fn sha256_multistate_digest(
    states: &[Sha256],
    digests: &mut [Sha256Digest],
) -> Result<(), Status>

Errors

Returns Status::UnexpectedDimensions if digests does not have exactly one entry per lane.

hmac_sha256Computes HMAC-SHA256, the Hash-based Message Authentication Code, for the given key and message.
hash.rs:326 ↗
pub fn hmac_sha256(key: &[u8], message: &[u8]) -> Sha256Digest

Returns

A 32-byte HMAC-SHA256 digest

Arguments

  • key - The secret key (can be any length, will be hashed if > 64 bytes)
  • message - The message to authenticate

Examples

use stringzilla::hmac_sha256;
let key = b"secret_key";
let message = b"important message";
let mac = hmac_sha256(key, message);
assert_eq!(mac.len(), 32);
hmac_sha256_multistateAuthenticates many messages under one key at once, one message per lane.
hash.rs:405 ↗
pub fn hmac_sha256_multistate<Element: AsRef<[u8]>>(
    key: &[u8],
    messages: &[Element],
    states: &mut [Sha256],
    tags: &mut [Sha256Digest],
) -> Result<(), Status>

Authenticating one message is a serial dependency chain, but independent messages compress in parallel lanes - sixteen at a time on AVX-512, eight on AVX2 - and both the message pass and HMAC's outer wrap ride those same kernels. Intended for batches of short messages, such as tokens or webhook bodies sharing a secret; there is no streaming form, since the message never spans calls.

Borrows the caller's storage throughout and allocates nothing, so it works under no_std. states is scratch of one state per message, reused for both passes; only tags is written for keeps.

Errors

Returns Status::UnexpectedDimensions unless messages, states and tags all have the same length.

Examples

use stringzilla as sz;

let messages: Vec<String> = vec!["alpha".into(), "beta".into()];
let mut states = vec![sz::Sha256::new(); messages.len()];
let mut tags = vec![[0u8; 32]; messages.len()];

sz::hmac_sha256_multistate(b"secret", &messages, &mut states, &mut tags).unwrap();
assert_eq!(tags[0], sz::hmac_sha256(b"secret", b"alpha"));
bytesumComputes the checksum value of unsigned bytes in a given byte slice text.
hash.rs:540 ↗
pub fn bytesum<Text>(text: Text) -> u64
where
    Text: AsRef<[u8]>,

This function is useful for verifying data integrity and detecting changes in binary data, such as files or network packets.

Returns

A u64 representing the checksum value of the input byte slice.

Arguments

  • text: The byte slice to compute the checksum for.
hash_with_seedComputes a 64-bit AES-based hash value for a given byte slice text.
hash.rs:573 ↗
pub fn hash_with_seed<Text>(text: Text, seed: u64) -> u64
where
    Text: AsRef<[u8]>,

This function is designed to provide a high-quality hash value for use in hash tables, data structures, and cryptographic applications. Unlike the bytesum function, the hash function is order-sensitive.

Returns

A u64 representing the hash value of the input byte slice.

Arguments

  • text: The byte slice to compute the checksum for.
  • seed: A 64-bit value that acts as the seed for the hash function.
hashComputes a 64-bit AES-based hash value for a given byte slice text.
hash.rs:606 ↗
pub fn hash<Text>(text: Text) -> u64
where
    Text: AsRef<[u8]>,

This function is designed to provide a high-quality hash value for use in hash tables, data structures, and cryptographic applications. Unlike the bytesum function, the hash function is order-sensitive.

Returns

A u64 representing the hash value of the input byte slice.

Arguments

  • text: The byte slice to compute the checksum for.
hash_multiseed_intoHashes one byte slice under many seeds at once, writing the results into out.
hash.rs:628 ↗
pub fn hash_multiseed_into<Text>(text: Text, seeds: &[u64], out: &mut [u64]) -> Result<(), Status>
where
    Text: AsRef<[u8]>,

Equivalent to out[i] = hash_with_seed(text, seeds[i]), but normalizes the input into AES blocks once and replays the cheap per-seed rounds - markedly faster for short strings under many seeds (feature hashing, Count-Min sketches, Bloom/cuckoo filters, MinHash/LSH).

Arguments

  • text: The byte slice to hash.
  • seeds: The 64-bit seeds to hash under.
  • out: The output buffer, filled with one hash per seed. Must be the same length as seeds.

Errors

Returns Status::UnexpectedDimensions if out.len() != seeds.len().

Constants

AES256_KEY_LENGTHBytes in an AES-256 secret key.
cipher.rs:13 ↗
pub const AES256_KEY_LENGTH: usize = 32;
AES256_NONCE_LENGTHBytes in the nonce both AES-256 modes accept, the width NIST SP 800-38D recommends.
cipher.rs:16 ↗
pub const AES256_NONCE_LENGTH: usize = 12;
AES256_TAG_LENGTHBytes in a Galois/counter mode, or GCM, authentication tag.
cipher.rs:19 ↗
pub const AES256_TAG_LENGTH: usize = 16;
SHA256_DIGEST_LENGTHBytes in a SHA256 digest, fixed by FIPS 180-4.
hash.rs:28 ↗
pub const SHA256_DIGEST_LENGTH: usize = 32;
SHA256_BLOCK_LENGTHBytes in a SHA256 message block, fixed by FIPS 180-4.
hash.rs:31 ↗
pub const SHA256_BLOCK_LENGTH: usize = 64;