01 · What it does
A KV cache compression algorithm.
TurboQuant is a KV cache compression algorithm published by Google. It applies a random orthogonal rotation to stored key-value vectors, exploits the resulting Beta-distributed coordinates to fit a precomputed Lloyd-Max scalar quantizer, and uses a 1-bit residual correction stage. Output: 6x memory reduction, 8x attention logit speedup on H100, zero calibration, zero fine-tuning.
Its operating surface is what is stored between forward passes. It does not read model signals during inference. It does not modify the compute graph. It has no mechanism to route compute based on token difficulty. It is precisely a data representation scheme over a memory buffer.
02 · The stack
Two layers. Two systems.
Neither system modifies them.
Reads attention entropy, logit margin variance, and hidden state norms per token. Modulates compute execution in real time: layer skipping, attention head scaling, FP16 and INT8 precision, sparsity injection.
Compresses stored key-value vectors to 3-bit representations. Acts on data in memory. Does not observe or influence the forward pass compute graph.
Silicon execution substrate.
A transformer running TurboQuant still executes every layer, every attention head, at full depth, for every token. TurboQuant has no mechanism to avoid that waste. PrizmalRun is that mechanism.
03 · Comparison
Memory representation vs runtime control.
| Dimension | TurboQuant | PrizmalRun |
|---|---|---|
| Acts on | Stored KV vectors, data in memory | Compute graph, forward pass execution |
| Timing | At storage, before attention scores | Real time, token by token, mid-inference |
| Signal used | None, data-oblivious by design | Attention entropy, logit variance, hidden norms |
| Optimizes | DRAM footprint, bandwidth | FLOPs and R/E, reasoning quality per joule |
| Compute allocation | Unchanged, every layer runs fully | Dynamic, hard tokens get full depth and easy tokens skip |
| Quality proof | Near-optimal distortion bound | 99% reasoning retention at 50% sparsity, +251% R/E vs dense |
| Composable | Yes, stackable with PrizmalRun | Yes, compatible with TurboQuant |
04 · Investor implications
Composable, not competitive.
TurboQuant does not enter PrizmalRun's operating domain. It cannot adapt to token difficulty. It cannot route compute.
TurboQuant reduces memory bandwidth cost. It does not improve R/E. Reasoning yield per joule is a compute allocation problem, not a compression problem.
Prizmal's provisional patents cover guided blockwise structured pruning, disagreement-aware adaptive pruning, layer-sensitivity pruning caps, and oracle-guided routing policy training. Zero overlap with vector quantization prior art.
A joint deployment addresses both memory-side and compute-side efficiency simultaneously. Additive positioning, not conflict.
KV cache compression is near the ceiling for its problem class. Beyond that point, further compression destroys effective information nonlinearly. The compute allocation problem has no such ceiling. Every token is different, every context is different.
05 · Bottom line
The runtime layer is unsolved.
TurboQuant is a strong result in KV cache memory compression. It is mathematically rigorous, calibration-free, and close to the theoretical optimum for its problem class. It also has a hard boundary: it cannot observe the forward pass, cannot read token difficulty, and cannot modulate compute. Its ceiling is physical. PrizmalRun operates above that ceiling entirely, reading live model signals and modulating compute in real time, token by token.
Reasoning quality per joule.
At 50% sparsity.
FLOPs reduction at inference time.