Language models thatrun on select-accumulate
A ternary inference stack with one weight format, consumed byte-for-byte by CPU SIMD today and multiplier-free silicon next.
No multiplier. Two bits per weight.
Every weight is −1, 0 or +1, so the dominant operation of transformer inference reduces to add, subtract, or skip. 42.19% of weights are exactly zero — measured with tritc verify.
No multiplier. Two bits per weight. 42.19% of weights are exactly zero — measured, tritc verify.
AMD Ryzen 9 8945HX, BitNet b1.58 2B4T, greedy, 4 threads. Reproducible with tritd bench; frozen reference: docs/06-GATES.md. Quality vs the HF reference: mean logit cosine 0.9991, top-1 100%.
The bottleneck is the multiplier.
4-bit and 8-bit quantization on edge companion computers — drones, robots, edge IPCs — bottlenecks on the memory bus and dissipates heat doing continuous floating-point matrix multiplication. BitNet-style 1.58-bit models put every weight in {-1, 0, +1}, which turns the inner loop from multiply-accumulate into select-accumulate.
No multiplier. Two bits per weight on the wire. Tritium is the runtime that cashes that check on hardware people already own, and keeps a synthesizable RTL path open for hardware they don't yet.
| Metric | Before the pivot | Now |
|---|---|---|
| Decode | 0.16 tok/s | 27.9 tok/s |
| Time to first token | ~33 s | 239 ms |
| Peak RSS | 6.10 GiB | 1.19 GiB |
| Bytes per token | 1834 MB | 1178 MB |
| Achieved bandwidth | ~0.3 GB/s | 32.9 GB/s |
A memory-mapped .trit is the beat stream.
Ternary weights are stored as bit planes: each 16-byte beat carries 64 columns of one row as {u64 pos, u64 neg}. Both masks for a column block arrive together — which is what the SIMD kernel wants and what the RTL core consumes directly. One beat is exactly one hardware cycle's weight input; nothing between the page cache and the accumulator rewrites a byte.
Three implementations, one arithmetic.
The oracle, the vectorized runtime, and the hardware must agree exactly — integer accumulators, no tolerance. Lane order, thread count and the RTL's beat-serial accumulation all produce identical bits.
| Crate | What it is | Status |
|---|---|---|
| crates/tritc | Converter: HF BitNet checkpoint to packed .trit v1. Folds norms, quantizes, verifies. | working |
| crates/trit-core | Format, transformer, KV cache, RoPE, sampler, tokenizer traits. No unsafe outside one mmap. | working |
| crates/trit-cpu | Bit-sliced ternary SIMD kernels, dense bf16/f32 kernels, and the worker pool they share. | verified on x86 and aarch64 in CI |
| crates/tritd | Host daemon and C runtime: run, serve, bench, info, plus a C ABI. | working |
| crates/trit-rtl | Hardware-in-the-loop backend over the Verilated core. | working |
| crates/tritsim | Independent golden reference. The oracle every other path is diffed against. | working |
| rtl/ | Multiplier-free SystemVerilog tritcore + Verilator testbenches. | 64 lanes, simulation-first |
| benches/ | Comparative harness vs llama.cpp / bitnet.cpp baselines. | both baselines measured |
Tritium leads at 1, 2 and 4 threads.
1.43x faster than bitnet.cpp per core on the identical checkpoint. A persistent worker pool replaced per-matvec fork/join, and the converter stopped widening the checkpoint's bf16 embeddings to f32 — 656 MB fewer bytes per token at no cost in accuracy.
Above four threads it loses: the curve peaks and declines where bitnet.cpp keeps climbing to 32.49 at eight. Closing that is the open work.
| threads | tritium | llama.cpp Q4_K_M | bitnet.cpp I2_S |
|---|---|---|---|
| 1 | 19.40 | 15.40 | 13.61 |
| 2 | 26.73 | 22.06 | 19.10 |
| 4 | 27.90 | 27.83 | 26.72 |
| 8 | 25.93 | 26.18 | 32.49 |
| 16 | 20.72 | 23.88 | 31.21 |
What it does not do.
- Batch size 1. No batched prefill, no continuous batching.
- Context capped at 2048; the KV cache is f32 and preallocated (315 MB).
- Performance is measured on x86-64 only. The NEON kernels match the reference exactly on real aarch64 CI hardware, but no ARM timing has been taken and no ARM performance number is claimed.
- No FPGA silicon yet. The RTL is simulation-first: fine under Verilator, not yet timing-closed on a board.
- Scaling stops at four threads. Decode peaks there and declines, where bitnet.cpp climbs to eight. Fixed in part: a persistent worker pool lifted one-to-eight scaling from 1.15x to 1.34x, but bitnet.cpp still reaches 2.39x.
- Energy per token is reported only where a real counter exists. It is never estimated.
Misses get published rather than buried. This list shrinks by measurement, not by rewording.
The primitive was proved on an FPGA.
Tritium is the direct continuation of ternoise — ternary compute on FPGA, built in public. The denoiser proved the primitive; tritium scales the same primitive (ternary MAC to add/sub tree) to transformer inference, and now runs it on commodity silicon as well.