Technical report · August 2026
Behaviour-Relearned
Quantization
Recovering one-bit mixture-of-experts weights by reserving structure and relearning expert codes.
BRQ is the main under-2-bit paper, but not a release victory lap.
The paper shows why one-bit MoE compression has to move from static range selection to recovery training. On OLMoE, binary expert weights were nearly dead at the floor and then recovered substantial teacher-forced structure under hard-forward STE-KD.
The same receipts also block overclaiming. The saved generations still looped or fragmented, V1a did not produce a packed artifact, and the run did not meet the 80% behavior-promotion gate.
BRQ supports the research lane: reserve MoE structure and relearn binary expert codes. It does not claim a released one-bit model, 90% behavior retention, or reproduction of Bonsai's proprietary transformation.
The paper is built around receipts.
It includes the OLMoE V0 and V1a curves, Qwen dense controls, free-generation failure analysis, and the minimum V1b receipt schema required before scale-up.
- 01
Reserve structure
Keep the MoE spine stable by rule: routing, attention, embeddings, norms, and interfaces are not where the one-bit risk is priced.
- 02
Binarize experts
Store routed expert weights as group-128 signs plus scales, while keeping floating latents only during recovery training.
- 03
Relearn the codes
Train through the hard binary forward pass with windowed STE-KD instead of treating one-bit compression as nearest rounding.
- 04
Read generations
Teacher-forced top-1 recovered from 3.81% to 49.38%, but free-running outputs remained degenerate, so the result did not promote.
- 05
Gate before scale
V1b must prove behavior, receipts, and a packed artifact before any 35B or K3-class spend is justified.
Read the paper
The BRQ report is the detailed version.
This is the artifact that separates BRQ from RBR: RBR explains the stock-format two-bit BTL-4 release; BRQ explains the one-bit recovery lane and why it has not promoted yet.