November 22, 2026Open SourceTool

LookAtMyQuant Fits Inkling 2 Mini in 5.4 GB, a Day After Release

Thinking Machines shipped Inkling 2 Mini on Friday. By Saturday a single developer had it running in 5.4 GB. LookAtMyQuant, an MIT-licensed "quantization compiler," compressed the model's 18.89 GB of text weights to 5.38 GB, 1.63 bits per weight, and hit 266 points on Hacker News. Repo: https://github.com/ShinMegamiBoson/lookatmyquant , weights: https://huggingface.co/shinmegamiboson/Inkling-2-Mini-LookAtMyQuant-5.4GB

The idea in one breath: measure how much each stored piece of the model hurts the output when compressed, then pick a code for each piece so the whole thing fits a size budget with the least damage. Uniform 2-bit everywhere is the dumb way. Spend bits where the model is sensitive, starve the parts that don't care. Inkling 2 Mini makes this harder than usual: 26.4B physical parameters, 5.4B active, a mixture of experts (128, top-8) whose middle layers loop three times, so a bad weight gets hit three times per token.

The numbers, against the BF16 original. GSM8K on 200 problems: original 147, the 5.4 GB file 134, Unsloth's 7.76 GB UD-IQ1_S 127, Unsloth's 9.14 GB UD-Q2_K_XL 132. HumanEval on 164: 66, 60, 46, 49. So the smallest file beats Unsloth files up to 41% bigger. The 8.9 GB version matches the original on GSM8K, 148. The vision tower is left untouched at 637 MB. Same recipe earlier took Qwen3.8-27B from 53.7 GB to 7.2 GB, solving 401 of 540 math problems, as many as the original. The recipe costs about $34 of GPU time.

Now the honest part. These are one maintainer's own runs, on a 200-problem GSM8K slice and HumanEval, with greedy decoding and reasoning off. That is a smoke test, not an eval suite, and nobody has reproduced it yet. And check the bottom of the table: Unsloth's 11.66 GB UD-IQ3_XXS scores 148 and 68, slightly above the original. If you have the RAM, plain 3-bit still wins. The runtime is Apple Silicon only (lamq, on MLX).

Still, look at the speed. A looped open-weight agent model goes from release to a 5 GB laptop file in about a day, with a public recipe. The gap between "the lab released weights" and "it runs on your MacBook" used to be weeks. Now it is one weekend, and the people closing it are individuals, not labs.
← Previous
China Wants to Standardize the Token, Like a Kilowatt-Hour
Next β†’
ClawDOS: A Whole Coding Agent in 176 KB, on a 1983 PC
← Back to all articles

Comments

Loading...
>_