Nanocode: Build Your Own Coding Agent for $200
What does it cost to train a coding agent from scratch? If you have a TPU, about $200.
Nanocode by Salman Mohammadi is the full recipe. Not a fine-tune, not a LoRA adapter. The entire pipeline: train a tokenizer, pretrain on code and text, generate synthetic data with constitutional critique, do agentic supervised fine-tuning with tool use, then align with DPO. Everything in pure JAX, everything on TPU. The 1.3B parameter model takes 9.3 hours on a TPU v6e-8 pod. The 477M version costs $34. The 135M version costs $3 and finishes in 9 minutes.
The project includes datasets on HuggingFace, speedrun training scripts, and evaluation on the CORE benchmark. It is not going to replace Claude or Codex. That is not the point. The point is that the entire knowledge required to build a coding agent, from tokenizer to alignment, is now compressed into one readable repository with a single-digit dollar cost to reproduce.
A year ago, training a coding model was a black box guarded by frontier labs. Today a solo developer publishes the complete blueprint and it lands on the Hacker News front page with 119 points. The ceiling for what individuals can build keeps rising. The floor for what it costs keeps dropping.
https://github.com/salmanmohammadi/nanocode
← Back to all articles
Nanocode by Salman Mohammadi is the full recipe. Not a fine-tune, not a LoRA adapter. The entire pipeline: train a tokenizer, pretrain on code and text, generate synthetic data with constitutional critique, do agentic supervised fine-tuning with tool use, then align with DPO. Everything in pure JAX, everything on TPU. The 1.3B parameter model takes 9.3 hours on a TPU v6e-8 pod. The 477M version costs $34. The 135M version costs $3 and finishes in 9 minutes.
The project includes datasets on HuggingFace, speedrun training scripts, and evaluation on the CORE benchmark. It is not going to replace Claude or Codex. That is not the point. The point is that the entire knowledge required to build a coding agent, from tokenizer to alignment, is now compressed into one readable repository with a single-digit dollar cost to reproduce.
A year ago, training a coding model was a black box guarded by frontier labs. Today a solo developer publishes the complete blueprint and it lands on the Hacker News front page with 119 points. The ceiling for what individuals can build keeps rising. The floor for what it costs keeps dropping.
https://github.com/salmanmohammadi/nanocode
Comments