NVIDIA Molt: an RL framework small enough for the agent to read itself
NVIDIA's NeMo team dropped Molt today and it shot to the top of Hugging Face papers with 647 upvotes. It's a PyTorch-native training framework for agentic reinforcement learning, and the whole design philosophy is a reaction against how bloated these stacks have become. The RL core is around nine thousand lines of code, training models up to 700 billion parameters. That number is the point: it's small enough that a researcher can hold the entire algorithm in their head, and small enough that a coding agent can read the whole thing and reason about it end to end.
Under the hood one asynchronous loop trains multimodal and mixture-of-experts policies, with vLLM running over Ray as the rollout engine for fast async serving up to 1T-class MoE scale. The design keeps the RL logic on top small and hackable by pushing the heavy serving down into vLLM. The correctness detail that matters: it never trains on a token it did not generate, staying consistent across tokens, policy versions and model semantics, which is exactly the class of subtle bug that wrecks async RL runs.
The claim that earns the attention is the performance one. Under a matched, fully asynchronous protocol, Molt comes out statistically comparable to a state-of-the-art Megatron-based stack. So you get the same results as the heavyweight framework from a codebase a fraction of the size, where the agent is just an ordinary program you can trace and change.
Why this matters beyond NVIDIA bragging rights: training agents with RL has been gatekept by frameworks so complex that only a handful of labs could actually modify them. A nine-thousand-line core that matches Megatron flips that. It's also a quiet statement about where coding agents are headed, when a hardware giant designs its RL framework so an AI assistant can read the whole thing. Paper and repo: https://github.com/NVIDIA-NeMo/labs-molt
← Back to all articles
Under the hood one asynchronous loop trains multimodal and mixture-of-experts policies, with vLLM running over Ray as the rollout engine for fast async serving up to 1T-class MoE scale. The design keeps the RL logic on top small and hackable by pushing the heavy serving down into vLLM. The correctness detail that matters: it never trains on a token it did not generate, staying consistent across tokens, policy versions and model semantics, which is exactly the class of subtle bug that wrecks async RL runs.
The claim that earns the attention is the performance one. Under a matched, fully asynchronous protocol, Molt comes out statistically comparable to a state-of-the-art Megatron-based stack. So you get the same results as the heavyweight framework from a codebase a fraction of the size, where the agent is just an ordinary program you can trace and change.
Why this matters beyond NVIDIA bragging rights: training agents with RL has been gatekept by frameworks so complex that only a handful of labs could actually modify them. A nine-thousand-line core that matches Megatron flips that. It's also a quiet statement about where coding agents are headed, when a hardware giant designs its RL framework so an AI assistant can read the whole thing. Paper and repo: https://github.com/NVIDIA-NeMo/labs-molt
Comments