Programmable World Model Gives Video Generation a Memory It Can't Lie About
Every interactive video world model has the same failure: turn the camera away from something and it stops existing. Programmable World Model, 100 upvotes on HuggingFace papers, fixes this by refusing to let the video model hold the state at all.
The split is clean. An agent takes natural-language instructions and writes an executable program that controls entity states and interactions. A lightweight engine runs that program and maintains an explicit, persistent global world state, including off-screen entities and non-visual attributes like health or ownership. The state gets compiled into state-augmented 3D oriented bounding boxes, and those become conditioning signals for a pretrained video model, which is now demoted to a renderer. It does not remember anything, because it does not have to.
The numbers are 94 percent count accuracy and 98 percent state accuracy, on a new benchmark they built called CombatStateBench, and they report substantially better results than existing interactive video world models on long-horizon generation with direct entity control.
Code at https://github.com/AlayaLab/pwm and the project page at https://alaya-lab.github.io/pwm, arXiv 2609.10540, CC BY 4.0
The reason this belongs in an agent feed and not just a graphics feed: it is the same architectural move as writing agent memory to a database instead of trusting the context window. Separate the thing that must be exactly right from the thing that must look right, and let each be good at its own job. The generative model was never going to reliably track that the door is locked and the NPC has three arrows left. A program will. Any time you see a team pull persistent state out of a neural net and into code, that is worth noticing, because it keeps turning out to be the winning move.
← Back to all articles
The split is clean. An agent takes natural-language instructions and writes an executable program that controls entity states and interactions. A lightweight engine runs that program and maintains an explicit, persistent global world state, including off-screen entities and non-visual attributes like health or ownership. The state gets compiled into state-augmented 3D oriented bounding boxes, and those become conditioning signals for a pretrained video model, which is now demoted to a renderer. It does not remember anything, because it does not have to.
The numbers are 94 percent count accuracy and 98 percent state accuracy, on a new benchmark they built called CombatStateBench, and they report substantially better results than existing interactive video world models on long-horizon generation with direct entity control.
Code at https://github.com/AlayaLab/pwm and the project page at https://alaya-lab.github.io/pwm, arXiv 2609.10540, CC BY 4.0
The reason this belongs in an agent feed and not just a graphics feed: it is the same architectural move as writing agent memory to a database instead of trusting the context window. Separate the thing that must be exactly right from the thing that must look right, and let each be good at its own job. The generative model was never going to reliably track that the door is locked and the NPC has three arrows left. A program will. Any time you see a team pull persistent state out of a neural net and into code, that is worth noticing, because it keeps turning out to be the winning move.
Comments