Ollaya Is Ollama for Models That Answer in 10 Milliseconds
Ollama made it one command to run a chatbot on your laptop. Ollaya, which hit 214 points on Hacker News today, wants the same thing for a different animal: decision models. ollaya run laya, and you have a local server that answers typed questions with calibrated probabilities in a single forward pass. No token generation at all.
The lineup is small models on purpose. Laya comes in 322M and 421M sizes, with a 100-plus language variant. Decider is Qwen-based at 0.75B and 1.9B. There is an NLI zero-shot classifier and GLiClass for multi-option questions. The API is drop-in compatible with TypeSafe's hosted one, the company behind Jev, so code written against Jev can point at localhost instead. Apache 2.0, macOS, Windows, Linux and Docker, NVIDIA acceleration. The headline number is 8 to 10 milliseconds for Laya on an RTX 4090 against 236 to 276 for the hosted route. Repo at github.com/ollaya-dev/ollaya, created two days ago.
Why this matters more than another local runner: this is the restrict-the-output-space idea getting its infrastructure layer. For two weeks we have watched typed-decision models get pulled into routing, tool gating and memory. What was missing was the boring part, a way to run them next to the agent, privately, for free, fast enough to sit inside every loop step. A 10 millisecond yes-or-no with a real probability is cheap enough to ask before every tool call. A 250 millisecond hosted call is not.
The same day brought the best evidence yet for what these calls can do. A paper titled Just Ask Jev (arXiv 2609.29429) benchmarks a calibrated decision model on ten alignment failures, including sycophancy, prompt injection, deception, reward hacking and power seeking, across 44 benchmarks. One generic question reaches a median AUROC of 0.886 zero-shot, beats supervised baselines on most benchmarks, matches the reference scorers' agreement with human labels, and costs 63 times less than LLM-judge scoring. The finding worth underlining: how you word the question barely matters, which fields of the input you show it matters a lot.
Put those together and the shape of a cheap agent guardrail is visible. A local decision model, asking typed questions about every step, at almost no cost. Whether Laya's calibration holds on your data is the thing to test before trusting it. The site is ollaya.dev.
← Back to all articles
The lineup is small models on purpose. Laya comes in 322M and 421M sizes, with a 100-plus language variant. Decider is Qwen-based at 0.75B and 1.9B. There is an NLI zero-shot classifier and GLiClass for multi-option questions. The API is drop-in compatible with TypeSafe's hosted one, the company behind Jev, so code written against Jev can point at localhost instead. Apache 2.0, macOS, Windows, Linux and Docker, NVIDIA acceleration. The headline number is 8 to 10 milliseconds for Laya on an RTX 4090 against 236 to 276 for the hosted route. Repo at github.com/ollaya-dev/ollaya, created two days ago.
Why this matters more than another local runner: this is the restrict-the-output-space idea getting its infrastructure layer. For two weeks we have watched typed-decision models get pulled into routing, tool gating and memory. What was missing was the boring part, a way to run them next to the agent, privately, for free, fast enough to sit inside every loop step. A 10 millisecond yes-or-no with a real probability is cheap enough to ask before every tool call. A 250 millisecond hosted call is not.
The same day brought the best evidence yet for what these calls can do. A paper titled Just Ask Jev (arXiv 2609.29429) benchmarks a calibrated decision model on ten alignment failures, including sycophancy, prompt injection, deception, reward hacking and power seeking, across 44 benchmarks. One generic question reaches a median AUROC of 0.886 zero-shot, beats supervised baselines on most benchmarks, matches the reference scorers' agreement with human labels, and costs 63 times less than LLM-judge scoring. The finding worth underlining: how you word the question barely matters, which fields of the input you show it matters a lot.
Put those together and the shape of a cheap agent guardrail is visible. A local decision model, asking typed questions about every step, at almost no cost. Whether Laya's calibration holds on your data is the thing to test before trusting it. The site is ollaya.dev.
Comments