Prompts Are Not Contracts: Hard Numbers for the Harness
The harness-beats-prompting argument finally has clean numbers. From Prompts to Contracts, a paper posted July 9 by Joongho Ahn and Moonsoo Kim, takes a prototype enterprise LLM agent to production by moving behavior out of prompts and into code: schemas, validators, and what they call contracts — source-grounding, entity-routing, trace, output-hygiene, recommendation-language.
The head-to-head is the interesting part. Prompt-only enforcement let real safety violations reach users. External guardrails caught the violations but over-refused, dropping utility to 88 out of 120. The code-owned harness went 120 for 120 with zero violations getting through — fault-injection tests confirmed the validators actually catch deliberate sabotage, and the whole thing held across three different hosted models over 270 boundary-crossing runs.
Read that as a ranking: prompts are wishes, guardrails are a tax, contracts are engineering. The model can be swapped underneath and the guarantees survive, because they never lived in the model in the first place.
This is the enterprise chapter of a thread that has been building all year — harness optimization papers keep showing the scaffold matters more than the weights, and Anthropic open-sourced its own security harness a month ago. There is a reference implementation at github.com/hammerbaki/enterprise-llm-agent-harness if you want to steal the pattern.
https://arxiv.org/abs/2607.08028
← Back to all articles
The head-to-head is the interesting part. Prompt-only enforcement let real safety violations reach users. External guardrails caught the violations but over-refused, dropping utility to 88 out of 120. The code-owned harness went 120 for 120 with zero violations getting through — fault-injection tests confirmed the validators actually catch deliberate sabotage, and the whole thing held across three different hosted models over 270 boundary-crossing runs.
Read that as a ranking: prompts are wishes, guardrails are a tax, contracts are engineering. The model can be swapped underneath and the guarantees survive, because they never lived in the model in the first place.
This is the enterprise chapter of a thread that has been building all year — harness optimization papers keep showing the scaffold matters more than the weights, and Anthropic open-sourced its own security harness a month ago. There is a reference implementation at github.com/hammerbaki/enterprise-llm-agent-harness if you want to steal the pattern.
https://arxiv.org/abs/2607.08028
Comments