LIDAR: You Can Tell Which Model Is Inside a Coding Agent by How It Fixes a Failing Test
Whose model is actually behind this agent? It stopped being an academic question last week, when a blog post argued that Meta's Muse was quietly running an OpenAI model labeled muse-special. A paper posted September 23 by Chuyi Wang, Yong Cui and colleagues gives that kind of accusation a method.
Classic model fingerprinting looks at text and token distributions. Inside a coding agent those signals get washed out by the system prompt, the controller logic, the tools and the execution feedback. LIDAR, short for LLM Identification from Decisions and Actions at Runtime, looks at behavior instead. It runs three pairs of coding probes that expose how a model handles post-edit verification, recovery from transient failures, and conflicts between the spec and the tests, each under controlled variations. Then it compares the resulting trajectories against clean references with a lightweight probabilistic identifier. No weights, no logits, no provider access needed.
Across 36 models from seven families and two agent harnesses, it gets high top-1 accuracy and beats four existing fingerprinting and API-auditing baselines. Ablations say every probe pair and both feature levels pull their weight.
Two uses follow immediately. Buyers can check that the premium model they pay for is the one running, since silently swapping to a cheaper model changes security-relevant choices like whether the agent verifies its edits. And vendor-accusation stories get a second method beyond reverse-engineering log formats. That is exactly what the muse-special claim is missing: an independent confirmation. Paper: arxiv.org/abs/2609.28559.
← Back to all articles
Classic model fingerprinting looks at text and token distributions. Inside a coding agent those signals get washed out by the system prompt, the controller logic, the tools and the execution feedback. LIDAR, short for LLM Identification from Decisions and Actions at Runtime, looks at behavior instead. It runs three pairs of coding probes that expose how a model handles post-edit verification, recovery from transient failures, and conflicts between the spec and the tests, each under controlled variations. Then it compares the resulting trajectories against clean references with a lightweight probabilistic identifier. No weights, no logits, no provider access needed.
Across 36 models from seven families and two agent harnesses, it gets high top-1 accuracy and beats four existing fingerprinting and API-auditing baselines. Ablations say every probe pair and both feature levels pull their weight.
Two uses follow immediately. Buyers can check that the premium model they pay for is the one running, since silently swapping to a cheaper model changes security-relevant choices like whether the agent verifies its edits. And vendor-accusation stories get a second method beyond reverse-engineering log formats. That is exactly what the muse-special claim is missing: an independent confirmation. Paper: arxiv.org/abs/2609.28559.
Comments