When Is Enough Not Enough? Illusory Completion in Search Agents
Published in COLM 2026 Workshop, 2026
A correct final answer does not show whether a search agent found evidence for every constraint of a question. We identify illusory completion, where an agent concludes the task is complete while a constraint remains unverified. To evaluate verification where it happens, we introduce the Epistemic Ledger, which tracks evidence and the agent’s stated belief for each constraint along the trajectory. Across 13 agents, from 7B RL-trained models to frontier LLMs, constraints are left assumed, refuted, or unchecked, and while training and scale improve accuracy, they can increase other types of failure.
| Paper (arXiv) | Code |
Recommended citation: Dayoon Ko, Jihyuk Kim, Sohyeon Kim, Haeju Park, Dahyun Lee, Gunhee Kim, Moontae Lee, Kyungjae Lee. (2026). "When Is Enough Not Enough? Illusory Completion in Search Agents." COLM 2026 Workshop.
Download Paper