EpiSEM-Bench: Evaluating Safety and Evidence Recovery in LLM Agents under Missing Evidence
AI Digest - ArXiv AI
EpiSEM-Bench: Evaluating Safety and Evidence Recovery in LLM Agents under Missing Evidence
Tool-using language model agents rely on environmental facts to decide whether an action is permitted. When a tool response omits a safety-relevant fact, an agent may carry out a prohibited action even though the user request is benign and every returned statement is true. We introduce \bench{}, an executable benchmark for evaluating unsafe execution, safety boundaries, and evidence recovery in LLM agents under missing evidence. It contains 300 matched unsafe/safe task pairs across six domains.
Source: ArXiv AI