BLOOM-WILT: Logit Tilting for Behaviour Elicitation in Automated LLM Auditing

AI Digest - ArXiv AI

BLOOM-WILT: Logit Tilting for Behaviour Elicitation in Automated LLM Auditing

Users of a deployed language model routinely encounter behaviours that testing almost never surfaces, since deployment puts the model through orders of magnitude more interactions than any evaluation can simulate. Automated auditors make testing cheap to scale and flexible enough to cover almost any specified behaviour, yet their lack of optimisation pressure makes them sample-inefficient. To address this shortcoming, we introduce BLOOM-WILT, a full auditing pipeline that elicits natural multi-t

Source: ArXiv AI