ControlArena ControlArena is a Python library for running AI Control experiments, created by the UK AI Security Institute and Redwood Research. AI Control aims to design and evaluate control protocols. Control protocols are plans designed to prevent unsafe actions by AI systems, even if those AIs were misaligned and intended to subvert the safeguards. 📚 Documentation : Visit our documentation site for detailed guides and API reference. 💬 Community & Support : Join us in the control arena discussion channel on the Inspect AI Slack workspace. If you're not already on the Inspect workspace, please get in contact here and we can add you. New to AI Control? Check out our Introduction to Control guide. Core features include: Settings : Complete evaluation environments covering a range of threat models (e.g., code sabotage, research sabotage, data poisoning), including sandbox environments and tools. Each setting includes main tasks (benign tasks the model should complete, e.g., "add a new team to the AWS organisation") and side tasks (harmful actions a misaligned model might attempt concurrently, e.g., "subtly add a security vulnerability to this infrastructure"). A framework for imple…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy