tech
Pacing Model Development in an Era of Cyber-Critical Capabilities
Over the past several weeks, two developments have underscored the growing risks associated with increasingly capable AI systems: the OpenAI-Hugging Face incident and, separately, preliminary evidence that one of our upcoming models, Astra, may meet the Critical cybersecurity capability threshold under our Preparedness Framework. Together, these developments, combined with rapid progress in our internal research, have added urgency to our work on strengthening our monitoring, alignment, and containment safeguards across all stages of the training process.
TL;DR
- Two recent developments, the OpenAI-Hugging Face incident and evidence of Astra's critical cybersecurity capabilities, have raised concerns about AI risks.
- The company is slowing the pace of scaling and pausing certain training runs to strengthen monitoring, alignment, and containment safeguards.
- New security requirements, including workload and network isolation, are being implemented for research environments.
- A multistage monitoring system is being deployed to detect and respond to concerning AI behavior within 30 minutes.
- Alignment research is being intensified to ensure AI systems behave as intended and reduce risks like deception and unauthorized access.
- The Preparedness Framework is evolving to better reflect future model capabilities and operating environments.