OpenAI’s recent safety and alignment document, outlining its approach to developing Artificial General Intelligence (AGI), has drawn criticism from a former policy researcher. Miles Brundage, who previously worked at OpenAI, argues that the document misrepresents the company’s history and downplays legitimate safety concerns.
The document emphasizes OpenAI’s commitment to an iterative deployment process, suggesting a gradual approach to AGI development. This method, they claim, allows for continuous monitoring of safety issues and potential misuse. This contrasts with the supposedly “discontinuous” release of GPT-2, which OpenAI now states was initially withheld due to concerns about malicious applications.
Brundage, however, disputes this narrative. He asserts that the GPT-2 rollout was consistent with, and even foreshadowed, OpenAI’s current iterative deployment philosophy. He recalls the model being released incrementally, with lessons learned and shared at each stage, a process commended by many security experts at the time.
Beyond the historical revisionism, Brundage criticizes OpenAI’s apparent risk assessment methodology as outlined in the document. He expresses concern over what he perceives as a high burden of proof required to address safety issues. Brundage argues that this approach, seemingly requiring overwhelming evidence of imminent danger before taking action, is a “very dangerous mentality for advanced AI systems.”
The release of this document follows mounting public criticism of OpenAI. Experts have voiced concerns about potential inaccuracies in AI-generated information, particularly in areas like health and safety, as highlighted by incidents like Google’s AI search engine providing dangerous advice. Other concerns include the potential for political manipulation, misinformation, and scams.
OpenAI has also faced scrutiny regarding its lack of transparency in AI model development, particularly given the sensitive personal data these models can contain. This latest document appears to be an attempt to address these criticisms. However, Brundage’s critique raises questions about the sincerity and thoroughness of OpenAI’s commitment to safety, particularly in light of accusations that the company prioritizes product development over safety precautions.
This controversy underscores the growing tension between the rapid advancement of AI and the crucial need for responsible development and deployment. As AI systems become increasingly sophisticated, ensuring their safety and mitigating potential risks will remain a critical challenge.
