The pursuit of Artificial General Intelligence (AGI) has ignited a debate surrounding its potential benefits and risks. A recent paper titled “Superintelligence Strategy,” co-authored by former Google CEO Eric Schmidt and Scale AI founder Alexandr Wang, cautions against a U.S.-led “Manhattan Project” for AGI, highlighting the potential for global instability. Their core argument centers on the risk of international retaliation and sabotage as nations compete for dominance in this powerful technological arena. Instead, they advocate for developing defensive measures, such as cyberattacks, to neutralize threatening AI projects.
Schmidt and Wang, while acknowledging AI’s transformative potential in fields like medicine and workplace efficiency, express concerns about its militarization. They argue that governments, driven by a desire to maintain a technological edge in defense, are engaging in a dangerous arms race, developing increasingly powerful and potentially destabilizing AI-powered weaponry. Echoing international agreements that limit nuclear weapons development, they urge a cautious approach to AI development, discouraging a race towards autonomous killing machines.
Paradoxically, both Schmidt and Wang are involved in developing AI products for the defense sector. Schmidt’s White Stork is focused on autonomous drone technologies, while Wang’s Scale AI recently secured a contract with the Department of Defense to develop AI “agents” for military planning and operations. This highlights a growing trend: Silicon Valley, once hesitant to engage with military applications, is now actively pursuing lucrative defense contracts.
This inherent conflict of interest within the military-industrial complex fuels a cycle of escalating arms development. The rationale is often framed around national security: if other nations are developing AI weapons, the U.S. must maintain parity. However, this logic often overlooks the human cost of conflict.
Proponents of AI in warfare, such as Palmer Luckey, founder of Anduril, contend that targeted drone strikes powered by AI are safer than nuclear weapons or landmines. Anduril, which supplies Ukraine with drones for targeting Russian military equipment, recently launched a provocative ad campaign suggesting that working for the military-industrial complex is now a form of counterculture.
Schmidt and Wang emphasize the importance of human oversight in AI-driven decision-making. However, reports indicate that the Israeli military is already using AI, sometimes with flawed results, in lethal operations. Drones, a long-standing subject of controversy, raise concerns about the detachment of soldiers from the consequences of their actions. The potential for errors in image recognition AI further complicates the ethical implications of autonomous weapons systems.
The “Superintelligence Strategy” paper assumes that AI will soon achieve superintelligence, surpassing human capabilities in various tasks. This assumption remains debatable, given the persistent errors and unpredictable behavior exhibited by current AI models.
Critics argue that Schmidt and Wang, along with figures like OpenAI’s Sam Altman, are promoting a narrative of AI’s potential dangers to influence policy and secure their own position in the market. This raises the question of whether these warnings are genuine or strategically motivated.
Despite these warnings, the pursuit of AI dominance continues. The Trump administration’s rejection of Biden-era AI safety guidelines and the Congressional commission’s proposal for an AI Manhattan Project suggest a strong push towards prioritizing AI development over safety concerns. The paper warns of potential retaliation from countries like China, including the degradation of AI models or attacks on infrastructure. Such threats are not unfounded, given China’s documented involvement in U.S. tech companies and Russia’s alleged attacks on undersea cables.
Achieving international consensus on regulating AI weapons development remains a significant challenge. In this context, the concept of sabotaging AI projects as a defensive measure, while controversial, may warrant consideration.
