In contrast to a "Manhattan Project for AGI," Eric Schmidt

The head of the Center for AI Safety, Dan Hendrycks, Scale AI's big wig, Alexandr Wang, and former Google CEO Eric Schmidt are advising the U.S. in a recently released report. not to venture headlong into a frenzied race to crack the code of artificially intelligent systems with superior cognitive abilities, also known as AGI.

Their paper, named “Superintelligence Strategy," warns that the U.S.'s hasty quest to be the sole possessor of this superintelligence could have China up in arms, potentially triggering a cyber backlash that could seriously harm diplomatic relations.

In simple terms, they argue, planning a grand-scale strategy to monopolize AGI is highly likely to fire back. A blind pursuit of dominance, they caution, could unleash an avalanche of counterattacks, amplifying tensions, and undermining the stability that such a strategy was meant to ensure in the first place.

Penned by three of America's AI titans, the paper cleverly holds a mirror to a U.S. congressional commission’s recent idea of funding AGI—with an approach reminiscent of that used for the atomic bomb program of the 1940s. The U.S. Secretary of Energy Chris Wright has made a suggestion that the U.S. is embarking on a “new age of the Manhattan Project”, this time focused on AI.

As a result, the Superintelligence Strategy document boldly challenges the generally held belief that the best way for the United States to compete with China is to launch a government-led AGI program.

In what feels much like a game of AGI chess, our daring trio believes that the U.S. is locked in a cold war-like standoff reminiscent of mutually assured destruction. Just as world powerhouses avoid monopolizing nuclear weapons to prevent triggering an enemy's pre-emptive strike, they suggest the U.S. should tread with care in this high-stakes AI race.

Already viewed as a trump card in warfare, AI systems are increasingly considered as integral to military strategy as nuclear weapons. A growing testament to this, the Pentagon confirms that AI fast-tracks military operations.

In this paper, Schmidt and his co-writers introduce a concept they term "Mutual Assured AI Malfunction," proposing that countries pull the plug on threatening AI projects before they can be weaponized.

So, instead of becoming embroiled in a potentially catastrophic race for AI supremacy, the paper’s authors propose foregoing the ‘race to win' mentality. They suggest refocusing efforts on finding smart ways to discourage other nations from developing superintelligent AI themselves.

Ever wondered what it’s like in the AI policy world? Well, as per our authors, it’s a world of stark philosophical contrasts. You have the 'doomers,’ who whole-heartedly foresee calamitous endings and advocate for the brakes to be put on all AI development. And then there are the "ostriches," who passionately call for accelerating AI development, adopting a rose-tinted 'let’s see what happens' approach.

Instead, our bold trio recommends a middle-ground strategy: calculated, defensively-oriented AGI development.

Their new stance strikes a particularly interesting contrast to Schmidt’s earlier louder calls for the U.S. to aggressively compete with China in pushing the envelope of advanced AI systems. Just months prior, Schmidt had penned an op-ed heralding DeepSeek’s role in America’s AI face-off with China.

Despite the Trump administration's fervor in powering on with American AI advancement, our authors remind us that America's AGI moves don't occur in isolation.

Rather than pushing the envelope to claim AI superiority, Schmidt and his co-authors urge us to perhaps consider a more defensive play as the world watches America’s next moves in AI.

by rayyan