The real message from China’s latest AI sensation
Plus: Genetically modified pigs, weapons galore, backsliding at home, backsliding abroad, solar surge, and more!
—The New York Times reports that genetically modified miniature pigs are being produced via cloning in rural Wisconsin, where scientists are pursuing the dream of mass-scale xenotransplantation (the insertion of animal hearts and other organs into humans who need them). But the Times may be making such a future less likely by providing endearing pictures of the piglets—which are rendered feeble in the course of being optimized for organ production—and noting that the project raises challenging moral questions.
—America is expanding its dominance of the global weapons market, according to the Stockholm International Peace Research Institute. The institute’s latest report found that US arms exports accounted for 43 percent of the international weapons trade between 2020 and 2024.
—Romanian courts banned popular far-right candidate Calin Georgescu from running in the upcoming presidential election, drawing fierce backlash from Georgescu’s supporters and even some left-wing thinkers like former Greek Finance Minister Yanis Varoufakis, who called the decision “preposterous.” Romanian officials claim that Russian meddling was responsible for Georgescu’s victory in the first round of a presidential election last year—a victory that was annulled. (For more on the Georgescu case and the risks of democratic backsliding in Romania, see this edition of the Earthling from last year.)
—Solar panels made up 66 percent of new electricity generation capacity installed in the US in 2024, according to a new report published by a solar industry group and a data analytics firm. Solar installations are expected to slow down under the Trump administration, which wants to slash federal funding for the clean-energy sector.
—Writing in Nature, four scientists have sounded an alarm about microplastics alarmism—and they cite as one of their examples a study, also published in Nature, which NZN covered last month (alongside a memorable spoon graphic, to illustrate the study’s finding that the average American brain contains a plastic spoon’s worth of plastic). The authors say that confident conclusions about the quantity of microplastics in human tissue are rendered difficult by the abundance of plastics in testing and storage environments and inadequate safeguards for preventing the contamination of post-mortem samples.
—The Kurdish-led Syrian Democratic Forces, which have controlled Syria’s northeast for roughly a decade, agreed to come under the authority of the interim government in Damascus, marking a key step toward creating a united Syrian state following the fall of the Assad regime and the end of a 14-year civil war. American officials had urged the SDF to take the deal, according to Reuters, which also reported that the SDF’s leader arrived in Damascus for the signing ceremony in an American military helicopter. (See below for more on recent Middle East news, including the massacres of Alawites in northwestern Syria.)
This week the question in tech circles was: Is Manus—a new “agentic” AI from China—bringing us another “DeepSeek moment”? The answer depended on whether you were reading Bloomberg (“China’s Manus Follows DeepSeek in Challenging US AI Lead”) or TechCrunch (“Manus probably isn’t China’s second ‘DeepSeek moment’”) or Forbes (“China’s Autonomous Agent, Manus, Changes Everything”) or a different article in Forbes (“Overhyped: Manus”) or the NonZero Newsletter: Manus is very important, but not in the way people are saying.
To put a finer point on that NZN take: Manus is the first AI product to powerfully demonstrate why, as AI progress continues, the US will need to forge an agreement with China and other nations on some basic rules of the road. Regulation of AI at the national level alone won’t be enough to keep America, or other nations, secure; there will have to be some degree of international governance.
The logic behind that conclusion will be easier to understand after a brief recap of the argument that unfolded online this week about how important Manus is or isn’t. At the risk of oversimplification, here’s that entire argument depicted as a short exchange between two imaginary people:
Manus Booster: Manus is the most powerful AI agent yet! Just look at the things it does in this demo video from the Chinese company that created it and this video by a Manus fan. Manus can go online, research stocks, and then depict the results as interactive graphs. It can go online, research that Manus fan (by, among other things, combing his social media feed) and then create a whole website about this fan for his fans!
Manus Skeptic: It’s true that no single US AI product can do all those things. But can’t all those things be done by some combination of US AI products? In fact, on Sunday, shortly after the hype started, a crafty techie discovered that if you rip away the Wizard-of-Oz curtain surrounding Manus, you’ll find Anthropic’s Claude large language model pulling many of the levers! And there are already Claude-powered AI agents that could have designed and built that website or those interactive graphs. And as for the information that populated them: that could have been gathered by OpenAI’s Deep Research or Google’s Deep Research or Perplexity’s Deep Research. Manus is basically just a combination of the functionality of a Deep Research agent and OpenAI’s Operator (which can go online and order pizza and stuff). So it’s not like the engineers who created Manus are on par with the undeniably ingenious team at DeepSeek. They just took a bunch of existing AI capabilities and cobbled them together.
Booster: Well if that’s so easy to do, then why hasn’t some American company done it?
Skeptic: Because big American tech companies are less inclined than this upstart Chinese company to rush half-baked and potentially dangerous products to market. Did you see what prodigious AI watcher Zvi Mowshowitz said about Manus—that it’s “glitchy and lacks any sorts of guardrails”? And remember: Once you unleash these “agentic” AIs onto the web, bad things can happen—like “prompt injections” by malicious actors, who can then commandeer the AI and get it to wreak havoc online. And as Mowshowitz wrote, Manus “definitely is not making any attempt to defend against prompt injections” or other threats that would also materialize “if there was wide distribution and use of such an agent.” Dean Ball of George Mason University’s Mercatus Center wrote that “every use case of Manus in the company’s demo video would be an enormous liability and regulatory risk for American companies.”
Booster: Well that’s the trouble with America’s oppressive regulatory regime and its rampant litigiousness! We’ll never be able to compete with China if we don’t get the government, and all those money-hungry lawyers, off the backs of our entrepreneurs!
OK, we can end our recap of the online conversation there. Not because nobody of prominence said anything else of interest about Manus, but because, so far as we can tell, nobody of prominence said what NZN would say: Manus portends an increasingly critical non-zero-sum game among nations—either they cooperate to control transborder AI risk (win-win) or they face growing chances of disaster (lose-lose).
Tyler Cowen noted in his reflections on Manus that there’s no clear way to keep a foreign-made AI agent out of the American computer ecosystem. That would seem to mean that perils posed by AI agents are in the same policy category as other transnational threats, such as pandemics (whether started naturally or through a genetic engineering mishap or even a bioweapon) and some environmental issues (climate change or ozone layer depletion or transborder pollution flow). In fact, even if you could somehow keep all Americans from using a foreign-made AI agent, it could still be a transborder threat, since online havoc, wherever it originates, can travel far and wide. So AI agents are yet another technologically created risk that can’t be fully controlled via policy at the national level alone; national security requires international cooperation.
Since Cowen is a libertarian, his mind doesn’t naturally turn toward global governance. And since online AI discourse has a heavily libertarian flavor, it doesn’t naturally turn in that direction either. So apparently it’s NZN’s job to underscore the point: China and the US and all other nations are in the same boat; as AI agents get more powerful, we’ll need some combination of new norms and binding agreements, along with some degree of guaranteed transparency. We may even need to re-introduce a once-familiar word that has virtually disappeared from the American political vocabulary (trigger warning): treaty.
If you ask a good LLM to list some dangers posed by AI agents that roam the web and are vulnerable to prompt injections, you’ll probably get a long list. Claude, for example, gave us no fewer than ten bullet points. Perhaps the most notable is the one about a kind of meta-risk—the possibility that, as Claude put it, “vulnerable agents could be instructed to spread malicious prompts to other agents, creating cascading effects throughout AI systems.”
Last week Anthropic submitted a report to the US government that said its latest LLM, Claude 3.7, “demonstrates concerning improvements in its capacity to support aspects of biological weapons development.” Current trends, Anthropic says, suggest “that numerous AI systems will increasingly embody significant national security implications in the coming years.” The report offered some policies that might help stem this threat, but none of them involved cooperation with China. Instead, Anthropic recommended strengthening export restrictions intended to hobble China’s AI effort.
But the real “DeepSeek moment,” the one in January, suggested this may be a fool’s errand; the fairly draconian chip restrictions that were already in place, we learned, had failed to keep China away from the frontier of AI advance. Meanwhile, this ongoing American assault on China’s tech industry makes US-China cooperation only less likely. It also, as we’ve explained, makes war in Taiwan (and hence, perhaps, World War III) more likely. After all, the chip restrictions deny China the most precious output from Taiwanese factories that would be enduringly disabled by war—so Xi Jinping has less to lose from war than he had before President Biden imposed the restrictions.
All told, steadily mounting risks of international catastrophe seem like a high price to pay for the promise of living in a libertarian paradise.
Four Middle East updates:
—Syria’s interim government is nearing a deal with Druze leaders in the country’s south that would integrate Druze militias into state security forces and establish a police force drawn from the local population, according to several Arab media outlets. The potential agreement comes as Israel has pledged support for the Druze, an Arab minority group with Islamic origins but a distinct spiritual identity, and attempted to expand its influence with the community, which is concentrated near the border with Israel.