Researchers Say the Deepfake Biden Robocall Was Likely Made With Tools From AI Startup ElevenLabs

This is not the first time that researchers have suspected ElevenLabs tools were used for political propaganda. Last September, NewsGuard, a company that tracks online misinformation, claimed that TikTok accounts sharing conspiracy theories using AI-generated voices, including a clone of Barack Obama’s voice, used ElevenLabs’ technology. “Over 99 percent of users on our platform are creating interesting, innovative, useful content,” ElevenLabs said in an emailed statement to The New York Times at the time, “but we recognize that there are instances of misuse, and we’ve been continually developing and releasing safeguards to curb them.”

If the Pindrop and Berkeley analyses are correct, the deepfake Biden robocall was made with technology from one of the tech industry’s most prominent and well-funded AI voice startups. As Farid notes, ElevenLabs is already seen as providing some of the highest-quality synthetic voice offerings on the market.

According to the company’s CEO in a recent Bloomberg article, ElevenLabs is valued by investors at more than $1.1 billion. In addition to Andreessen Horowitz, its investors include prominent individuals like Nat Friedman, former CEO of GitHub, and Mustafa Suleyman, cofounder of AI lab DeepMind, now part of Alphabet. Investors also include firms like Sequoia Capital and SV Angel.

With its lavish funding, ElevenLabs is arguably better positioned than other AI startups to pour resources into creating effective safeguards against bad actors—a task made all the more urgent by the upcoming presidential elections in the United States. “Having the right safeguards is important, because otherwise anyone can create any likeness of any person,” Balasubramaniyan says. “As we’re approaching an election cycle, it’s just going to get crazy.”

A Discord server for ElevenLabs enthusiasts features people discussing how they intend to clone Biden’s voice, and sharing links to videos and social media posts highlighting deepfaked content featuring Biden or AI-generated dupes of Donald Trump and Barack Obama’s voices.

Although ElevenLabs is a market leader in AI voice cloning, in just a few years the technology has become widely available for companies and individuals to experiment with. That has created new business opportunities, such as creating audiobooks more cheaply, but also increases the potential for malicious use of the technology. “We have a real problem,” says Sam Gregory, program director at the nonprofit Witness, which helps people use technology to promote human rights. “When you have these very broadly available tools, it’s quite hard to police.”

While the Pindrop and Berkeley analyses suggest it could be possible to unmask the source of AI-generated robocalls, the incident also underlines how underprepared authorities, the tech industry, and the public are as the 2024 election season ramps up. It is difficult for people without specialist expertise to confirm the provenance of audio clips or check whether they are AI-generated. And more sophisticated analyses might not be completed quickly enough to offset the damage caused by AI-generated propaganda.

“Journalists and election officials and others don’t have access to reliable tools to be doing this quickly and rapidly when potentially election-altering audio gets leaked or shared,” Gregory says. “If this had been something that was relevant on election day, that would be too late.”

Updated 1-27-2024, 3:15 pm EST: This article was updated to clarify the attribution of the statement from ElevenLabs.
Updated 1-26-2024, 7:20 pm EST: This article was updated with comment from ElevenLabs.