TECH

Iran and China use AI agents to create fake social media profiles

Iran and China have used autonomous artificial intelligence agents to create and manage networks of fake accounts on platforms such as Instagram, Facebook, X, and TikTok. These operations were identified by U.S. authorities and security researchers.

The groups combined open-source Chinese AI models with agents capable of operating software and executing tasks independently. Hundreds of these systems were reportedly employed to open profiles, generate political posts, and coordinate messaging aimed at influencing opinions and stoking online debate.

These so-called AI agents differ from conventional chatbots because they are not limited to answering questions or generating text. They can be assigned a goal and execute a sequence of actions to achieve it—such as accessing a platform, filling out forms, creating accounts, and publishing content. In the identified campaigns, this technology reportedly allowed for the automation of virtually the entire process, from profile creation to message dissemination, with minimal human involvement.

According to a report by *The New York Times*, operators in most cases disabled the safeguards that would otherwise prevent the models from creating fake accounts or manipulating online discussions.

Based on the characteristics of these operations, U.S. authorities and cybersecurity experts attributed the Iranian and Chinese campaigns to initiatives linked to their respective governments. Two other similar operations, associated with Israel, were reportedly conducted by private companies.

The campaigns were considered technically rudimentary, and there is no indication that they achieved significant reach. Nevertheless, they drew the attention of intelligence agencies, technology companies, and researchers by demonstrating how autonomous agents can reduce the cost and effort required to maintain networks of fake profiles.

Operations of this type previously relied on teams tasked with creating accounts, drafting posts, and coordinating interactions. With AI agents, a smaller number of people can control hundreds of profiles and continuously produce content. The technology also makes it possible to rapidly adapt messages to different platforms, topics, and audiences. The discovery points to a new phase in digital influence campaigns. The risk lies not only in the production of fake text, images, or videos, but in the automation of the entire infrastructure needed to distribute these materials and create the appearance of spontaneous support for specific narratives. 

First time Chinese models were used like this...US officials told the NYT that these instances marked one of the first times AI agents from Chinese-produced models were used to handle every stage of an online influence campaign, from creating new fake accounts to coordinating messaging and framing online.

Unlike proprietary AI models, the open-source models were manipulated by developers to allow the agents to evade safeguards designed to verify users and prevent the manipulation of online discourse, the US officials added.

The US intelligence community, along with cybersecurity experts, looked into the groups behind the AI-powered campaigns and traced those efforts back to state-backed Chinese and Iranian groups, as well as to Israeli campaigns stemming from two private firms.

Kyle Crichton, a fellow at the Georgetown Center for Security and Emerging Technology who studies AI and cybersecurity, told the NYT that “Agent-led campaigns” will likely become more common as the technology spreads.

“Agents can be fairly sophisticated, and so in addition to just scale, they have an ability to blend in and look more like a human user,” he said.

The investigation was published in the wake of other major technology news, as AI leaders warned of the risks of developing AI too quickly and without proper safety measures. These warnings followed public concern over recent stories of AI models going rogue and hacking other companies.

In a report last month, Meta said it had seen a “technically significant development” of bad actors using AI for malicious purposes.

AI is now “embedded within automated systems that can produce, adapt, and distribute content with minimal human intervention,” Meta added.

The investigation also traced AI-agent campaigns back to Israel...The report mentioned that IntelEye, a Tel Aviv-based company founded by former NSO Group employees, operated roughly 1,000 AI-powered fake accounts on social media to manipulate political discourse around the upcoming Israeli elections.

The company used DeepSeek, a Chinese open-source AI model, to create its agents. According to the investigation, the bots then published content both supporting and opposing Prime Minister Benjamin Netanyahu.

IntelEye disputed claims that its work was intended to influence online discussions, stating the company was conducting a defensive research experiment to expose weaknesses in AI systems.

“IntelEye does not operate influence campaigns,” co-founder Maor Sellek told the Times. “Our work with DeepSeek was conducted for defensive research and testing, to understand how models with insufficient safeguards could be misused and to improve detection of that activity.”

Sellek said IntelEye bought 10,000 social media accounts and handed over control to autonomous AI agents, though he added that only about 1,000 of the accounts were operational.

The agents used the accounts to leave comments on posts to boost engagement, although Sellek said they were not very successful.

Meta told the New York Times that the vast majority of the AI-powered agents set up on Facebook and Instagram by IntelEye, other companies, and state-backed groups had been removed by its automated detection and verification systems.

Networks of fake accounts...According to the publication, publicly available Chinese artificial intelligence models were used to create autonomous agents capable of performing online tasks. Hundreds of such agents were allegedly deployed to create networks of fake accounts on the American platforms Instagram, Facebook, X and TikTok.

The agents filled the accounts with posts about politics and current events, and also coordinated messages and online interactions. U.S. officials and cybersecurity researchers called this among the first known cases in which agents based on Chinese AI models were involved in almost the entire influence campaign process.

More current news is available on the UA.News Telegram channel Telegram. The operations were described as relatively primitive, but as demonstrating the possibility of automating tasks that previously required significant human involvement.

Iranian and Israeli operations...U.S. intelligence services, technology companies and security researchers investigated this activity and, according to The New York Times, linked the Chinese and Iranian operations to state-backed actors. Iranian AI accounts posed as ordinary Americans in major cities, shared popular memes, tagged journalists and politicians, and promoted views critical of the Republican Party.

Nearly 80,000 people followed these accounts in the first half of the year. The report also describes two Israeli campaigns linked to private companies. One of them, linked to the Tel Aviv-based company IntelEye, involved using AI agents to manage thousands of accounts and participate in discussions about upcoming elections in Israel. The company said it was a defensive research experiment to test the potential misuse of AI models.