AI Agents Could Be Turned Into Botnets Through Hallucinations, Researchers Warn

By Jason Nelson

AI hallucinations may be more than incorrect answers—they could become a way for hackers to compromise computers, according to new research from Tel Aviv University, Technion, and Intuit.

In the paper, “Beware of Agentic Botnets: Scalable Untargeted Promptware Attacks via Universal and Transferable Adversarial HalluSquatting,” researchers demonstrated a technique that exploits AI models when they generate fake links to software repositories and other online resources.

“The growing adoption of agentic LLM applications has introduced a new threat previously named as promptware,” the researchers wrote. “While prior work has established that adversaries can exploit direct channels to LLM applications to apply promptware under weak threat models, many applications do not provide any direct channels that could be exploited for prompt injection beyond the Internet.”

Known as adversarial hallucination squatting or “HalluSquatting,” the attack involves predicting which fake resources AI models are likely to create, registering those names, and adding malicious instructions. If an AI agent later retrieves the hallucinated resource, it may treat the attacker-controlled content as legitimate.

The researchers said the threat emerges as AI assistants move beyond answering questions and gain the ability to interact with computers—accessing files, searching the web, writing code, and running commands.

Those abilities can create security gaps when agents act on information they retrieve without confirming whether the source is real.

“Ongoing studies have demonstrated various variants of Promptware attacks against real-world systems, including ChatGPT, Google Assistant, Copilot, and various additional applications,” they wrote. “These works demonstrated that Promptware can lead to financial, privacy, and safety impacts.”

Researchers warned the technique could allow attackers to build AI-enabled botnets. A botnet refers to a network of infected computers or devices controlled remotely by an attacker. Botnets are commonly used in cyberattacks, including denial-of-service attacks, cryptocurrency mining, malware distribution, and ransomware campaigns.

In testing, the researchers found AI-generated resource hallucinations occurred at rates as high as 85% in repository cloning scenarios and 100% in skill installation tests.

The team evaluated the technique against AI coding assistants and agents, including Cursor, GitHub Copilot, Gemini CLI, and OpenClaw.

HalluSquatting is similar to typosquatting, a cyberattack tactic where attackers register domain names resembling legitimate websites or software packages to trick users. Instead of exploiting human typing mistakes, HalluSquatting targets mistakes made by AI models.

The news comes as researchers continue to test how attackers can manipulate AI agents.

In April, Google researchers detailed malicious websites designed to hijack AI agents through indirect prompt injection attacks, including attempts to steal passwords, delete files, and manipulate payments. A separate study on the “CopyPasta” attack showed how hidden prompts inside developer files could manipulate AI coding assistants into spreading malicious code.

In June, an OpenClaw user reported facing more than 6,000 attempts from attackers attempting to trick the AI agent into leaking sensitive information.

Recommended News