AI Assistants Left Vulnerable to Malware Attacks Through Hallucinated Code

A new threat has emerged in the world of AI assistants, where hackers can exploit a flaw in these tools’ coding capabilities. Cybersecurity researchers have discovered that attackers can use ‘adversarial hallucination squatting,’ or ‘hallusquatting,’ to install malware directly on users’ computers. This method takes advantage of the tendency for large language models to produce inaccurate results, known as ‘hallucinations.’

The attack works by identifying package names that AI assistants are likely to reference and registering them as real repositories. Malware is then inserted into these packages, which wait to be accessed by an AI assistant. Once downloaded, the malware executes code on the user’s machine without their knowledge or consent.

Researchers at Tel Aviv University and Intuit found that this scenario can occur in common AI coding tools, including Cursor, Microsoft’s Copilot, and others. The rates of occurrence vary depending on the specific task being performed, but range from 85 to 100 percent. This means that nearly all users who rely on these AI assistants are at risk.

The attack is particularly insidious because it relies on an automated process. As a result, victims may not even realize their AI assistant has downloaded malware until well after it begins executing code. This makes it difficult for users to detect and prevent the attack in real-time.

AI companies have been notified about this exploit, but the underlying problem remains: AI assistants are prone to producing inaccurate results, making them vulnerable to manipulation by attackers. The researchers held back some sensitive details that could aid hackers in improving their workflows, but the core issue persists.