Every building block needed to compromise AI-generated code at scale has been independently demonstrated in research labs and real-world incidents, according to a War on the Rocks analysis by Markus Sandelin, AI Lead at the NATO Communications and Information Agency. The concern is no longer reliability but deliberate compromise, "Hvylya" reports.

Researchers at USENIX Security 2024 showed that contaminating just 0.2 percent of a model's training data - 160 files out of 80,000 - embedded backdoors that evaded all standard detection tools. The poisoned output looks identical to clean code because the model learns the backdoor as a secondary pattern triggered by specific context, while the rest of the training data teaches it to write normal code.

Anthropic published research in January 2024 demonstrating that a model could write secure code under normal conditions while injecting exploitable vulnerabilities when triggered by a specific signal - in their experiment, the calendar year changing. "The backdoor survived every standard safety technique, including the reinforcement learning process specifically designed to remove unwanted behaviors," Sandelin wrote. Larger models proved harder to fix because they have more capacity to compartmentalize behaviors.

Trail of Bits demonstrated in August 2025 that an attacker could file a normal-looking bug report on GitHub containing invisible instructions in the page's HTML. When a coding assistant read the page, it followed the hidden instructions and installed a backdoor. The developer saw nothing unusual. In July 2025, an attacker exploited a flaw in the build process for Amazon Q Developer and injected a malicious instruction into the official product distributed through Visual Studio Code's marketplace. The compromised extension had over 964,000 installations and was publicly distributed for two days.

"The only reason it caused no damage was a syntax error in the attacker's payload," Sandelin noted. "A typo is the current margin of safety for AI coding tool supply chains."

Also read: how a four-star general's admission shattered the Pentagon's official line on its secret AI.