does AI actually write insecure code? what the evidence says

a tour through the studies, the benchmarks, and the enterprise telemetry, and what they actually let you claim

Joakim Nordheim Larssen

// why this question is worth getting right

This is no longer academic. Satya Nadella said in April 2025 that 20 to 30 percent of code in Microsoft's repositories is now AI-written, and Sundar Pichai said AI generates more than 30 percent of new code at Google [21]. Take those with salt: it is unclear how either company measures "AI-generated," and these are CEO talking points, not audited metrics [21]. But the direction is corroborated by Stack Overflow's 2025 survey, which found 84 percent of developers use or plan to use AI tools, up from 76 percent the year before [23].

But trust is falling as the volume grows. In the same survey, trust in AI accuracy fell to around 30 percent, down from roughly 40 percent the year before [23]. The top frustration, cited by 66 percent, was AI solutions that are almost right but not quite; a separate 45 percent said debugging AI-generated code is more time-consuming [23]. Near-universal adoption, collapsing trust, growing review burden.

One-sentence thesis: the security risk of AI-generated code is real and measurable, but it concentrates in human over-trust and weak review loops, not in the model being categorically worse than the humans it learned from.

// the scary headline, and what it actually measured

The single most-cited statistic comes from a 2021 NYU study (published at IEEE S&P 2022), "Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions," by Pearce and colleagues [1]. They built 89 scenarios deliberately targeting high-risk weaknesses from MITRE's "Top 25" CWE list, had Copilot generate completions, and ended up with roughly 1,689 programs; about 40 percent contained security vulnerabilities, the origin of "40% of AI code is insecure" [1].

But read the denominator. This measured Copilot's raw completions on deliberately security-sensitive prompts, in 2021, in the Codex era, engineered to steer the model toward risky patterns, not typical day-to-day code. The authors' own explanation is garbage in, garbage out: Copilot reproduces insecure patterns common in its training data, and they recommend pairing it with security tooling rather than trusting it raw [1]. So the 40 percent measures how often a model emits a vulnerable pattern when you push it toward one: the upper bound, not the production rate.

// the actual mechanism: the over-trusting human

If "Asleep at the Keyboard" is the famous study, the keystone is from Stanford: "Do Users Write More Insecure Code with AI Assistants?" by Perry, Srivastava, Kumar, and Boneh, at ACM CCS 2023 [2]. It studies people, not just the model: 47 participants (33 with AI access, 14 control), an assistant built on OpenAI's codex-davinci-002, across five security tasks in Python, JavaScript, and C [2].

Participants with AI access wrote significantly less secure code, with statistically significant gaps on four of the five tasks. On ECDSA message signing, only 3 percent of the AI group produced a secure solution versus 21 percent of the control (p = 0.039) [2]. On a JavaScript SQL insert, the rate of SQL-injection-vulnerable code was 36 percent with AI versus 7 percent in the control (p = 0.041) [2].

The keystone result: participants with the AI were more likely to believe they had written secure code, even though they often had not [2], and the protective behavior was behavioral, not technical: those who trusted the AI less and engaged more with their prompts produced fewer vulnerabilities [2]. What predicted security was not the model but how much you doubted it.

N is small (47) and codex-davinci-002 is now old, so do not generalize the exact percentages. But the mechanism, over-trust producing under-review, is the durable finding.

// the honest counter-evidence: the model alone is not uniquely evil

Two rigorous studies cut against the alarmist reading. The first is "Lost at C," from NYU, at USENIX Security 2023: 58 participants implementing a singly-linked list in C, with an assistant based on code-cushman-001 [10]. AI-assisted users produced critical security bugs at a rate no more than about 10 percent higher than the control, with no statistically significant difference per line of code. The authors concluded the LLM "does not introduce new security risks" [10].

The second is "Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities?" by Asare and colleagues [11]. They took real C/C++ commits where humans had introduced vulnerabilities, re-prompted Copilot at exactly that point, and checked what it produced: it reproduced the original vulnerable code about 33 percent of the time and the fixed version about 25 percent [11]. In roughly two-thirds of cases it did not reproduce the human's vulnerability. Their conclusion: Copilot is "not as bad as human developers at introducing vulnerabilities" [11].

How do you reconcile Stanford with these? Effect size depends heavily on task, language, model, and user behavior. Cryptography and path validation (Stanford's hard tasks) are the subtle domains where a confident wrong suggestion does damage; pointer juggling in C (Lost at C) is where the human was already going to struggle. That variance is the argument: if the model were uniformly dangerous, you would not get this spread. The spread tells you the risk is contextual and mediated by the human and the task.

// the modern, large-scale evidence: real, and not improving

The recent evidence is large-scale and sharpens two points: the problem is real, and bigger models are not solving it. The headline benchmark is Veracode's 2025 GenAI Code Security Report, which tested more than 100 LLMs across four languages [4]. Across the test cases, 45 percent of code samples failed security tests by introducing an OWASP Top 10 vulnerability, from Java worst at roughly 72 percent down to roughly 38 percent for the lowest-risk languages (Python, JavaScript, C#) [4]. And the finding that matters most for the "wait for better models" crowd: while the models got better at writing functional code, security performance stayed flat regardless of model size [4].

A second Veracode update, in March 2026, found the trend held: two years of "revolutionary" releases moved the security needle from about 55 percent to about 55 percent, while syntax correctness now exceeds 95 percent [5]. The structural insight is stable across both reports: models handle localized, pattern-matchable weaknesses well (SQL injection pass rate around 82 percent, insecure cryptography around 86 percent) but fall apart on context- and dataflow-dependent flaws (XSS around 15 percent, log injection around 13 percent) [5]. They can pattern-match a parameterized query; they cannot trace tainted data across a whole request flow.

The caveat: Veracode sells scanning, so weigh it as a vendor report. (The much-repeated "2.74x more vulnerabilities than human code" claim attributed to them is not in the primary blog text, so I will not state it as fact [4].)

But the flat-security finding is more than one vendor's claim: it converges with a peer-reviewed result. Meta's CyberSecEval, covering 50 CWEs across 8 languages, found models suggested insecure code roughly 30 percent of the time on average, and a negative correlation between coding capability and security: models better at reproducing training patterns were more likely to produce insecure code [3]. The better a model learns its training data, the more faithfully it learns the insecure parts. Two independent sources agree that capability gains do not buy you security.

// the rates converge (which is the actual signal)

Look at the spread of "how often does AI emit insecure code":

• NYU's adversarial scenarios: about 40 percent [1].
• Meta's CyberSecEval, per completion: about 30 percent [3].
• Veracode's large-scale OWASP test: 45 percent [4].
• 452 real Copilot snippets merged into GitHub projects (Fu and colleagues): 29.6 percent contained security weaknesses, across 38 CWEs [9].
• CWEval, scoring functionality and security jointly: requiring both drops the pass rate about 30 percent versus correctness alone, so roughly a third of functionally-correct LLM code still fails its security requirements [8].

Different methodologies, roughly agreeing on a 30 to 45 percent band. When a synthetic benchmark, a vendor scan, and a study of shipped code all land in the same neighborhood, you are looking at a genuine property, not an artifact. Denominators still matter: the high end comes from prompts stressed toward risky CWEs (NYU at 40; SecurityEval's 68 to 74 percent) [1][7], while the in-the-wild number is closer to 30 [9].

// "it works" and "it's safe" are different axes

One benchmark nails the core conceptual error people make: BaxBench, "Can LLMs Generate Correct and Secure Backends?", from ETH Zurich and LogicStar, at ICML 2025 [6]. It is 392 backend tasks, each with both functional tests and expert-written exploits, run against 11 models including reasoning models like o1 and DeepSeek-R1 [6]. Even the best do not exceed roughly 35 percent correct-and-secure [6]. OpenAI's o1 hit about 62 percent on correctness alone, but roughly half of those functionally correct programs were still exploitable [6].

This is the cleanest demonstration that "it passes the tests" and "it is safe" are orthogonal: the more you rely on correctness-only benchmarks like HumanEval, the more you overstate how production-ready the output is. And reasoning models, the great hope for closing the gap, did not on BaxBench [6].

// the new failure modes that are unique to AI

So far, "AI writes the same bugs humans write, just at scale." But three failure modes exist because the code came from a model.

slopsquatting: hallucinated dependencies as an attack surface

The clearest is package hallucination. The primary study is "We Have a Package for You!" by Spracklen and colleagues, at USENIX Security 2025: 16 LLMs, 576,000 code samples, with hallucination rates of at least 5.2 percent for commercial models and 21.7 percent for open-source ones, producing 205,474 unique fake package names; coverage summarizes the average as around 19.7 percent, roughly one in five recommended packages not existing [13].

The part that turns a hallucination into a weapon is persistence: when a hallucinated name was re-queried, a large share recurred, with coverage citing 43 percent appearing in all ten re-runs [13]. A name the model invents consistently is predictable, and predictable means an attacker can pre-register it.

The term is "slopsquatting," coined by Seth Larson of the Python Software Foundation [14], and it is not theoretical: Bar Lanyado, of Lasso Security, noticed LLMs repeatedly hallucinating a non-existent PyPI package called huggingface-cli, registered an empty package under that name, and watched it pull in more than 30,000 downloads over three months; even Alibaba copy-pasted the fake install command into a public README [15]. The limit: it was a benign proof of concept, with no confirmed report of a malicious slopsquatted package causing a breach at scale [15]. The mechanism is proven; the mass-casualty event is not yet documented.

secrets leakage

The second is secret exposure. GitGuardian analyzed about 20,000 public repositories with Copilot active and found that 6.4 percent leaked at least one secret, against a 4.6 percent baseline, roughly 40 percent higher [16]. Their own interpretation is on-thesis: largely it is that AI-assisted developers prioritize speed over hygiene [16]. A human-in-the-loop effect, again. Models can also regurgitate real credentials from training data: in a study GitGuardian cites, 900 prompts extracted 2,702 hard-coded credentials from Copilot, at least about 200 of them real, identifiable secrets [16]. (Caveats: GitGuardian sells secret-scanning, and the comparison is correlation, not causation.)

iterative degradation: the "make it better" trap

The third is the most counterintuitive. A study at IEEE ISTAS 2025, "Security Degradation in Iterative AI Code Generation," took 400 code samples through 40 rounds of "improvement" and found a 37.6 percent increase in critical vulnerabilities after just five iterations, even when the prompt asked the model to make the code more secure [18].

If you have ever sat in a "vibe coding" loop telling a model "fix it, try again" a dozen times, this should give you pause: the loop you think is converging on quality may be drifting away from it, and you would not know unless something outside it checked.

// the enterprise telemetry: where this goes at scale

Studies are clean but synthetic. What happens in real companies? Apiiro, an application-security vendor, published telemetry in September 2025, "4x Velocity, 10x Vulnerabilities," from tens of thousands of repositories across Fortune 50 enterprises [12]. AI-assisted developers produced 3 to 4 times more commits, and by June 2025 AI-generated code drove more than 10,000 new security findings per month, a 10x spike in six months [12].

But the shape matters more than the volume. Trivial syntax errors dropped about 76 percent, but privilege-escalation paths rose 322 percent, architectural and design flaws rose 153 percent, and exposed cloud credentials roughly doubled [12]. (The widely-repeated "40% jump in secrets exposure" from this report is wrong; Apiiro's own post says secrets exposure roughly doubled [12].)

This is Veracode's pattern-matchable-versus-dataflow split at enterprise scale: AI is fixing the shallow bugs and multiplying the deep ones. The risk migrates to exactly the places where humans were the last line of defense and are now reviewing least because the volume is 4x. (Apiiro sells code-risk tooling and the data is self-reported, but the qualitative shift corroborates an independent pattern.)

// the process evidence: it's the batch size, not the model

Google's DORA team, in the 2024 Accelerate State of DevOps report, found that a 25 percent increase in AI adoption was associated with an estimated 1.5 percent decrease in delivery throughput and a 7.2 percent decrease in stability, even as around 75 percent of respondents reported individual productivity gains [20]. DORA's interpretation is the punchline: AI makes it easy to produce more code in larger batches, and large changesets are the real risk driver [20].

This pairs with METR's July 2025 randomized controlled trial, which documents the cognitive bug behind the whole problem. Sixteen experienced developers worked 246 real issues on their own mature repositories, randomized to AI-allowed or not, mostly using Cursor Pro with Claude 3.5 and 3.7 Sonnet [22]. AI made them 19 percent slower. And yet going in they expected a roughly 24 percent speedup, and even after the slowdown their post-task estimate was still about a 20 percent speedup [22].

That perception gap is the same one Stanford measured for security. When you overestimate speed, you ship a bit too eagerly; when you overestimate security, you ship something exploitable while feeling safe. (Caveat: METR's N is 16, on complex mature repos with early-2025 tooling.)

// what actually helps

The evidence points at a specific division of labor between deterministic tools and AI. AI is a good fixer of known issues: GitHub's Copilot Autofix, which pairs CodeQL with a model, reported median remediation in 28 minutes versus 1.5 hours manually [19]. The deterministic scanner finds the issue, the model drafts the fix; that pairing produces the speedup, not the model alone (and GitHub's data measures fix speed, not correctness [19]). But AI as a detector on its own is weak: a 2025 study, "Can AI Spot Security Flaws Before You Commit?", found Copilot's review frequently failed to detect SQL injection, XSS, and insecure deserialization, and missed configuration-file security flaws [17]. Deterministic SAST to find, LLM to fix.

The standing recommendations are unglamorous and correct: pair AI generation with SAST and policy gates, run secrets scanning, pin and review dependencies (which defuses slopsquatting), keep changesets small, and put security requirements into prompts. OWASP has a Top 10 for LLM applications [24], and NIST published a Generative AI Profile (SP 800-218A) in July 2024 [25]. None of this is "stop using AI." It is "do not skip the review just because the code came out fast and looks confident."

// my reasoned position

The "AI writes insecure code" claim is true in a narrow sense: models emit a vulnerable pattern roughly 30 to 45 percent of the time, and that has not improved as models got larger [3][4][5][9]. But the alarmist version, that AI is a uniquely dangerous menace, does not survive Lost at C and the Asare study, which found the model alone is no worse, and in head-to-head reproduction actually better, than the human baseline [10][11].

The speed and scale act through the human: Stanford on feeling secure while being less so [2], Snyk on three-quarters of developers trusting AI code more than human code yet bypassing security policies [26], METR on misjudging even raw speed [22], DORA on larger batches degrading stability [20]. Over-trust plus weak review.

So the headline I would write is not "AI writes insecure code." It is: AI writes about as insecure as its training data, confidently and quickly, and confident-and-quick is precisely the combination that erodes human review. The vulnerability is in the loop, not the model. AI did not invent insecure code; it learned from us, and just learned to do it faster and to sound sure of itself. The job now is to stay un-sure on its behalf.