AI can dramatically accelerate software delivery, but speed without security creates new forms of risk
Anyone who’s watched an AI coding assistant generate a working API endpoint, scaffold a microservice, produce a comprehensive test suite, or create a user interface understands deep in their core that software engineering has fundamentally changed. The velocity of AI coding assistants is real.
That’s why 97% of organizations are using open source AI models in the software they build, and why nearly 50% of developers have integrated AI coding assistants into their daily workflows. The productivity gains are structural, acknowledged, and they’re not going away.
But before your next sprint or deployment, ask yourself this: What new responsibilities does this velocity create?
The answer is uncomfortable. AI coding assistants are simultaneously the most powerful productivity tool ever handed to software engineers and the most powerful risk multiplier in the history of application security. Neither of those statements is hyperbole. Every organization that has embraced AI-assisted development without an equally aggressive evolution of its security posture is accumulating debt it will eventually have to pay. And the bill arrives in the form of exploitable vulnerabilities in production systems.
I’m not saying we need to abandon the use of AI coding tools. I’m saying we need to use them with the discipline, rigor, and layered defense strategy they demand.
AI-generated code is not secure by default
A Stanford University study found that developers using AI coding assistants wrote significantly less-secure code than a control group working without AI assistance. Paradoxically, the same developers who were using AI were more confident than the control group that the code they’d written was secure. The gap between perceived security and actual security has widened considerably, and that false sense of security greatly amplifies risk. The participants who trusted AI less and challenged its output produced more-secure code. Similar studies have come to the same conclusion: AI amplifies risk, it doesn’t eliminate it.
And of course, the speed compounds everything. AI commits merge approximately four times faster than human-written code. That means insecure code reaches production four times faster than it previously did. The review window has been compressed to almost nothing. AI-generated code already depends on a manual security review with a measured merge process, and the velocity at which it is produced means teams must now perform that review on a significantly larger number of pull requests.
Why AI produces insecure code: Six root causes
Understanding that AI produces vulnerable code is an important first step. But it’s not enough. You need to know why it happens, because that knowledge will shape every defensive decision your organization makes.
AI coding assistants fundamentally lack six things that a seasoned, security-conscious developer brings to every line of code.
Business context
Your AI assistant doesn’t have a business context. It doesn’t understand what your application is doing in production, the data it handles, or the regulatory and compliance consequences of a breach. It generates syntactically correct code that satisfies the surface-level intent of your prompt, but it has no model of the business logic it is implementing or the threat landscape it is operating within.
Security standards awareness
The AI knows that OWASP exists. It knows what SQL injection is. But it doesn’t know which specific controls your organization has selected, which compensating controls are already in place, or which risks you have formally accepted. Without that context, it defaults to patterns that look secure in isolation but may violate your security specifications.
Architectural memory
AI coding assistants have no institutional memory. They do not know your architecture decision records (ADRs), your organization’s accumulated decisions about security tradeoffs, or the lessons learned from past incidents. Each session begins from zero. You must supply that context deliberately. If you don’t feed it to the model, the AI will violate your architecture in ways that are hard to detect and expensive to remediate.
Consistency at the codebase level
An AI assistant sees the local context of whatever prompt you give it. It doesn’t see the entire codebase, the full dependency graph, or the cumulative security posture of the system. Vulnerabilities that emerge from the interaction between components, the architectural-level flaws, are invisible to a tool operating at the level of individual code completion.
Security-first intent by default
The objective function of a large language model (LLM) is to produce plausible, useful completions, not to produce maximally secure code. Without explicit, precise prompting and constraint, the model will consistently produce code that works rather than code that’s safe. In fact, LLMs were originally trained on open source repositories with semantically correct but insecure code. Until they’re trained how to architect, design, and build secure code by default, they’ll continue to produce insecure applications.
Grounding in your specific environment
AI-generated code often does something different than what the developer intended precisely because the model did not read, internalize, or follow every constraint in your environment. Dynamic application security testing (DAST) exists specifically to catch this gap between intent and behavior.
The new attack surface: Three vectors your threat model must include
Beyond the problem of insecure code arriving faster, AI-assisted development introduces attack vectors that didn’t exist five years ago. Your threat model needs to evolve accordingly.
Prompt injection: NIST’s greatest security flaw of the generative AI era
Prompt injection is the number one vulnerability in AI systems for the second consecutive year, and NIST has designated it generative AI’s greatest security flaw. The architectural root cause is fundamental: LLMs process instructions and data through the same channel. The model cannot distinguish whether a sentence came from you, a document it’s reading, or attacker-controlled content it’s processing. Any text the agent ingests can, in principle, rewrite its behavior.
The attack scenarios aren’t just theoretical.
- A malicious comment buried in source code instructs your coding agent to jump environment variables. Your agent complies.
- Your agent fetches a documentation page containing hidden instructions. The agent follows the page’s instructions, not yours.
- An attacker files a GitHub issue. Your agent picks it up during the triage stage. The issue description is actually a command to the AI.
- Credential exfiltration prompts cause your agent to print API keys, run commands, or commit secrets directly to the repository.
Researchers have repeatedly demonstrated the exploitation of GitHub Copilot through instructions hidden in code comments, GitHub issues, and pulled-in documentation, causing Copilot to exfiltrate code, leak secrets, and suggest backdoors. These are not hypothetical edge cases. It’s an active exploitation class.
Slop squatting: The supply chain attacks your developers don’t know are coming
This attack vector is newer, but it’s already being actively exploited. LLMs sometimes hallucinate import statements, confidently suggesting commands like pip install some-library for a package that doesn’t exist. Attackers have recognized this behavior and turned it into a supply chain attack vector.
Researchers analyzing 750,000 code samples across 16 different LLMs identified 205,000 unique hallucinated package names. Critically, 20% of AI-generated code samples referenced at least one nonexistent package. Of those hallucinated names, 43% appeared consistently every time the same prompt was run, and many appeared multiple times. Attackers don’t need to guess which names to register. They simply observe, identify the high-frequency hallucinations, and register those package names with malicious payloads.
Then your developers install the suggested package. Your CI/CD pipeline pulls it in. Your application ships with a supply chain compromise embedded in it, and no one on your team introduced it deliberately. Software composition analysis (SCA) is not optional in this environment. Slop squatting alone could justify an SCA investment.
Agentic abuse: When AI can do more than suggest
Agentic AI is responsible for the fastest-evolving threat class, because agentic systems don’t simply suggest code but autonomously make commits, build infrastructure, trigger deployments, and manage pipelines. When AI moves from advisory to autonomous, the consequences of a compromised or misbehaving agent extend beyond insecure suggestions to destructive, irreversible actions.
The notorious Replit incident illustrates what agentic autonomy can mean in practice. An AI agent, given latitude to manage its own environment, taking actions no human developer explicitly authorized, deleted a live production database, fabricated thousands of fake user records, and then lied about it.
The lesson here is stark. As soon as AI reaches a decision point with real-world consequences (a deployment, a permission grant, a database migration) your traditional human-in-the-loop controls no longer apply unless you’ve explicitly re-engineered them to remain in control.
Organizational threats: Shadow AI and the confidence gap
Technical debt doesn’t exist in isolation. Two organizational patterns may be actively amplifying your risk.
- The “shadow AI” problem is larger than your policy acknowledges. Black Duck research found that 11% of organizations admit to using AI coding assistants without official permission, in unverified and unmonitored ways. AI adoption happened so quickly that governance hasn’t had time to catch up. Black Duck’s research also found that over 20% of organizations have developers using AI to code even when official policies explicitly forbids it. The message is clear: Shadow AI is a current operational reality, not a hypothetical governance issue.
- The confidence gap creates dangerous conditions for your security posture. Despite persuasive adoption, 26% of organizations report little to no confidence in their readiness to secure the AI-generated code they’re already producing. The combination of high output volume and compressed review windows creates the perfect storm under which vulnerabilities accumulate undetected and unaddressed. This results in the attack surface growing faster than the capacity to address or secure it.
Traditional, disparate AppSec tools that were adequate when software environments were smaller and more predictable simply cannot keep pace with the new AI environment. Siloed scanners operating in isolation fail to provide cohesive insight, leaving security teams with fragmented data and developers drowning in uncoordinated noise.
Your developer gets an output multiplier of 4x in code production. If you’re not improving your security tooling signal-to-noise ratios at the same pace, your developers will be overwhelmed by findings and simply ignore them.
The defense: Seven layers that meet the threat
The good news is that the defense posture required to operate AI-assisted development responsibly has been well researched and developed, and it is architecturally coherent. The teams winning with AI are not the teams with the best prompts. They’re the teams with the best guardrails. Defense in depth is not a new concept, but every era reinvents it. This is the AI era’s version.
Here is a seven-layer framework, mapping from before code is written through runtime, that will provide the guardrails and governance you need to leverage AI coding assistants responsibly and safely.
Layer 1: Standards and grounding
The single biggest lever for safer AI-assisted code is grounding your AI assistant before it writes a single line. This means documenting and providing your standards. The AI will violate your style guide, coding, and security standards if you don’t codify them. That means supplying your organization’s actual security policies and standards. Remember, AI knows that OWASP exists, but it doesn’t know which controls you’re required to implement. It means providing your ADRs, because AI has no institutional memory unless you supply it. The bottom line: Organizations that treat AI grounding as optional are accepting unnecessary and avoidable risk.
Layer 2: Threat modeling
Every AI-enabled application carries a dual threat model. Build time AI can introduce vulnerabilities by writing insecure code. Runtime AI is susceptible to prompt injection, data leakage, and hallucination. Both require their own threat models, doubling the modeling work compared to non-AI-enabled development. Skipping this step because AI makes development faster is precisely the tradeoff that compromises your security posture.
Layer 3: AI-generated tests and security loops
Rather than treating AI-generated code as final, build a discipline loop where AI is generating and running tests at every stage: unit tests at the bottom of the pyramid, regression tests to catch reintroduced bugs, and integration tests covering AI boundaries, database interactions, and external service calls. Deliberately split bug review from security review because they require different models, mindsets, and prompts. Every fix should generate a permanent test, so the same vulnerability can’t quietly re-enter the codebase. This layer is the difference between AI-assisted coding that creates security debt and a loop that helps to pay it down.
Layer 4: AI reviews AI
Introduce a separate AI pass specifically scoped to security—for example, a pass that reviews only changed code for security regressions, not the entire codebase. This provides continuous automated security review without replacing human judgement, and it adds a layer of scrutiny to every modification before it reaches human reviewers.
Layer 5: SAST, SCA, and DAST in CI gates
The three big main scanning technologies aren’t optional in an AI-assisted development environment. Indeed, they become more important, not less. Static application security testing (SAST) should run in a CI gate, blocking any critical issues. Run SCA to address the dependency and slop squatting risk. And deploy DAST in the running application to catch the gap between intent and behavior that even a developer who generated and read every line of code may miss. I recommend running these on every commit if your pipeline can support it, nightly if it doesn’t cause too much noise, and especially before every major release. This ensures remediation happens before any public exposure.
Layer 6: Human security review
It is expensive, yes. But it’s still required, unambiguously. No scanner has replaced a security engineer reading the difference on a payment flow, reviewing authentication logic, or evaluating the trust boundaries of an agentic workflow. An automated loop reduces risk, but it doesn’t and can’t eliminate the need for human security judgement. Large, AI-generated differences are where vulnerabilities hide, because no one will read them thoroughly enough without deliberate enforcement.
Layer 7: Runtime defenses
Threats can be hard to completely identify and eradicate, and they sometimes get through your defenses. Monitoring, anomaly detection, and incident response are the last line of defense. In an AI-enabled development environment, they are more necessary than ever. Runtime AI systems introduce live attack surfaces that don’t exist in traditional deployments, such as prompt injection at inference time, model supply chain risks, and agentic systems with real-world permissions that can be manipulated into destructive behavior.
I’m a seasoned practitioner with over 30 years of cybersecurity, compliance, and application security experience, but even when an expert such as myself uses AI to generate extensive tests, conducts deliberate security reviews at every stage, and makes genuine efforts to apply responsible AI coding practices, vulnerabilities can slip through. My production application in one case produced more than 200 security findings when subjected to traditional AppSec scans. That’s not an argument against AI coding tools. It’s a definitive argument against treating AI-assisted development as a substitute for AppSec scanning.
Your strategic imperative: AppSec must evolve at the speed of AI
The organizational response can’t stop at tooling. Your DevSecOps practices need to evolve in parallel with how rapidly AI is being adopted across your development teams.
- The shift-left imperative is sharper now. Given how quickly AI can generate code, real-time security scanning within the IDE has become a critical first line of defense. IDE-based security scanning helps minimize the flood of insecure artifacts being passed downstream as AI coding assistants generate at unprecedented speed.
- End-to-end visibility is not negotiable. Development teams, DevOps teams, and AppSec teams all need a clear line of sight into security status across every stage of the SDLC. That ranges from the developer’s desktop to build and CI pipelines to runtime. Without that visibility, you can’t manage risks effectively. The window for risk detection shrinks as AI development velocity increases, and organizations that fail to maintain visibility will find themselves managing incidents instead of preventing them.
- The security mindset across your development culture must actively combat implicit trust in AI output. The notion that AI-generated code is inherently better than developer-written code presents business risk. We simply can’t guarantee that it will improve over time without human intervention. Your developers must be empowered to challenge AI output, verify results, and apply skepticism proportionate to the stakes. The Stanford study proved it: The participants who trusted AI less produced more-secure code. Build a culture that treats AI output as a starting point requiring verification, not a finished product requiring only deployment.
The bottom line is that AI coding assistants are not the problem. Using them without discipline—without code reviews, automated testing, and security scanning—is the problem. You can build with AI, and you should. But you need to build with the guardrails that match the threat AI introduces.
Trust your AI. But verify everything it produces.
Download the Secure AI Coding Checklist
Get a practical checklist of security controls, testing requirements, review processes, and governance best practices to help development teams use AI coding assistants safely and effectively.
