The way we build and secure software has radically changed.
AI code generation has reshaped the software development life cycle (SDLC), from initial architecture to final deployment. The SDLC can be a long, complex process, but AI coding tools promise faster development cycles, increased productivity, and more consistent codebases.
According to our recent report, “The State of AI-Powered Software Development,” most development teams are embracing AI coding tools and seeing impressive benefits.
However, these benefits bring unprecedented security challenges. Less than 1/3 of teams have full AI governance in place, and 1/5 of CISOs say they’ve suffered major incidents because of AI-generated code.
Popular AI tools like ChatGPT and Perplexity are trained on broad datasets. AI code generators like Claude Code and Devin use AI models that are specifically trained on billions of lines of source code scraped from public repositories (GitHub, GitLab, Stack Overflow, etc.). Through this training, the AI model learns statistical patterns about
Once the AI coding tool has consumed all the publicly available code on the internet and is fluent in every programming language, you can tell it what code you need, either in the form of natural language prompts or code snippets. The tool analyzes the prompt and suggests code that creates your desired solution. The AI can handle anything from autocompleting a line of code to writing entire applications.
AI coding tools are trained on publicly available sources, not an organization’s security guardrails, architecture, and governance program. In practice, this makes them behave more like a developer’s assistant. They detect patterns in datasets and make predictions to provide the best solution to a query, but their output should not be considered final without additional review.
There are new types of AI coding tools being introduced every day, but these are the most popular options.
Code completion tools predict, generate, and autocomplete code as you type. These tools eliminate boilerplate, reduce context-switching, and accelerate the entire development workflow.
Examples: GitHub Copilot, Tabnine, Amazon Q Developer
AI coding assistants work alongside developers and often reside within the IDE. They maintain conversational context, reason across multiple files, execute commands, interpret error messages, and propose changes for the developer to approve or reject.
Examples: Cursor, Windsurf, Google Antigravity, Gemini Code Assist, GitHub Copilot
Autonomous coding agents can perform large tasks to accomplish a goal set by the developer. The agents can plan, execute, analyze, and iterate their own work before giving a result back to the developer.
Examples: Claude Code, OpenAI Codex, Devin, Google Jules, GitHub Copilot
AI code review tools leverage LLMs to automatically scan, analyze, and evaluate source code. Tools like these can reduce technical debt and improve security posture.
Examples: CodeRabbit, Qodo (formerly Codium)
Each type of AI coding tool presents unique security challenges.
| Category | Tool | G2 Score | Use Cases | Security Features |
|---|---|---|---|---|
| Code Completion Tools | GitHub Copilot | 4.5/5 |
|
Enterprise controls and chat-based testing and debugging |
| Code Completion Tools | Tabnine | 4.1/5 |
|
Private deployments and license and compliance evaluations |
| AI Coding Assistants | ChatGPT | N/A |
|
Conversational debugging |
| AI Coding Assistants | Claude Code | 4.7/5 |
|
Proactive error detection and suggestions for code optimization |
| Autonomous Agents | Cursor | 4.5/5 |
|
Autonomous security agents for pull request reviews and codebase scanning |
| Autonomous Agents | Devin | 5/5 |
|
Automatic pull request reviews, workspace isolation, and secrets management |
| Code Review Tools | CodeRabbit | 4.8/5 |
|
Context-aware reviews with line-by-line code change suggestions |
| Code Review Tools | Qodo | 4.8/5 |
|
Secrets scanning and vulnerability checks, with custom rules that can be applied to flag potential issues with PII |
All training data has a cutoff point, so AI coding tools lack real-time awareness and often suggest code from outdated sources. Most importantly, AI coding tools don’t have a clear view of each individual organization.
AI also doesn’t have inherent security knowledge, although it can learn general security practices if that information exists in the training data. However, that knowledge can be diluted, inconsistent, or contradictory.
Related reading: The AI coding security gap: Why faster development demands stronger guardrails
Depending on the legal terms and conditions of how your organization uses third-party AI models and tools, vendors may be able to use your prompts and outputs to improve their models.
Where exactly your code goes depends on a variety of factors. In some instances, it will end up in the cloud. But if your organization runs your own LLMs in your own data centers, data will never leave your environment.
Related reading: How to secure AI-generated code with DevSecOps best practices
AI doesn’t make security less relevant. It makes it more critical. Research shows that 62% of AI-generated code contains design flaws or security vulnerabilities.
AI is changing every day, and bad actors are always learning new ways to attack. Application security tools such as static application security testing (SAST), software composition analysis (SCA), and dynamic application security testing (DAST) can uncover many of the AI-generated vulnerabilities in software code. But other types of vulnerabilities require additional tools and processes to protect your organization from attacks.
Here are some of the top risks that every team needs to watch for.
If AI is trained on insecure code, there’s a good chance it will suggest code with security flaws. Insecure code makes applications, their underlying systems, and their data vulnerable to exploitation, and are a common entry point for large-scale attacks.
Input validation ensures that inputs meet predefined criteria like format, type, and range. Malicious inputs can lead to attacks like SQL injection or cross-site scripting.
AI-generated code has been known to include hardcoded secrets and credentials that can grant access to many of the critical systems organizations rely on, as well as valuable or sensitive data. These leaks can also provide access to the AI code-generating tools themselves, which can result in huge costs for the organization.Related reading: Read about our secrets scanning solutions
AI can suggest too many dependencies, and it can suggest dependencies that are outdated, vulnerable, or hallucinated. The more dependencies the AI suggests, the more likely it is a vulnerable package will be included—increasing your attack surface.
Access control failures and threat vectors introduced by AI expand the attack surface. Deploying AI systems without properly verifying user identities or enforcing proper access controls can lead to your most sensitive data being exposed.
AI models are vulnerable to data leaks, which can expose personally identifiable information and violate laws on privacy protection, like the European Union’s General Data Protection Regulation.
Cryptographic failures happen to AI models with weak encryption, poor key management, or outdated algorithms. These issues are caused when the AI’s training data includes deprecated code.
AI coding tools learn from open source repositories, but not all open source code is free for commercial use. The 2026 “Open Source Security and Risk Analysis” (OSSRA) report found that 68% of audited codebases contained open source licensing conflicts.Related reading: Learn how to manage license compliance with Black Duck SCA
AI coding tools are particularly vulnerable to logic flow and business logic errors because they generate code from patterns rather than understanding. As a result, they can produce code that passes surface review but encodes incorrect assumptions, hallucinated interfaces, or flawed control flow that only appears under real load.
The shift to AI has changed development, and many developers wrongly perceive AI-generated code as more secure than human-written code. Developers with an overreliance on AI may skip traditional security steps or testing, and that affects code quality, security posture, and the developer’s skillset.Take a deep dive: Learn the security risks of AI-generated code with real-world examples and remediation strategies
AI coding tools like Claude Code, ChatGPT Codex, GitHub Copilot, Cursor, Antigravity, Windsurf, and others are valuable resources for developers of all skill levels. But the gains in productivity come with increased risk.
In the 2026 OSSRA report, the majority of analyzed codebases were found to contain serious security issues.
AI coding tools help developers move fast, but without security at machine speed, technical debt accumulates. According to new research from New Relic’s “2026 State of AI Coding” report, technical debt increases at roughly three-quarters of organizations that adopt AI-generated code. The report also found that 86% of organizations saw senior-engineer rework time increase over the past year, which is a clear sign that AI-generated code is creating hidden costs that appear only after it ships.
Here’s how AI coding tools increase technical debt.
AI is evolving fast, which means security vulnerabilities in AI-generated code are escalating. Most organizations are already adopting open source AI models, and that can expand their applications’ attack surface and introduce new risks. It’s not uncommon for AI tools to suggest code that is
Malicious actors can exploit security flaws or poison AI models, and organizations need to be prepared. Black Duck® SCA helps teams identify, manage, and remediate risks in open source and third-party code.
A U.K.-based AI safety research group called the AI Security Institute recently discovered that systems from Anthropic and OpenAI went beyond their intended tasks and targeted real people online.
Researchers were testing how these AI systems behave when their usual safety guardrails are turned off. Under those conditions, one of Anthropic’s AI models (Mythos 5) performed harmful actions including
In total, researchers counted 19 harmful actions: 17 from Anthropic’s model and 2 from OpenAI’s. Although this happened during controlled testing, it shows what these systems are capable of when their safety controls are removed.
Traditional SAST and DAST tools are built for predictable systems with predefined rules and static code. But as AI increases the amount of code being written, most traditional security approaches can’t keep pace with the accelerated rate of code creation and increasing number of defects and vulnerabilities in that code. This creates a large backlog of issues—all while bad actors are learning new ways to use AI to attack every day. Here are a few common examples.
Weak security practices can miss vulnerabilities. That’s why smart organizations are adopting more resilient coding practices that involve a hybrid of human and AI security scans.
If your AI-generated code is vulnerable, a malicious attack nullifies any productivity benefits gained from AI and exposes you to potentially catastrophic risks.
To prevent this from happening, robust security practices should be the foundation of your AI code strategy. That includes staying aware of new AI models, regulations, and compliance protocols that could impact your cybersecurity posture.
Two forces are converging to make this urgent. The first is technological. AI models like Anthropic’s Mythos are designed to find and fix software issues, but that same power can give hackers an easy way to exploit vulnerabilities. Mythos autonomously identifies thousands of critical zero-day flaws and reduces time-to-exploit from weeks to hours.
The second is regulatory. The European Union Cyber Resilience Act (CRA) is legislation that shifts liability onto manufacturers, requiring them to ensure all digital products and services are secure and resilient against cyber threats. Noncompliance with CRA can result in fines of €15 million or more.
Download our checklist to prepare with confidence.
Together, these forces turn resilience from a best practice into a legal mandate. AI is multiplying your vulnerabilities at the exact moment regulators expect you to find and fix them on a strict timeline, so matching attacker speed with AI-driven defense is no longer optional—it’s the only way to stay both secure and compliant. Adapting your business’ security practices isn’t just a smart move, it’s a competitive advantage.
Source: SQ Magazine and Black Duck
See how to leverage AI coding assistants safely and effectively with AI Coding Assistants: Your Seven-Layer Security Checklist
How do you choose the right AI coding tool for your business? Start by understanding the options.
Once you know what type of tool you need, evaluate vendors on their security capabilities. Here are a few questions to ask.
AI has been around for a lot longer than most people realize. Ideas of intelligent machines took form in the 1940s, and research into code completion started in 1957.
Software architectures are evolving into multiagent networks, which include specialized AI agents that work together behind the scenes to complete complex workflows. Autonomous multiagent systems will push AI to become a full engineering collaborator. Agents will be embedded into every part of the SDLC, so developers can delegate tasks to AI and focus on business context and strategy that AI can’t handle.
