AI Coding Assistants and Security: What Developers Should Check Before Trusting Generated Code

AI coding assistants can make software development faster by suggesting functions, tests, configuration, documentation and even larger code changes. That speed is useful, but it creates a new review problem: generated code can look polished enough to feel trustworthy before anyone has verified what it actually does.

The safest mental model is simple: treat AI-generated code like code submitted by an unfamiliar contributor. It may be useful, correct and well structured, but it still needs normal engineering controls before it can become trusted application code.

That principle is consistent with current guidance. GitHub’s responsible-use documentation warns that syntactically correct suggestions may still be insecure and tells developers to follow secure coding and code review practices. OWASP’s LLM05:2025 guidance similarly says model output should be treated as untrusted input before downstream use. NIST’s Secure Software Development Framework remains relevant because AI assistance does not remove the need to protect the software development lifecycle.

The main mistake: reviewing whether code works, but not whether it is safe

When a coding assistant generates a solution, the first question is often, “Does this compile?” or “Does this pass the test?” Those checks matter, but they are not enough.

A security-sensitive function can produce the expected output and still be vulnerable. A login endpoint may authenticate normal users correctly while failing to rate-limit attempts. A file upload handler may save an image successfully while trusting the supplied filename. A database query may return exactly the requested record while concatenating untrusted input into SQL. An API route may work perfectly for its intended user while missing an authorization check that prevents access to someone else’s data.

This is why generated code should be reviewed against both functional requirements and security invariants.

A functional requirement says, “A user can download their invoice.”

A security invariant says, “A user can download only invoices they are authorised to access.”

AI-generated code often needs the second statement made explicit. If the prompt contains only the happy-path requirement, a model may reasonably produce code that solves the visible task without implementing the surrounding access-control assumptions.

My article on broken access control explains why this distinction matters in ordinary web applications. The same principle applies when the implementation was suggested by an AI assistant.

A practical trust pipeline for generated code

Instead of deciding whether an AI assistant is “safe” as a product, evaluate each generated change through a controlled pipeline.

The important idea is that AI generation belongs near the start of the development process, not at the end of the assurance process.

A suggestion should not bypass review because it was produced quickly, and it should not gain more trust simply because it came from a well-known model.

Check input handling, especially at trust boundaries

Generated code should receive extra scrutiny wherever untrusted data crosses into a sensitive operation.

Typical boundaries include:

• HTTP request parameters entering database logic;
• user-controlled text entering HTML or templates;
• filenames entering filesystem operations;
• URLs entering server-side fetch functions;
• model output entering shell commands, SQL, code interpreters or workflow tools;
• uploaded content entering parsers or media-processing libraries.

OWASP LLM05:2025 specifically warns about passing model-generated output directly into downstream interpreters and privileged functions. It gives examples involving shell execution, SQL, file paths and browser content.

For an AI coding assistant, there are two layers to consider. First, the generated source code itself may implement unsafe handling. Second, an AI-enabled application may later consume model output at runtime. Those are different risks, but the secure coding response is similar: validate data, enforce context-appropriate encoding, parameterise database operations and avoid sending untrusted strings directly into command interpreters.

For example, prefer parameter binding over constructing SQL from text:

# Safer pattern: keep data separate from SQL syntax.
cursor.execute(
    "SELECT id, email FROM users WHERE email = ?",
    (email_address,)
)

The placeholder syntax differs between database drivers, so developers should verify the correct parameterisation mechanism for the actual library in use. The security principle is to keep untrusted values separate from query structure.

Verify authentication and authorization separately

AI-generated endpoints sometimes look secure because they include a login check. Authentication, however, answers only who the caller is. Authorization answers whether that caller may perform the requested action on the requested object.

Suppose an assistant generates:

GET /api/documents/{document_id}

A correct implementation needs more than “user is logged in.” It needs a server-side decision that the authenticated identity is permitted to access that specific document_id.

Review generated routes for object ownership, role checks, tenant isolation and privilege boundaries. Do not rely on a hidden button, client-side route guard or identifier that is difficult to guess. Security decisions belong on the trusted server side.

This is particularly important when AI assistants generate repetitive CRUD endpoints because small omissions can be copied across several routes.

Inspect dependency suggestions before installing anything

One of the easiest ways to turn a coding suggestion into a supply-chain problem is to copy an unfamiliar package name directly into a package manager.

OWASP’s current LLM guidance warns that models can produce non-existent package names. More broadly, any generated dependency recommendation should be treated as a proposal, not as proof that the package is legitimate or appropriate.

Before adding a dependency suggested by an AI assistant, verify:

1. that the package actually exists in the expected official registry;
2. that the publisher or project identity is what you expect;
3. that the package name is not a confusing typo or lookalike;
4. that the project is maintained and compatible with your environment;
5. that you understand why the dependency is necessary;
6. that version selection follows your normal vulnerability and update policy.

Then let existing software supply-chain controls do their job. Lockfiles, version pinning where appropriate, dependency review, SBOM practices, vulnerability scanning and provenance checks remain valuable regardless of who suggested the package.

For a deeper treatment, see software supply chain attacks and AI supply chain security.

Look for secrets and sensitive data in both directions

There are two separate secret-exposure questions when using a coding assistant.

First, what did you send to the assistant? A developer may paste a configuration file, error log, database connection string or production snippet containing credentials. Whether that is acceptable depends on the organisation’s approved tooling, contracts, retention controls and data-handling policy. Sensitive values should not be casually included just because they are convenient context.

Second, what did the assistant generate? Generated examples can contain hard-coded placeholders that developers later replace with real values, or they can encourage configuration patterns that put secrets directly in source code.

Review for API keys, passwords, tokens, private keys and connection strings before committing. Repository secret scanning is useful, but prevention is better than discovering a credential after it has entered version history.

If a real secret is accidentally committed, removing the visible line is not enough. The credential should normally be rotated because it may remain accessible in repository history, logs, caches or cloned copies.

Do not trust generated cryptography from appearance alone

Cryptographic code is especially dangerous to approve by visual confidence.

An AI assistant may choose an inappropriate primitive, misuse a nonce, generate weak randomness, store keys insecurely or build a custom construction that looks plausible but has no sound security argument. Even when every individual API call exists, the composition may be wrong.

For security-critical cryptography, prefer established high-level libraries and documented patterns from the platform or standard you are implementing. Verify algorithm choices, key lifecycle, randomness requirements, authenticated encryption, nonce rules and error handling against current documentation.

The article on cryptographic failures explains why implementation details such as key management and nonce reuse matter as much as the algorithm name.

Treat generated tests as useful, not independent proof

Asking the same assistant that wrote a function to also write its tests can be productive, but those tests may reproduce the same assumptions as the implementation.

A generated test suite might confirm the happy path while missing abuse cases such as:

• unauthorized object access;
• malformed input;
• boundary values;
• duplicate or replayed requests;
• unexpected encodings;
• race conditions;
• privilege transitions;
• unsafe failure behavior.

Security-focused tests should therefore come from the system’s threat model and requirements, not only from the implementation that already exists.

For a payment endpoint, test more than whether the correct user can pay. Check whether a different user can alter the target account, whether amount values are validated server-side, whether repeated requests are idempotent where required, and whether authorization is enforced before state changes.

AI can help draft those tests, but a developer still needs to decide which security properties must hold.

Use automated scanning, but understand what each scanner proves

Static analysis, dependency scanning, secret scanning and linters are useful gates for AI-generated code because they apply repeatable checks without being influenced by how convincing the suggestion looks.

However, scanners have boundaries.

A static analyzer may detect a dangerous API call but miss a business-logic authorization flaw. A dependency scanner may identify a known vulnerable package but cannot prove that a newly introduced dependency is trustworthy. A secret scanner may catch a token pattern but not recognise sensitive business data embedded in a test fixture.

Use automated tools as layers, not as a substitute for understanding the change.

NIST’s SSDF emphasises secure development practices across the lifecycle rather than relying on one final security activity. AI-assisted development should fit into that same model: prepare the organisation, protect software and development environments, produce well-secured software, and respond to vulnerabilities.

Be careful when the assistant can modify files or run commands

Some coding assistants now operate as agents. Instead of suggesting text, they may read a repository, edit files, run tests, invoke tools or prepare commits.

That changes the threat model.

A chat suggestion has limited direct power. An agent with shell access, package installation rights, cloud credentials or deployment permissions can cause real state changes. The principle of least privilege therefore matters more as assistant capability increases.

Give the coding agent only the repository, environment and credentials required for the task. Prefer isolated development environments for risky operations. Do not expose production credentials merely because the agent can use them conveniently. Require human approval before sensitive actions such as changing infrastructure, rotating data, modifying access controls or deploying to production.

This is closely related to the broader design problem covered in how to build a secure AI chatbot: model capability should not automatically become application authority.

A short review checklist before accepting generated code

Before merging an AI-assisted change, a developer should be able to answer these questions confidently:

1. Can I explain the code? If not, do not merge it merely because it passes a test.
2. What data is untrusted? Identify inputs and apply validation, encoding or parameterisation at the correct boundary.
3. Where is authorization enforced? Check object-level and function-level permissions on the trusted side.
4. Did the change add a dependency? Verify its identity, provenance, version and necessity.
5. Could secrets or sensitive data be exposed? Check prompts, source, logs, fixtures and configuration.
6. Does it use cryptography or authentication correctly? Verify against current authoritative documentation.
7. What happens on malformed or hostile input? Test failure cases, not just expected usage.
8. What automated checks have run? Use SAST, dependency, secret and quality checks that fit the project.
9. What permissions does the resulting code or agent receive? Apply least privilege.
10. Can the change be reviewed and rolled back normally? Keep the standard pull request, audit and deployment process.

This checklist is intentionally ordinary. That is the point. AI-generated code should not create a parallel software-development process with weaker controls.

What AI coding assistants are good at, from a security perspective

Security guidance should not imply that AI assistance has no value.

A coding assistant can help explain unfamiliar code, draft unit tests, suggest safer APIs, identify places worth reviewing, generate documentation, propose refactoring and help developers reason about threat scenarios. It can also reduce the time needed to create repetitive secure patterns when the developer already knows what correct behavior should look like.

The strongest use is often augmentation rather than delegation. Let the model accelerate implementation and exploration, while developers and engineering controls remain responsible for trust decisions.

That approach also avoids a false binary. The real choice is not “AI code” versus “human code.” Human-written code can contain severe vulnerabilities, and AI-generated code can be secure. The meaningful question is whether the development process has enough evidence to trust a specific change.

The security boundary is the review process

AI coding assistants change how source code is produced, but they do not change what production systems require: correct authorization, careful input handling, protected secrets, trustworthy dependencies, sound cryptography, tested failure behavior and controlled deployment.

Generated code becomes trustworthy through evidence, not origin.

A useful workflow is therefore: generate, understand, verify, test, review, then merge. If an AI assistant makes the first step dramatically faster, the answer is not to remove the later steps. It is to make those assurance steps efficient enough to keep pace.

Sources and further reading

• OWASP LLM05:2025 Improper Output Handling
• GitHub Copilot responsible use for inline suggestions
• NIST SP 800-218: Secure Software Development Framework
• NIST SP 800-218A: Secure Software Development Practices for Generative AI and Dual-Use Foundation Models