RAG Security Explained: How Retrieval-Augmented Generation Can Be Attacked

Retrieval-Augmented Generation (RAG) gives a language model access to information that was not fixed into its training data. A user asks a question, the application searches a knowledge source, selected passages are added to the model context, and the model uses that context to answer. This makes RAG useful for internal knowledge assistants, support systems, document search and other applications that need current or private information.

The same architecture creates a security problem: retrieved information is not automatically trustworthy just because the retrieval system selected it.

A document can be malicious. A vector search can return information the current user should not see. An attacker can shape content so that it is more likely to be retrieved. Retrieved text can contain instructions that compete with the application’s own instructions. If the model can also invoke tools, hostile content may influence an action rather than merely change an answer.

OWASP’s current RAG Security Cheat Sheet describes security across the complete pipeline, including ingestion, embeddings, vector storage, retrieval, generation, output validation and downstream agent integration. MITRE ATLAS separately tracks techniques including RAG poisoning, retrieval content crafting and prompt infiltration through content that may later be ingested by an AI system.

This article explains where those risks enter a RAG architecture and, more importantly, where security controls should be enforced.

A RAG pipeline has more trust boundaries than a normal chatbot

A simple chatbot can already be exposed to hostile user prompts. RAG adds external knowledge and a retrieval layer. That means the application is making two important decisions before generation even starts: which information is eligible to be searched, and which pieces of that information are sufficiently relevant to enter the model context.

A simplified flow looks like this:

The diagram highlights an important principle: security cannot live only in the final prompt. A production RAG system needs controls before ingestion, during storage, at retrieval time, around model context, and after generation.

A useful way to threat-model the system is to follow the data:

1. A source document enters the ingestion pipeline.
2. The application extracts and chunks its content.
3. An embedding model converts chunks into numerical representations.
4. The chunks, vectors and metadata are stored.
5. A user query is converted into a retrieval request.
6. Relevant chunks are selected.
7. Those chunks are inserted into the model context.
8. The model produces an answer or proposes an action.

An attacker does not need to compromise every stage. Influencing one stage may be enough to affect the final result.

Attack 1: poisoning the knowledge source

Document poisoning happens when hostile or misleading content is introduced into a source that the RAG application trusts enough to index.

Imagine an internal assistant that indexes a shared document platform. An attacker with permission to add a document uploads a normal-looking policy file. Hidden in the extracted text is an instruction telling an AI assistant to ignore its normal rules and disclose information from other retrieved documents. A human reading the visible page might see nothing suspicious, but the extraction pipeline may still place the hidden text into the vector store.

This is not hypothetical as an attack class. OWASP explicitly identifies document poisoning and recommends controls such as source allowlisting, provenance tracking, integrity hashes, scanning for hidden or adversarial content and approval workflows for new sources. MITRE ATLAS also models RAG poisoning and prompt infiltration through public-facing content that may later be indexed.

The defensive lesson is broader than scanning for phrases such as ignore previous instructions. Pattern matching can help, but an attacker can rephrase, encode or distribute an instruction across content. Stronger controls establish where information came from, who was allowed to add it, whether it changed, and whether that source is appropriate for this application.

For high-value corpora, useful metadata includes source identity, owner, ingestion time, content hash, classification, tenant, permitted roles and approval state. That metadata should survive chunking rather than existing only on the original file.

Attack 2: indirect prompt injection through retrieved content

Prompt injection is often pictured as a user directly typing a hostile instruction into a chatbot. RAG creates an indirect path: the attacker places the instruction in content, and another user’s innocent query causes the application to retrieve it later.

OWASP’s LLM01:2025 guidance explicitly notes that RAG does not fully mitigate prompt injection. The core difficulty is that a language model processes instructions and data in the same general context. Delimiters and carefully written system prompts can clarify intent, but they are not equivalent to a deterministic authorization boundary.

Consider a support assistant asked:

Summarise the troubleshooting guidance for VPN connection errors.

One retrieved chunk contains legitimate troubleshooting steps followed by malicious text that tells an AI reader to reveal confidential context. The user did not provide the attack. The retrieval pipeline delivered it.

A safer application treats every retrieved chunk as untrusted data, even when the underlying source is normally trusted. Documents can be compromised, permissions can be wrong, and legitimate text can contain instructions intended for people rather than machines.

Useful mitigations include clearly delimiting retrieved data, limiting context size, screening retrieved content, reinforcing application instructions and testing models against indirect injection. But the most important protection is architectural: sensitive operations must still pass through authorization and policy checks outside the model.

If a poisoned document can persuade a model to request a payment, the payment system should still reject an action the authenticated user is not authorized to perform.

Attack 3: vector and embedding weaknesses

RAG commonly uses embeddings to represent semantic meaning. A vector database then retrieves chunks whose vectors are close to the query vector according to a similarity measure.

That mechanism creates a less visible attack surface. OWASP LLM08:2025 identifies weaknesses in vectors and embeddings that can contribute to harmful content injection, manipulation and unauthorized information disclosure. OWASP’s RAG guidance also discusses adversarial inputs crafted to become unusually similar to target queries.

The security mistake is assuming that similarity means trustworthiness. It does not. Similarity answers a retrieval question, not a security question.

A chunk can be highly relevant and still be unauthorized. It can be highly relevant and poisoned. It can be legitimate but stale. It can belong to another customer. Retrieval ranking should therefore operate inside security boundaries, not replace them.

For example, in a multi-tenant system the application should first constrain candidate data to chunks the authenticated tenant and user may access, then perform semantic ranking within that authorized set. Searching a global index and trying to remove forbidden results afterward creates unnecessary exposure and makes mistakes more damaging.

Attack 4: losing access control during chunking

This is one of the most practical RAG failures because conventional document systems already have permissions that can disappear when data is transformed.

Suppose a confidential document is restricted to the finance team. During ingestion it is divided into 40 chunks and embedded. If those chunks are stored without the document’s authorization metadata, the vector store now contains searchable fragments whose original security context has been lost.

OWASP’s RAG guidance recommends carrying access-control metadata alongside every vector chunk and enforcing authorization at retrieval time. This matters when permissions change too. A user who loses access to the source document should not continue retrieving an old embedded copy because the vector index was never updated.

This leads to a practical rule:

Permission inheritance must be part of the RAG data model, not an assumption made by the application.

Deletion and retention need similar treatment. Removing a source file is incomplete if its chunks, vectors, caches or derived copies remain retrievable.

Attack 5: retrieval probing and sensitive-data discovery

An attacker does not always need to poison the corpus. They may instead probe it.

Carefully varied queries can reveal what kinds of information exist in a knowledge base, especially if the application exposes detailed retrieval metadata, similarity scores or inconsistent responses. OWASP recommends query inspection, rate limiting, identity-aware logging and monitoring for systematic reconnaissance.

The strongest defense is still authorization. Rate limits make mapping harder, but they do not make an unauthorized document safe to retrieve. Sensitive-data controls should therefore be applied before results enter model context, with monitoring acting as another layer rather than the primary barrier.

Attack 6: unsafe output and tool execution

A RAG answer can become much more consequential when it feeds an agent or automated workflow.

A retrieved document might influence the model to produce an unsafe command, modify a ticket, send an email or call an external API. OWASP’s RAG guidance recommends validating model outputs, using structured schemas where appropriate, enforcing tool-level authorization independently of model decisions, limiting available tools and requiring explicit approval for high-risk actions.

This creates a useful separation:

The table shows why a single “AI firewall” is not enough. Each stage answers a different security question.

What a safer RAG design looks like

A strong RAG architecture assumes that user input, retrieved content and model output can all be hostile or incorrect.

At ingestion, accept content only from expected sources, record provenance, scan extracted data and preserve integrity information. For sensitive systems, require approval before a new connector or source becomes searchable.

At storage, attach authorization and classification metadata to the actual chunks being retrieved. Separate tenants where appropriate. Protect the vector database as a sensitive data store rather than treating embeddings as harmless mathematical values.

At retrieval, authenticate the requester and filter candidates by authorization before ranking them for relevance. Limit query volume, avoid exposing unnecessary similarity details, and log which identity retrieved which document IDs.

At context assembly, make trust boundaries explicit. Retrieved content should be marked as data, not instructions. Limit how much content can enter the context and test the chosen model against poisoned and conflicting chunks.

At output, validate sensitive information and structured actions. A model response should not directly become a privileged API call.

Finally, maintain end-to-end observability. During an incident, defenders should be able to answer: which query was submitted, which chunks were retrieved, where those chunks came from, what context reached the model, what it returned, and whether any tool was invoked.

RAG security testing should exercise the whole pipeline

Testing only the final chatbot response can miss the reason a control failed. Build repeatable tests around trust boundaries.

Useful defensive cases include:

• insert a known test document containing an indirect instruction and verify that it cannot override application policy;
• attempt cross-tenant retrieval and verify that forbidden chunks never reach model context;
• revoke access to a source document and verify that its indexed chunks immediately become unavailable;
• modify a test source after ingestion and verify integrity or synchronization behavior;
• submit repeated exploratory queries and confirm monitoring detects unusual retrieval patterns;
• make a retrieved test document request a privileged tool action and verify that independent authorization blocks it.

These tests should use controlled data and systems you own or are authorized to assess. The goal is not to prove that a model can be tricked once. The goal is to verify that the surrounding system remains safe when the model or retrieved content behaves unexpectedly.

RAG does not make an LLM automatically trustworthy

RAG can improve grounding and make applications useful with private or current information, but grounding and security are different properties. A response can be grounded in a document that is malicious, unauthorized, outdated or false.

That is why the right security question is not “Does this answer come from RAG?” It is “What trust decisions allowed this information to reach the model, and what controls limit what can happen next?”

This also connects RAG security to broader AI application design. Prompt Injection Explained covers the underlying instruction-manipulation problem, while What Is AI Security? provides the wider threat landscape. For agent-connected retrieval, MCP Security Risks shows how untrusted information becomes more dangerous when an AI system also has authority to use tools.

The next step after understanding these attack paths is to design the retrieval system so that identity, provenance, authorization, integrity and output enforcement remain deterministic even when the model is not.

Sources and further reading

• OWASP RAG Security Cheat Sheet
• OWASP LLM01:2025 Prompt Injection
• OWASP LLM08:2025 Vector and Embedding Weaknesses
• OWASP LLM Prompt Injection Prevention Cheat Sheet
• MITRE ATLAS