How do I secure a RAG feature?

Last reviewed 2026-09-30Written by Jacob Masse, TrazTech Inc.

Secure a RAG feature by filtering what is retrieved by the current user's permissions before it reaches the model, treating every retrieved document as possibly containing instructions, escaping the model's output, and logging what was retrieved for each answer. Most RAG leaks are an authorization bug in the retrieval step, not a model failure.

The controls

Store a tenant and access list with every chunk in the vector store.
Filter by tenant and user in the query, not after.
Keep outsider-supplied content (web pages, emails, uploads) separate from trusted content, and label it in the prompt.
Do not give the model tools that act on data while it is processing untrusted content, or require confirmation.
Escape and validate the answer before rendering it; links and HTML especially.
Log retrieved chunk IDs per answer so leaks can be traced.

Pitfalls

One shared index for all customers with filtering only in the prompt.
Documents deleted from the app but not from the vector store.
Embeddings of sensitive fields users should not query.

These fall under LLM08 in the OWASP LLM Top 10.

Testing it

Two tenants with distinctive documents; as one, try to retrieve the other's by question, paraphrase and injection. Plant an instruction in an uploaded document and see if it is followed.

Updates and deletion

Retrieval stores drift out of step with the app. When a document is edited, re-embed it; when it is deleted, delete its chunks; when a user leaves an organization, make sure their access to that organization's index ends; when a customer closes their account, remove their data from the store as well as the database. Each of these is easy to forget, because the vector store is a separate system the AI tool added once.

Test them deliberately: delete a document and ask the assistant about its contents; remove a user from an organization and query as them. Stale retrieval is both a security finding and a privacy one, since people have the right to have their information deleted.

Getting it checked

TrazTech offers AI and LLM security assessments, listed from $4,000 CAD. Get at least one other quote on the same scope; the questions to ask a testing firm help compare them.

Get a scope for your app

Tell us what you built, what it stores and who is about to use it.

Get matched

Common questions

Should each customer have a separate index?

It is the strongest isolation and simple to reason about. Metadata filtering in one index works if it is enforced in the query and tested.

Does deleting a file remove it from RAG?

Only if your code deletes its chunks from the vector store too. Check it, because privacy law gives people a right to have data deleted.