RAG Security: How to Protect the Documents Behind Business AI
Control the documents an AI can retrieve, who can retrieve them, and how retrieved content can affect an answer or action.
RAG security is the practice of controlling which documents an AI system can retrieve, who is allowed to retrieve them, and how retrieved information can influence an answer or action. Retrieval-augmented generation can help an AI use current business knowledge, but it also connects the model to email, files, databases, and client information that require access control.
A RAG system is not secure because its answers include sources. The business must govern the full path from document ingestion to retrieval, output, logging, and deletion.
How RAG works in business terms
A user asks a question. The system searches an approved knowledge source for relevant content. It places selected passages into the model's context, and the model creates an answer.
The quality and safety of that answer depend on several decisions:
- who can add or change source documents
- which sources the system is allowed to search
- whether the user's permissions are checked during retrieval
- how client or department data is separated
- whether the answer shows reliable citations
- what the AI can do with the result
OWASP's RAG Security Cheat Sheet treats document ingestion, embeddings, vector storage, retrieval, generation, output validation, and downstream agent use as one security pipeline. OWASP
Five risks to address
Unauthorized retrieval
The AI may surface a document the user could not open directly if retrieval does not enforce the same identity and access rules as the source system.
Cross-client exposure
A shared index can mix information across clients or business units unless the system applies strong tenant and context separation.
Document poisoning
A person or compromised account may add malicious or misleading content to the knowledge base. The AI can then retrieve that content and treat it as context.
Stale or incorrect sources
Old policies, superseded contracts, duplicate procedures, and unapproved drafts can produce answers that look grounded but are no longer valid.
Sensitive output and logs
The answer, retrieved passages, prompts, and logs may contain confidential information. Protection must continue after retrieval.
A practical control framework
Approve the sources
Maintain a list of repositories the AI can use. Record the owner, data classification, permitted audience, update process, and retention requirement.
Enforce permissions at retrieval
Check the user's identity and rights when the query runs. Do not rely only on access applied when documents were first indexed.
Separate sensitive contexts
Use separate indexes, filters, or enforcement boundaries where client, legal, HR, or regulated information should not mix.
Control document ingestion
Limit who can add content. Scan files, record provenance, flag unusual changes, and require approval for high-trust knowledge sources.
Validate output
Show citations where appropriate, confirm that they support the answer, scan for sensitive information, and require human review for higher-consequence use.
Monitor and respond
Log retrieval sources, access decisions, output, tool calls, and policy violations. Define how to remove a poisoned document, rebuild an index, revoke access, and investigate exposure.
Questions to ask before connecting SharePoint or OneDrive
- Will the AI respect existing Microsoft 365 permissions?
- Can it retrieve externally shared or archived material?
- Are client workspaces separated?
- Who can add documents to indexed locations?
- How are deleted or superseded documents removed from the index?
- Can the business see which sources supported an answer?
- Who responds if restricted information appears?
FAQ
Does RAG make AI answers accurate?
It can provide relevant context, but accuracy still depends on source quality, retrieval, model interpretation, and output review.
Is a vector database the same as an access-control system?
No. Storage and similarity search do not automatically enforce the business's identity, client, and data-handling rules.
What is the first security step?
Inventory the proposed sources and confirm who should be able to retrieve each one before indexing them. RAG should extend approved access, not create a new path around it.
Put this control into practice
Start with one AI workflow that handles sensitive, operational, or client information. Document its source permissions, retrieval boundaries, and access evidence. Test the process with a normal request, an unsafe request, and an error case before expanding its use.
Quantm helps Canadian SMBs connect AI governance with identity, Microsoft 365, cybersecurity, and documented business controls. If your team needs a practical baseline, an AI and Cyber Governance Diagnostic can identify the first control gaps to address.
Related AI security guidance
- AI Security Learning Centre
- Context and Retrieval Control
- AI Data Loss Prevention for Businesses Using Generative AI
- What Is Prompt Injection?