AI Agents and Sensitive Data: What You Need to Secure

67% of employees share internal company data with generative AI tools without authorization. This figure from a 2024 Kaspersky study captures the scale of the problem facing businesses that deploy AI agents without a strategy for securing sensitive data. AI agents—systems capable of acting autonomously on business data—multiply attack surfaces and leakage risks, against a tightening regulatory backdrop shaped by the European AI Act and CNIL recommendations.

This article details the concrete risks your data faces when it passes through LLMs, vulnerabilities documented by OWASP and major reference studies, and the technical and organizational measures needed to secure AI agents without slowing innovation.

TL;DR — AI agents handle sensitive data at scale, often without adequate governance. Breaches cost an average of $5.2 million per incident (IBM). To protect yourself, deploy private instances, implement granular access controls, train teams and prepare for AI Act obligations applying from August 2026.

The Rapid Growth of Enterprise AI Agents: Fertile Ground for Data Leaks

80% of the Fortune 500 Already Deploy Active AI Agents

Enterprise AI agent adoption has moved beyond experimentation. According to the Microsoft Cyber Pulse Report, 80% of Fortune 500 companies use active AI agents in daily operations. Agentic AI—systems making decisions and executing actions autonomously—is present in 67% of organizations, according to the Proofpoint Data Security Landscape 2025 report.

This acceleration brings a worrying development: widely available low-code and no-code tools now let business users without particular technical skills create their own AI agents. These agents access customer databases, financial documents and contractual correspondence without the IT department necessarily approving the data flows.

Shadow AI: A Rapidly Growing Invisible Threat

Shadow AI means using AI tools that the company has not approved. According to the Varonis 2025 State of Data Security report, 98% of companies have employees using unapproved applications, averaging 1,200 unofficial applications per organization.

The consequences are measurable. According to the Netskope Cloud and Threat Report 2026, organizations experience an average of 223 AI-related data policy violations each month. Source code accounts for 42% of incidents, regulated data for 32%. In October 2025, anomalous data uploads to generative AI services increased by 39%, with an average volume of 75 MB per transfer.

17% of companies have no visibility into employees' use of AI tools. In other words, one company in six does not even know which data passes through which models.

Five Concrete Risks of Exposing Data to LLMs

1. Data Leakage to Public Models

The most direct risk is confidential information being sent to LLM APIs hosted by third parties such as OpenAI, Anthropic and Google. According to Microsoft Security Research, 42% of enterprise data leaks in 2024 were traced to public AI services handling sensitive information.

27% of companies admit that more than 30% of information sent to AI tools contains private data: social security numbers, medical records and financial data. Every prompt sent to a public model is a potential leakage point if data is not filtered beforehand.

2. Prompt Injection and Agent Manipulation

OWASP ranks prompt injection first in its 2025 Top 10 LLM risks. The principle: an attacker inserts malicious instructions into data processed by the agent, diverting its intended behavior. An AI agent analyzing emails could be manipulated through a booby-trapped email to exfiltrate customer data.

Agent architecture amplifies this risk. Unlike a simple chatbot, an agent can act: query databases, send messages and modify files. OWASP calls this dimension “excessive agency”: an agent with too many permissions becomes a prime attack vector.

3. Data and Model Poisoning

Data poisoning—the third major risk in OWASP's LLM Top 10—involves corrupting a model's training, fine-tuning or embedding data. Consequences include biased responses, security flaws introduced into generated code and backdoors that alter model behavior through specific triggers.

For a company using a model fine-tuned on business data, this risk is particularly critical. If training data is compromised by a malicious provider, a failing supply chain or an unverified source, every decision the agent makes becomes suspect.

4. System Prompt and Business Logic Leakage

In 2025, OWASP added a specific new risk: system prompt leakage. System prompts often contain the agent's business logic and decision rules, sometimes even API keys or credentials. An attacker who extracts the prompt gains a precise map of internal company processes.

Between January and February 2025, five major LLM-related breaches exposed sensitive data globally, including conversation histories, API keys and access credentials.

5. Vector Vulnerabilities in RAG Architectures

RAG—Retrieval-Augmented Generation—architectures connect an LLM to an enterprise document collection and introduce a specific attack vector. If document access controls are not properly implemented in the retrieval system, users can access documents through the agent that they would normally be unable to view.

OWASP classifies these vector and embedding weaknesses among emerging risks for 2025. The attack surface combines traditional vector database vulnerabilities with LLM-specific risks, a combination few companies know how to audit.

Risk	OWASP LLM 2025 ranking	Main impact	Observed frequency
Prompt injection	#1	Agent hijacking, data exfiltration	High
Sensitive data leakage	#2	Exposure of confidential information	Very high
Data poisoning	#4	Bias, backdoors, corrupted decisions	Medium
System prompt leakage	#7	Exposure of business logic	Medium
Vector vulnerabilities: RAG	#8	Unauthorized document access	Emerging

$5.2 Million per Incident

According to IBM's Cost of a Data Breach 2024 report, an AI-related data breach costs an average of $5.2 million—28% more than a conventional breach. The additional cost reflects complex remediation: identifying which data was exposed, to which model, for how long, and what outputs were generated from it.

The Stanford 2025 AI Index Report confirms this trend with a 56.4% year-on-year increase in AI-related privacy incidents. The trajectory is clear: the more AI agents companies deploy, the more incidents multiply.

The Hidden Cost: Access Debt

The Varonis 2025 report reveals a structural problem: companies average 15,000 inactive accounts that remain operational, with 31,000 obsolete permissions. When an AI agent inherits these permissions through an overly broad service account, it gains access to data nobody should still be able to consult.

This access debt existed before AI, but agents turn it into active risk. An employee with excessive permissions rarely looks at things they should not access. An AI agent systematically uses its full set of permissions on every request.

Regulatory Penalties on Top

The European AI Act, fully applicable on August 2, 2026, introduces penalties of up to €35 million or 7% of global revenue for the most serious violations. GDPR provides for up to €20 million or 4% of revenue. These regimes are cumulative: a data leak through an AI agent can trigger penalties under both simultaneously.

59 new data protection regulations were adopted worldwide in the past year alone. The regulatory environment is becoming more complex, not simpler, and AI agents are on the front line.

AI Agent Governance: An Alarming State of Affairs

86% of Companies Lack Visibility into AI Data Flows

Governance figures in the Proofpoint 2025 report paint a troubling picture:

86% of organizations have no visibility into data flows through their AI tools
55% have no AI governance framework
44% lack adequate oversight of generative AI use
Only 12% have established a dedicated AI governance structure

The gap between perception and reality is striking. While 23% of companies consider themselves ready in AI governance, only 9% actually reach a maturity level described as ready. This 14-point gap between stated confidence and actual maturity reflects collective blindness.

The Human Factor: A Dangerous Skills Gap

According to the Cloud Security Alliance, 52% of executives claim to understand AI technologies, compared with only 11% of operational employees using the tools daily. This creates a paradox: deployment decision-makers overestimate their understanding of risks, while those actually handling the tools lack the skills to identify leaks.

Almost Nonexistent Technical Controls

A lack of governance translates into inadequate technical controls:

Only 17% of companies have technical controls blocking access to public AI tools with Data Loss Prevention (DLP) analysis
40% rely solely on training and audits—necessary but insufficient against systemic risk
13% have no controls at all

From the field — An industrial SME deploying an AI agent to analyze supplier contracts without DLP controls potentially exposes all its commercial terms, margins and contractual commitments to a third-party model. One employee copying a contract into ChatGPT is enough for that data to leave the company's perimeter.

Seven Best Practices for Securing AI Agents

1. Deploy Private Instances Instead of Public APIs

This is the most impactful measure. According to Gartner's 2025 AI Security report, organizations using private AI instances experience 76% fewer data exposure incidents than those relying exclusively on public LLMs.

A private instance—whether a self-hosted open-source model such as Llama or Mistral, or a dedicated cloud deployment such as Azure OpenAI Service, AWS Bedrock or GCP Vertex AI—guarantees that data stays within your perimeter. The infrastructure premium is real, but should be compared with the $5.2 million average cost of a breach.

Approach	Estimated monthly cost	Data control	Risk reduction
Public API: OpenAI, Anthropic	€500–€5,000	Low: data passes through the provider	Baseline
Dedicated cloud deployment: Azure OpenAI, Bedrock	€2,000–€15,000	High: data in your tenant	−60–70%
Self-hosting: Llama, Mistral on private GPUs	€5,000–€30,000	Complete: no data leaves	−76% (Gartner)
Hybrid: public for nonsensitive, private for sensitive data	€3,000–€12,000	Adjustable by classification	−50–65%

2. Classify Data Before Any Deployment

Before connecting an AI agent to your data, you must know which data is sensitive. That seems obvious, yet 86% of companies lack this visibility. Classification should cover four levels:

Public: freely shareable data, such as marketing content and product documentation
Internal: internal-use data without critical consequences if leaked, such as generic meeting notes and organizational charts
Confidential: sensitive business data, including contracts, financial data and sales strategy
Restricted: regulated or critical data, including GDPR personal data, health data and intellectual property

An AI agent should never access Restricted data through a public API. Confidential data requires a private instance. This classification matrix drives the entire security architecture.

3. Apply Least Privilege to Agents

An AI agent should have only the permissions strictly necessary for its task. OWASP identifies excessive agency as a major risk: an agent with read and write access across an entire customer database represents a disproportionate attack vector.

Practical measures:

Create a dedicated service account for each agent with explicit, limited permissions
Apply role-based access control (RBAC) in the RAG retrieval system
Limit authorized actions: an analysis agent should not be able to modify or delete data
Add output guardrails, filtering sensitive information before returning it to the user
Revoke permissions as soon as an agent is disabled; never leave orphaned service accounts

4. Implement DLP Filtering on Inbound and Outbound Flows

Only 17% of companies have DLP controls on AI flows. This is the most underused technical measure. A DLP system adapted to AI agents should:

Analyze prompts before sending them to the model to detect sensitive data such as payment card numbers, credentials and health data
Filter outputs to remove confidential information generated from context
Log every interaction for auditing
Automatically block transfers exceeding a defined sensitivity threshold

Several market solutions now provide these capabilities: Microsoft Purview for Azure environments, Netskope for cloud flow control, and specialist solutions such as Protect AI or Robust Intelligence for LLM pipeline security.

5. Log and Audit Every Agent–Data Interaction

CNIL and European regulators require evidence: logs, audit reports and technical attestations. For every production AI agent, you must be able to answer:

Which data did the agent access?
Which prompts were sent to the model?
Which responses were generated?
Which actions did the agent execute?
Who initiated the interaction?

Logs must be immutable—unalterable after writing—timestamped and retained in accordance with regulatory requirements. For high-risk systems under the AI Act, this traceability becomes a legal obligation from August 2026.

6. Secure RAG Architectures Specifically

RAG architectures, which let an agent query an enterprise document collection, have specific vulnerabilities. The French Directorate General for Enterprise (DGE) guide to RAG recommends several measures:

Document-level access control: retrieval must respect existing Access Control Lists (ACLs); users must not access through the agent any document they cannot view directly
Prompt filtering: detect and block injection attempts aimed at bypassing access restrictions
Vector index segmentation: separate vector databases by confidentiality level to prevent cross-boundary leaks
Query monitoring: watch for abnormal access patterns, such as users querying large volumes of documents outside their usual scope

7. Train Teams and Establish a Culture of Confidentiality

Technology alone is insufficient. 52% of executives overestimate their understanding of AI risks, and 67% of employees already share data without authorization. An effective training program should:

Explain concrete risks using examples from the company's industry
Provide simple rules about which data may be shared with which tools
Establish a nonpunitive AI incident reporting process to encourage reporting
Organize practical exercises: an internal AI red team can test deployed agents' resilience

What GDPR Already Requires for AI Agents

In 2025, CNIL published finalized recommendations on applying GDPR to AI system development. Three foundational principles apply:

Protection by design. Personal data must be protected at every stage: in training datasets, within models that have memorized it, and during model use through prompts. Privacy by design applies from the choice of technical architecture.

The right to be informed. When personal data is used to train a model or may be memorized, data subjects must be informed. For an AI agent handling customer data, this means updating the privacy policy and, in some cases, obtaining specific consent.

Exercising rights. Rights of access, rectification, objection and erasure remain applicable even when technically difficult to implement with LLMs. CNIL asks organizations to make every effort to incorporate these rights from the design stage.

The AI Act: What Changes in August 2026

The European AI Act becomes fully applicable to high-risk systems on August 2, 2026. AI agents used in certain fields—human resources, credit, health and justice—will probably be classified as high risk and must comply with:

Registration in the European AI systems database
CE marking before marketing or deployment
A documented risk management system
Complete technical documentation
Operational human oversight of agent decisions

Penalties are significant: up to €35 million or 7% of global revenue for the most serious violations, cumulative with GDPR penalties.

Gartner Predicts 40% of Breaches Will Be Cross-Border by 2027

Gartner anticipates that 40% of AI-related data breaches will arise from poorly controlled cross-border generative AI use by 2027. For a French company using a US-hosted model with European customer data, regulatory risks accumulate: GDPR for transfers outside the EU, AI Act compliance obligations and potentially local regulations in third countries.

Security Checklist: 12 Questions Before Deploying an AI Agent

Practical guide — Pre-Deployment Assessment

☐ Where is the data processed by the agent hosted: France, EU or outside the EU?

☐ Is the model a private instance or a public API?

☐ Which data categories can the agent access? Has classification been completed?

☐ Do agent permissions follow least privilege?

☐ Is DLP filtering in place for prompts and outputs?

☐ Are interactions logged immutably?

☐ Does the RAG system respect existing document ACLs?

☐ Is human oversight planned for sensitive agent actions?

☐ Has the privacy policy been updated to cover AI use?

☐ Have teams been trained on AI-agent-specific risks?

☐ Is an AI incident response process documented?

☐ Has AI Act compliance been assessed for this use case?

If you answer “no” to more than three questions, pause deployment until the gaps are addressed.

Building a Trusted Architecture: The Layered Model

Securing AI agents requires defense in depth rather than a single solution. Here is a four-layer model in which each layer reinforces the others:

Layer 1 — Infrastructure. Sovereign or dedicated hosting, encryption at rest and in transit, and network isolation for AI environments. This is the foundation: if data travels unencrypted to a public API, no application-level measure can compensate.

Layer 2 — Data. Classification, tokenization of sensitive data before prompt injection, anonymization of personal data and vector database segmentation by confidentiality level. This layer ensures that even if a leak occurs, exposed data is unusable.

Layer 3 — Application. DLP filtering on flows, input and output validation, behavioral guardrails and monitoring of abnormal usage patterns. This is where real-time detection and blocking of exfiltration attempts happen.

Layer 4 — Governance. Documented usage policies, ongoing training, regular audits, incident response processes and regulatory reporting. This turns technical measures into a lasting organizational posture.

According to Gartner, companies applying AI TRiSM—Trust, Risk and Security Management—controls consume at least 50% less inaccurate or illegitimate information, significantly reducing decision-making errors.

FAQ

Are AI agents riskier than conventional chatbots for data security? Yes, significantly. Unlike a chatbot limited to generating text, an AI agent can act: query databases, modify files and send communications. OWASP lists this excessive agency among critical risks. An agent's attack surface is proportional to its permissions.

Can companies use ChatGPT or Claude without risking sensitive data? Not without precautions. Consumer versions do not guarantee that your data will not be used for model training. Enterprise offerings such as ChatGPT Enterprise and Claude for Business provide contractual safeguards, but zero risk does not exist. For confidential or regulated data, private deployment remains recommended: Gartner documents 76% fewer incidents with this approach.

What penalties does the AI Act provide for noncompliant companies? The AI Act, fully applicable in August 2026, provides fines of up to €35 million or 7% of global revenue for the most serious violations involving prohibited AI systems. For noncompliance involving high-risk systems, penalties reach €15 million or 3% of revenue. These are cumulative with GDPR penalties.

How do you secure a RAG architecture without degrading agent performance? Three measures have minimal performance impact: implement access controls at the vector index level rather than the query level, prefilter sensitive documents during indexing rather than retrieval, and use a secure cache for frequent queries. Additional latency is generally below 200 ms, remaining imperceptible to users.

Does CNIL have specific requirements for AI agents processing personal data? CNIL applies the existing GDPR framework with clarifications published in 2025. Key requirements are data protection at every stage—training, memorization and prompt-based use—informing data subjects, and effectively implementing access, rectification and erasure rights even when technically complex. CNIL expects evidence: logs, audit reports and technical attestations.

AI Coder Squad: AI Agents Designed to Protect Your Data, Not Expose It

Securing an AI agent requires preparation. It demands mastery of LLM architecture, regulatory constraints and application security best practices—skills developed in the field, project after project.

AI Coder Squad designs custom applications and AI agents for businesses that want to move quickly without sacrificing quality—with senior developers and an AI-powered approach.

→ Start your project and discover how AI Coder Squad can accelerate your next delivery.