Adding an LLM without leaking customer data
Your first enterprise customer will ask where their data goes when your AI feature runs. Here is how to have a good answer before they ask.
The demo goes well, and then the security questionnaire arrives. Where does our data go when your AI feature runs? Is it used for training? Can another customer’s prompt pull our records? Who can see the logs?
If the honest answer is “we’re not sure”, that deal slows down, and it should. An LLM feature adds new ways for data to leave your system. None of them are hard to close, but you have to close them on purpose.
Know where every request goes
Start with a simple map. For each AI feature, write down:
- Which provider and model the request goes to, and in which region.
- What data goes in the prompt: user input, retrieved documents, account details, past messages.
- What the provider keeps, for how long, and whether it can be used for training. Read the API terms, not the consumer product’s terms. They are often different.
- Where you log prompts and responses on your side, and who can read those logs.
This one page answers most of a security questionnaire. It also tends to reveal the problem: the prompt usually carries more than the feature needs.
Send less
The safest data is the data you never send. Before a request leaves your system:
- Only include the fields the task needs. Summarising a support ticket rarely needs the customer’s billing address.
- Mask or replace identifiers where the model doesn’t need them: emails, phone numbers, account numbers. Swap them back in after the response if you have to.
- Keep secrets out entirely. API keys, passwords and tokens should never reach a prompt, even by accident through a pasted log.
Enforce permissions before retrieval, not after
If your feature retrieves documents to answer questions, the most dangerous bug is retrieving something the user isn’t allowed to see. Once it’s in the prompt, the model may repeat it, and no instruction will reliably stop that.
So filter by tenant and user permissions inside the retrieval query itself. Treat “the model was told not to mention other customers” as a hope, not a control. Then add evals that try to fetch another tenant’s data and make sure they fail.
Assume the input is hostile
Anything a user or a third party can put in front of the model can contain instructions: a support email, an uploaded PDF, a web page your agent reads. This is prompt injection, and there is no complete fix for it yet. You limit the damage instead:
- Give the model the smallest set of tools and data the feature needs.
- Require a human or a hard rule before anything irreversible: sending email, issuing refunds, deleting records.
- Validate outputs like any other untrusted input before they hit your database or another system.
Treat logs as production data
Prompt and response logs are incredibly useful for debugging and evals. They’re also a copy of your customers’ data in a new place. Apply the same retention, access control and deletion rules you use for your main database, and make sure a customer deletion request reaches them too.
Write it down for customers
Turn your map into a short, plain page: which providers you use, what data they receive, that it isn’t used for training (if that’s true), and how long you keep it. Update your privacy policy and subprocessor list to match. Being able to send that page in reply to a questionnaire builds more trust than any badge.
The boring checklist
- A one-page data map for every AI feature.
- Prompts carry only the fields the task needs, with identifiers masked where possible.
- Permission checks inside retrieval, with evals that try to break them.
- No irreversible action without a human or a hard rule.
- Prompt logs under the same retention and access rules as production data.
- A plain-language page and a privacy policy that match reality.
If an enterprise deal is waiting on answers about your AI feature, book a 30-minute call and we’ll go through it with you.