Enterprise data readiness for AI means getting your sources into a state an AI system can use: the right documents, current versions, clear owners and access rules. On top of that we implement retrieval-augmented generation (RAG) that answers from cited sources, filters by each user's permissions, is measured against real questions, and removes content when the source is deleted.
Key takeaways
- Most AI pilots stall on data, not on the model: stale versions, unclear owners and missing permissions.
- Begin with the questions users will ask, then pick the handful of sources that can answer them.
- Measure retrieval with a fixed set of real questions before you tune prompts or change models.
- Every indexed chunk should carry its source, version and access rules, so search can filter and deletion can reach it.
- For facts that change often, retrieval usually beats fine-tuning: you update a document, not a model.
A document assistant can only be as good as what its search turns up. Most enterprise pilots look impressive in a demo and then slow down on real data: five versions of the same policy, contracts in scanned images, customer records that contradict each other, and no clear rule about who may see what. The model is rarely the problem. The data underneath it usually is.
Our data readiness work fixes that layer, then builds the retrieval system on top, so the answers come from the right source, for the right user, and stay correct as the sources change.
Why AI projects stall on data
The same few problems come up in most organisations:
- Too many sources. Teams index every drive they can reach, and the useful documents get buried.
- No authoritative version. Drafts and superseded files rank alongside the current one.
- Broken structure. Tables flattened into text, scanned pages with no text layer, slides with the meaning in the images.
- No owner. Nobody can approve a source for AI use or say when it goes out of date.
- No permissions in search. The index ignores who is allowed to see a document, so an assistant can surface one team's files to another.
- No measurement. Without a test set of real questions, every change is judged by feel.
How we get your data ready
We take one use case at a time and follow five steps. Each step leaves something your team keeps and can repeat for the next use case.
The sources that matter
Quality fixed, personal data tagged
Owners, catalogue and access rules
A vector store that checks permissions
Delete from the index when the source goes
- Find. We collect the questions users will really ask and follow each one back to where its answer is kept. Each source is recorded with its system, owner, size, update rhythm and sensitivity. Anything that brings risk but no answers is left out.
- Clean and label. Duplicates and outdated versions go, the live version is flagged, broken structure is repaired, and personal and confidential content is tagged so it can be masked or excluded.
- Govern. The approved sources go into a catalogue with owner, sensitivity, access rules and retention. The pipeline only ingests what the catalogue allows.
- Index with permissions. We build the chunking, embedding and indexing pipeline. Every chunk carries its source ID, version, effective date and the permissions copied from its source, and the search filters by the user's rights before anything reaches the model.
- Forget. Changes and deletions in the source flow through to the index and to any answer caches, so a removed document stops being retrievable.
RAG implementation choices we make with you
A retrieval system has more moving parts than a prompt. These are the decisions that most affect answer quality:
- Chunking. Split by document structure (sections, clauses, table rows) rather than fixed lengths where the documents allow it.
- Hybrid search. Combine keyword and vector search, because product codes, names and clause numbers often match better on exact terms.
- Metadata filters. Department, region, product, effective date and access group narrow the search before ranking.
- Citations. Every answer links to the passages it used, so users can check and owners can fix the source.
- Evaluation. A fixed question set, scored for retrieval and for answer quality, run on every change.
- Freshness. Scheduled or event-driven sync, with a visible last-updated date per source.
Before launch, the system should be tested by people trying to make it misbehave. Our AI red teaming service covers that, and AI governance covers the approvals and monitoring around it. Where the sources hold personal data, the tagging builds on our privacy engineering inventory.
What you get
| Deliverable | What it is | What you do with it |
|---|---|---|
| Source inventory | Sources for the use case, owners and in or out decisions | Agree scope; extend to the next use case |
| Cleaning and labelling jobs | De-duplication, version marking, structure repair, sensitivity tags | Re-run as sources change |
| Catalogue entries | Owner, sensitivity, access rules and retention per approved source | Decide what may be indexed |
| Retrieval pipeline | Chunking, embedding, hybrid index, permission filters and citations | Run and extend in your environment |
| Evaluation set and report | Real questions with expected sources, scored for retrieval and answers | Judge every future change on evidence |
| Sync and deletion jobs | Change and removal events flowing to index and caches, with test results | Keep the index current and clean |
How to start
Bring one use case, such as a helpdesk assistant, an HR policy search or a bid library, and a rough list of where its answers live. We will scope the sources, agree the evaluation questions with your users and plan the first build. To begin, get in touch.
The DPDP side of this
Our DPDP practice, DPDP Ready, has a detailed guide on how this work maps to India's data protection law. Read it on DPDP Ready →
Frequently asked questions
How do you measure whether a RAG system is working?
Build an evaluation set of real questions from the people who will use the system, each with the source that should answer it. Then measure two things separately: whether retrieval found the right source, and whether the answer used it correctly and cited it. Re-run the set whenever you change chunking, the embedding model, the prompt or the model.
RAG or fine-tuning: which is better for company knowledge?
For knowledge that changes, such as policies, prices, product details or customer records, retrieval is usually the better fit, because you update the source and the next answer reflects it. Fine-tuning suits teaching a model a style, a format or a narrow task. Many production systems use retrieval for facts and keep fine-tuning for behaviour, if they use it at all.
Why does our AI assistant give confident answers from outdated documents?
Usually because superseded versions were indexed alongside current ones and nothing tells the search which is authoritative. The fix is in the data: mark the current version, exclude or demote old ones, record effective dates in metadata and re-sync when documents change. Prompt changes alone rarely solve it.
What data should not go into an enterprise AI assistant?
Anything the use case does not need, and anything you cannot later correct or remove from the index. Credentials and secrets should never be indexed. Personal data should be masked where the task does not require it, and each source should be approved by its owner before ingestion.
Do we need a data catalogue before starting an AI project?
You do not need a full enterprise catalogue. You do need a record, for the sources in scope, of the owner, sensitivity, access rules and retention. We usually build that for the first use case and let it grow from there.
Scope it with us
A 30-minute call to understand your systems and what good looks like. We aim to send a written, fixed-scope proposal within three working days.
Related services
This page describes engineering and security services. Where it mentions laws, rules or standards, it is general information, not legal advice.