Delete Every Copy of a Document in RAG

A document disappearing from a folder does not prove it has disappeared from an AI knowledge service. The service may hold extracted text, overlapping chunks, embeddings, search results and completed answers. Each copy needs an owner and a removal path.

This is a deployment question for a retrieval-augmented generation service you operate. Chat history and a vendor's retention process are separate stores; the ChatGPT deletion and archive note covers that distinction.

Give the document a durable identity

Start at ingestion. Assign a stable source ID, a version and a content hash. Every derived chunk should retain the source ID and version, together with its own unique key. A file path alone is fragile: a renamed folder can look like a new source while leaving the old chunks behind.

Keep a manifest of what ingestion created. For example, document POL-018, version 4, might produce 23 chunks across one search index and one extraction store. Record those destinations. This is a proposed inventory pattern; it does not depend on a particular vector database.

The manifest makes removal bounded. Without it, deletion becomes a search for strings that may have been paraphrased, split across chunks or stripped during extraction.

Check the connector's actual behaviour

Different ingestion paths have different deletion semantics.

Azure AI Search's storage indexer guidance distinguishes automatic change detection from configured deletion detection. It warns that a deletion policy added after the first run does not repair every earlier orphan. It also lists a limitation for one-to-many indexing scenarios, so do not assume a parent-file rule removes every child produced by your pipeline.

Amazon Bedrock requires a source sync after source changes. Its separate direct document deletion guidance warns that deleting from an S3-backed knowledge base leaves the S3 object intact; another sync can bring it back.

Translate those behaviours into a sequence for your own connector. Record what signals deletion, what consumes that signal and what happens if ingestion is stopped for a week. A retention window that expires before the indexer sees the signal is a failure case to test.

Use a removal job with explicit stages

For an application-managed pipeline, a useful sequence is:

1. Mark the source unavailable for new retrieval. Enforce this in application or index filtering, using the source identity.
2. Stop queued ingestion work for that version, or make workers check its deletion state before writing.
3. Remove the identified chunks, vectors and extracted text through the supported store operations.
4. Invalidate search and answer caches associated with the document or corpus version.
5. Complete the source removal according to the connector's required order.
6. Reconcile the manifest and record unresolved destinations.

Do not describe the first stage as physical deletion. It is containment while the other stages complete. Equally, a successful API acknowledgement is not the same thing as a verified absence from every read path.

If cached answers cannot be traced to their sources, invalidate the affected cache partition. That costs more computation, but guessing which paraphrases contain the removed material is a weak deletion method. Future cache entries should carry source IDs precisely to avoid that problem.

Prove absence through two different checks

Use harmless test documents before relying on the workflow. Give one a distinctive invented sentence and enough content to generate several chunks. Confirm it can be retrieved, then remove it through the normal route.

First inspect the stores directly. Query the source ID and version, count remaining children and check failed deletion operations. This finds records that a top-five semantic search might simply rank too low to show.

Then test the user path. Repeat the original question, a paraphrase and a question aimed at a later chunk. Test with the cache enabled and bypassed. Run another ingestion cycle and repeat the checks to catch reintroduction. Save timestamps because search availability and background cleanup can occur at different times.

The evidence should say exactly what passed: no active indexed children, no retrievable source content through the tested routes, and no reappearance after sync. It should not claim that a few prompts prove universal erasure.

Preserve recovery without silently restoring deleted data

Backups need a separate retention decision. A restore procedure should replay the deletion record before the recovered service accepts queries. Otherwise recovery can reverse weeks of authorised removals.

Keep deletion records minimal: identifiers, destinations, times and outcomes, rather than copies of the removed text. Where a removal is mistaken and restoration is authorised, create a new controlled ingestion version and test it. That preserves a clear history of what was removed and why the content returned.

References

• Microsoft, Change and delete detection using indexers. Accessed 17 October 2026.
• Amazon Web Services, Sync your data with your Amazon Bedrock knowledge base. Accessed 17 October 2026.
• Amazon Web Services, Delete documents from a knowledge base directly. Accessed 17 October 2026.