Is local RAG private?
The word "local" does a lot of work in that question. Here is what it has to mean for the answer to be yes.
It can be, when all three stages run on your machine: the embeddings that index your documents, the retrieval that searches them, and the inference that writes the answer. Local RAG is private exactly to the extent that none of those stages call a remote service, and that the tool itself sends no telemetry. "Local model, cloud embeddings" is a hybrid, not a local system, and your documents still travel.
What the three stages actually touch.
Retrieval-augmented generation sounds like one thing but is a pipeline, and privacy is decided stage by stage.
Embedding converts your passages into vectors. If this calls a cloud embedding API, every passage in your library has already been sent somewhere else, and it was sent in full. This is the stage most often overlooked, because the chat window feels local even when the index is not.
Retrieval searches those vectors. On a truly local system this is a database query on your disk, and it cannot leak anything because it never talks to anyone.
Inference reads the retrieved passages and writes the answer. A locally served model keeps this on your GPU. A model behind an API sends the retrieved passages, which is to say your documents, as part of every prompt.
If all three stages are local, the pipeline is local, and your sources do not leave the machine during a research turn.
What can still leave.
Two things deserve scrutiny even in a local setup.
Telemetry from the application itself: crash reports, usage pings, update checks that carry identifiers. None of it contains your documents, but a private tool that phones home is a contradiction, and you should be able to see it in a packet capture if it happens.
Web fetches you trigger. A research tool that can read the web sends a request when you ask it to, and the URL you asked for is visible to the site it reaches. That is inherent to fetching, and it is fine when it is your decision, made per fetch, carrying only the URL and nothing else.
Four questions to ask of any tool.
- What happens to a document between import and answer? Every stage should be nameable, and none of them should be a cloud API.
- What does the application contact when I have asked it for nothing? A packet capture answers this better than a privacy policy does.
- Where does the index live, and can I take it? A library you can export as files is a library you control.
- When the tool fetches a web source, what exactly is sent? The URL should be all of it.
Istor's answer to all four: embeddings, retrieval and inference run on your machine, it makes zero network requests unless you trigger a fetch, the library is a SQLite file that exports as a zip, and a fetched source sends only the URL you pointed at.