Every assistant session starts from amnesia. It does not know which client runs which stack, what was decided about backups in 2019 and reversed in 2023, why the monitoring alert thresholds are what they are or that the one server with the odd disk layout is that way on purpose. The person driving the session knows, and spends the first ten minutes pasting context in by hand, session after session, like briefing a brilliant contractor with no memory on their four-hundredth first day.
The industry's answer is a bigger context window. That answer misreads the problem. An organisation's memory is fourteen years of decisions, designs, client history and hard-won operational folklore, most of it irrelevant to any given question and all of it essential to some question eventually. It is not a document you load. You do not want it all in the window. You want the right ten paragraphs, found on demand. The name for that is retrieval, and retrieval is a solved- enough problem to go and build with.
What we have, and what we built¶
We have an unusual asset, built by accident of discipline: years of the company written down as plain markdown in one vault. Decisions, designs, runbooks, meeting notes, client records. No wiki product, no SaaS knowledge base, no export problem. Files in folders, under sync, on machines we own.
So the build is unglamorous, and the first version of it went together over the holidays. The vault lands on a small machine of its own. An embedding model running locally turns each note into vectors, in crude line-range slices. Markdown's headings would make the better boundary, and a disposable index is allowed to start crude. The vectors sit in SQLite next to a full-text index of the same notes, and a query runs both: semantic search for "what do we know about this shape of problem", keyword search for the exact error string, the two result lists fused by rank. An assistant session asks over a standard tool interface and gets back passages, with paths, from the organisation's own memory.
Nothing in that paragraph is novel, and nothing needed to be. The components are a local embedding model, SQLite with a vector extension and a sync tool we already ran. No new database server, no managed vector store, no per-query bill, and nothing about the company's internal history leaves the premises to be embedded on somebody else's machines. Whether this is possible was never in doubt. The question is whether the answers are good enough to change how the sessions run.
Early returns¶
Three weeks in: better than expected, with sharp edges. "Better than expected" rests on three weeks of daily use, not on any measurement. The case we keep retelling is a single one: the session that asks "have we dealt with this vendor's licensing before" gets the 2021 note that a person would have taken twenty minutes to remember existed. Keyword search rescues the queries where semantic similarity goes vague. The failure mode that matters is silent absence: a search that returns nothing proves nothing, because the note may exist under a heading no query reaches. The index is a view of the vault, never the record; when it matters, somebody still greps the files.
The design rules we are holding onto, stated now so we can be judged against them later:
- The vault stays the source of truth. The index can be rebuilt from it at any time, and deleted without losing a word.
- Read-only. The retrieval service answers questions; it does not write memory. What goes into the vault goes through a person.
- Local models, and an honest line about where that stops. The embedding runs on our hardware: the vault is never shipped out wholesale to be vectorised. But the passages a query retrieves go into the prompt of whatever model the session runs on, and when that model is hosted, the retrieved text leaves the premises at answer time. Local embedding bounds the indexing; it does not bound the withdrawal.
- One reader, for now. The vault holds client records alongside our own, and today the only thing that queries the index is us. The moment it gains a second reader, human or service, retrieval needs an authorisation story of its own: a scope on who may withdraw what. That control is not built, and until it is, the reader count staying at one is the control.
- Plain files underneath. The day this experiment dies, the vault is still markdown in folders, worth what it always was.
What it changes if it works¶
A session that can ask the organisation what it knows stops being a clever tool you brief and starts being something closer to a colleague with access to the filing. The ten minutes of pasted context goes. The answers improve in the specific way that matters: grounded in what this company actually decided, not in what the internet thinks companies generally do.
And the three questions we ask of everything have tidy answers for once. What it makes possible: sessions that know the estate. What it costs: one VM, one embedding model, the sync, indexing and backups that keep an index honest, vigilance about the silent-absence failure and an authorisation story it does not yet have. Who controls it: we do, end to end, because it is files we own, indexed by models we run, on hardware in the building. Memory is becoming the asset in this kind of engineering, and this one is ours.