Every team that builds an AI agent eventually hits the same wall. The agent is sharp inside a single conversation, then the conversation ends and it forgets everything. So someone says "let's give it memory," and reaches for the obvious tool: a vector store of past chats, an embeddings database, a scratchpad the model scribbles notes to between sessions.
That works right up until you put it somewhere that matters. In a regulated channel (healthcare, pharma, anything where you have to prove what you knew and when), the notebook is exactly what fails you. It drifts. It contradicts itself. And when someone asks "why did the system act on that," you are holding a pile of model-written notes with no way to reconstruct how it got there.
I ran into this while designing the next phase of an agent I already have running.
The agent lives on a live customer channel (Claude API into an Odoo ERP, over WhatsApp). It qualifies leads, quotes from the live catalog, and writes orders, all behind guardrails: a price-integrity rail so it cannot invent a number, confirmation gates, auto-pause the moment a human steps in, and one append-only log where every action lands as an event. That log is what makes the whole thing auditable. You can replay exactly what it did and when.
So when I started on memory (the system should remember that a patient did their intake at checkout, or called last week about the same problem), I almost reached for the notebook too. Then it clicked.
The reframe
Memory done right is not a new thing to build. It is the log I already have,
Do not picture memory as a place the model keeps notes. Picture it as a projection of an append-only log. A patient's current state (what they said at intake, what they bought, their verified address) is not a blob you store and mutate. It is derived by replaying the events that happened to them, in order. Which means at any moment you can answer "how and when did the system come to know this" by pointing at the exact events.
Say that out loud and the hard problem dissolves. Auditability stops being something you bolt on and becomes something that falls out of the design. Memory and the audit trail turn out to be the same log. You are just reading it two ways.
The one distinction that saves you
Hold one line here, or the whole thing rots. Two different things get called memory:
Facts of record (a date of birth, an address, what someone ordered, the answers they gave at intake). Structured, sourced, authoritative.
Working context (the gist, the summary, "seems frustrated," "probably wants the bundle"). Model-authored, useful, never authoritative.
The failure mode is when a model's summary quietly graduates into a fact an agent acts on. The rule that prevents it: the model can read facts freely, but it can only ever propose a new one through a constrained tool call (the same rail that stops it inventing a price). Its own summaries stay in a clearly labeled derived lane, always regenerable from the log, never the source of truth.
Sharing memory across agents
This also fixes a problem people usually solve badly. If you run a different agent for each phase (intake, checkout, follow-up), you do not want each one holding its own private memory, because private memory is neither shareable nor auditable. So none of them holds it. Memory lives in a context service the middleware owns. One agent writes an event, another reads the projection, and they never touch each other's memory directly. It is the same discipline as keeping the model and the ERP from talking directly. One point of trust.
The part no diagram hands you
Now the honest bit. An auditable memory is not the same as a good one. The log guarantees you can prove what the system knew. It does not guarantee that what it remembered was worth remembering. A perfectly auditable memory that feeds the checkout agent every intake event from eight months ago is not a feature. It is noise with a paper trail.
So the last layer is judgment: what is worth surfacing, how fast relevance decays, how much you feed a given agent for a given task. The cleanest way I have found to think about it borrows from how a computer manages memory (cache, RAM, SSD, cold storage), mapped to recency. This visit stays hot, in full context. This week is warm and surfaces automatically. This month becomes a summary on request. This year is a compact profile you only replay in full for a deliberate reason (a dispute, an audit). Going colder, the system surfaces less and summarizes more. Relevance decays. The record never does.
The thing to notice about that hierarchy: it is a read policy over one immutable log, not four separate stores. You never move the truth. You surface it differently depending on how relevant it still is.
Where this stands
To be straight about it: the agent is in production, and this memory architecture is the design for its next phase. I am writing it down because the design is the hard part, and because the principle travels far beyond my own use case.
If you take one line from this, take the one I keep coming back to: recall is a tool call, remembering is an event, and memory is a projection of the log, not a notebook. Get that right and you can give an AI a memory that a regulator could read without flinching.
If you are wrestling with the same thing (AI that needs to remember, somewhere you have to prove what it knew), I am always up to compare notes.