Agent Memory
Agents retain what they learn, that memory diverges per agent, and insights transfer across projects.
Memory in PolyphonyAI stands on three pillars: how each agent retains what it learns, how those memories diverge, and how insight transfers between projects.
Retention.
Each agent accretes its own memory: an append-only log the search index rebuilds from.
Divergence.
Visibility rules make each agent’s memory diverge.
Transfer.
Distilled insight carries deliberately across projects.
Retention.
Memory here isn't just persistent, it's per-agent, shaped by what each agent is allowed to see, and immutable once written. Each agent builds its own memory on an append-only log, and the search index is derived from that log and rebuilds from it, so it can't silently drift from the record.
Why it matters
An agent with memory becomes a collaborator that compounds: it builds on past sessions, stays aware of your corrections, and keeps that memory its own. A code-review agent that has seen fifty of your pull requests knows your conventions, remembers the architectural decision you made two months ago, and can flag when new code quietly contradicts it.
How it works
- An append-only log. Every recorded interaction becomes an event, and events are never edited or deleted: this is enforced by SQLite triggers that reject every UPDATE and DELETE on the memory log, so immutability lives in the schema, not in a prompt or app code.
- Four kinds of event. Observations (a distilled record of what the agent learned), reflections (a periodic synthesis of recent events), corrections (which supersede an old memory without erasing it), and seeds (context you add directly, plus a role-based baseline).
- An index that can't drift from the record. The database is the single source of truth, and the search index is derived from the append-only log and rebuilds from it, so it can't silently drift from the record.
- Focused recall. An agent searches its reflections by meaning, re-ranks the strongest matches, and pulls in only the handful that bear on the topic.
- Stays fast as it grows. Older events are compacted into summaries and archived, so recall stays quick.
- Traceable to its source. Every memory records where it came from. An observation points back to the message that produced it, and a reflection points back to the observations it distilled, so nothing an agent knows is untethered from how it learned it.
- You can always ask why. Follow any memory forward as well as back: which reflection consumed an observation, every time a memory was retrieved into a turn, and whether it was ever carried to another project. When an agent acts on something, you can see the chain that led there.
What it means
You can trust that what an agent remembers actually happened: history can't be rewritten, corrections override without erasing, and memories are appended, not overwritten, so the record stays intact.
Divergence.
Not every agent sees the same messages, so their memories diverge, and each brings genuinely different context to a problem.
Why it matters
In our experience, agents with genuinely different context evaluate a problem differently than the same model sampled repeatedly. A security-focused agent and a UX-focused agent, each given visibility into their own domain, come to evaluate a new feature from genuinely different vantage points, because they actually hold different context, not just a different instruction.
How it works
- Memory shaped by visibility. As each memory is written, it is checked against that agent's visibility, so an agent's memory only ever holds what it was cleared to see. Because agents see different things, their memories diverge.
- Separate memories. Every memory is tied to one agent, and every read is scoped to that agent. No agent reads another's memory on its own. The only way knowledge crosses is a reflection you explicitly mark for transfer.
- Its own reflections. Each agent reflects only over its own stream and starts from its own role-based baseline.
Where an agent is set to blind, the author's name is withheld and the content is shown, so what it remembers is the substance, not who said it.
What it means
A room of agents with real, distinct perspectives, and you control, per message and per agent, who knows what. It shapes what each agent knows, not how it talks.
Transfer.
Knowledge built in one project shouldn’t stay locked there. Distilled insight carries deliberately across projects.
Why it matters
Without it, every project starts from scratch, and an agent that remembers what it learned across your past sessions has to relearn it next door. Transfer moves the distilled knowledge, not the raw noise, to exactly where it helps.
How it works
- You choose what moves. Only reflections, the distilled summaries, can be marked for transfer, and the mark is guarded: it must belong to that agent, and each insight transfers once.
- The receiving project opts in. Nothing crosses by default.
- It arrives as a read-only reference. A transfer-enabled agent searches shared insights by meaning, includes only clearly relevant ones, never its own, and each is labeled so you can verify its source. The insight is never copied into the receiving agent's memory, so a receiver can read it but cannot edit, delete, or build on top of it.
What it means
Knowledge compounds across your whole body of work, but only where you send it, only where it's relevant, and a transferred insight can't be edited or overwrite anything in the receiving project. It arrives as an advisory reference, labeled and surfaced for you to verify.
Three pillars, one system.
Each agent retains its own memory, those memories diverge by visibility, and insight transfers between projects.