AI Agent Memory: Do You Need a Vector Database? (Files vs Frameworks)
For a single AI agent at solo scale you do not need a vector database. Plain markdown files the agent reads on startup and updates before it stops carry the operational memory reliably, and they beat a vector store on auditability, versioning, and portability. You reach for frameworks only when you have hundreds to thousands of documents to search.
Most writing on agent memory starts from the wrong end. It assumes you are building a memory layer with embeddings, chunking, and semantic retrieval, then asks which vector database to bolt on. That is the right question at scale and the wrong one for a solo operator running a single agent on a single project. At that size the memory that matters is small, structured, and changes every session, and the format that serves it best is a handful of text files you can open and read yourself.
We can be concrete because we run this way. Six plain text files run our entire revenue operation, with $0 spent on memory infrastructure - no vector database, no embedding API, no managed memory service. The agent reads those six files at the start of every session and updates them before it stops. That is the whole memory system, and it has never once needed semantic search to do its job.
What does file-based memory actually look like?
File-based memory is a small set of named markdown files, each owning one kind of durable knowledge, that the agent reads on startup and rewrites before it stops. There is no index and no retrieval step: the files are few enough that the agent simply reads them in full. Here is the exact six-file map that runs our operation.
| File | Role |
|---|---|
| STATE | Where things stand right now, the next concrete steps, and what is blocked. Read first, updated last. |
| LOG | Dated session journal - what got done, what failed, what was learned. Append-only history. |
| BACKLOG | Prioritized task list plus an ideas parking lot, so the agent always knows what to pick up next. |
| DECISIONS | Choices made and the reasoning, so a future session does not relitigate a settled call. |
| METRICS | Revenue, traffic, and conversion numbers pulled on a schedule - the ground truth for what is working. |
| FOR_HUMAN | The inbox: questions and requests for the owner, and the owner's answers back. The one asynchronous channel. |
The criterion for what goes in these files is simple and worth stating plainly, because it is the part people get wrong. Information belongs in a context file when it is non-discoverable - the agent cannot re-derive it from the code, the tools, or first principles - and it is durable and universal, meaning it matters across sessions rather than for one passing moment. A decision to abandon a niche is both non-discoverable and durable, so it goes in DECISIONS. The output of a command the agent can simply re-run is discoverable, so it does not. Apply that two-part test and the files stay small and every line earns its place.
This is a different question from which files to create, which we cover in building your AI agent a second brain. Here the point is narrower: for that job, at this scale, six flat files outperform any database. They are readable by a human in under a minute, they diff cleanly in git so every change has an author and a timestamp, and they move to any tool by copy and paste. A vector store gives you none of those things, and below scale it is not buying you anything in return.
When do you outgrow plain files?
You outgrow plain files only at large scale, when you have many hundreds to thousands of documents that need semantic retrieval - a knowledge base, a support corpus, years of tickets - and no agent could read them all into context each session. At that point you genuinely need embeddings and a vector database, because the retrieval problem is real: the agent has to find the three relevant chunks among ten thousand, and grep over keywords will not reliably surface them. Below that line the calculus is reversed. A solo operation touches a working set small enough that a state file plus grep covers every lookup you actually make, and that is not a compromise - it is the better tool for the size. The moment you introduce a vector store you take on cost, an embedding pipeline, and opacity: you can no longer open the memory and read what the agent will see, because it lives as numbers you cannot inspect. That is why the reason agents forget is solved by persistence, not by retrieval sophistication - the failure is having no durable store at all, and one plain file fixes it.
So the honest rule is a threshold, not a preference. If your agent's durable memory fits in files a person could skim, keep it in files and spend the $0. When the corpus you must search reliably crosses into the thousands and semantic recall becomes the bottleneck, add the vector database then - and because your memory was portable text all along, adding it is a migration, not a rescue.
FAQ
Is markdown reliable enough for agent memory?
Yes. Plain text is the most durable, tool-independent, and auditable format there is, and it is exactly what the agent reads directly, so nothing is hidden between you and the memory. It opens in any editor, diffs cleanly in version control so every change has a timestamp and an author, and will still be readable in decades. For a single agent's operational state, nothing beats it on reliability.
What about semantic search over my agent's memory?
You rarely need it below roughly a thousand documents. A state file plus grep covers a solo operation, because the working set is small enough that plain reads and keyword searches find everything you actually look for. A vector store adds cost and opacity you cannot easily inspect - the memory becomes numbers you cannot open and read - so at small scale it takes more than it gives.
Can I migrate to a framework later?
Yes, and plain files make it easy. Because the memory is portable text you own, you can move it into any store or framework later without being locked in now. Files impose no schema and no vendor, so when a growing corpus genuinely needs semantic retrieval, you feed that same text into an embedding pipeline as a straightforward migration. Starting simple costs you nothing in future options.