Back to Insights
Technical

AI Agent Memory Explained: How Agents Remember (and Why They Forget)

17 min read
Workingthis taskEpisodicwhat happenedSemanticwhat’s trueProceduralhow to do it123staleWRITEREMEMBERFORGET
Language models forget everything between calls, so any agent that seems to remember is running a memory system around the model. Here's how that system works, how it differs from RAG, why agents remember the wrong things, what can go wrong with privacy and security, and eight steps to keep memory accurate.

✦Key Takeaways

  • Language models remember nothing between calls. What looks like memory is software saving notes and putting the relevant ones back into the model's context window.
  • Short-term memory is the context window and ends with the session. Long-term memory lives in files or databases and only helps if something writes it, finds it and keeps it accurate.
  • Memory and RAG share plumbing but differ in who writes the material. RAG reads documents people wrote. Memory is written by the agent itself, often per user, and can change every session.
  • Saving more doesn't mean remembering better. In Anthropic's Pokémon test, the model that kept 413 files earned one badge, while the one that kept 10 earned four.
  • A wrong note does lasting damage because later sessions trust it. Decide how notes get checked, corrected and expired before launch, not after the first bad answer.
  • Memory is an attack surface. Researchers have planted false memories in ChatGPT and Gemini through web pages and documents, so control who and what can write to it.
  • Switch memory off where every case must be judged on its own, such as credit or hiring decisions, and look up fast-changing facts live instead of remembering them.
When Anthropic's researchers opened the memory folders of two AI models playing Pokémon Red, the contrast was hard to miss. Claude Sonnet 3.7 had treated everything as worth keeping: it ultimately wrote 413 memory files across 225 folders and earned one gym badge. Claude Fable 5 kept nine files in two folders, plus one top-level file, and earned four.
That was one run of each model, published in November 2026, so it proves little on its own. But it captures what teams building agents keep finding out the hard way. Storing information is the easy part of AI agent memory. The hard parts are deciding what's worth keeping, finding it again at the right moment and noticing when it has gone out of date.
This guide explains how agents remember when the models underneath them can't, the difference between short-term and long-term memory, how memory differs from RAG, the five reasons agents forget or remember the wrong thing, the privacy and security risks, and how to design memory that stays useful.

What Is AI Agent Memory?

AI agent memory is the system that lets an agent keep information beyond a single model call and use it later, whether "later" means the next step of a task, next week's session or another conversation with the same customer.
The model itself remembers nothing between calls. Its weights hold what it learned in training, and each call starts from whatever text it's handed. Everything that looks like memory is software choosing which saved notes to put back in front of the model, inside its context window. That makes memory one part of the wider job we described in our guide to context engineering: deciding what the model sees at each step.
Anthropic's research post draws a useful boundary. It treats agent memory as knowledge an agent "gains from experience" and "retains between sessions, saved as context it can read later". That rules out three things people often lump in: changes to the model's weights, learning that happens within a single conversation, and compaction, the summaries that replace older turns when a conversation runs long.
A hospital shift handover is a useful comparison. Nurses going off shift don't recount their whole day. They pass on what the next shift needs: what changed, what to watch and what has already been tried. A good handover note is short, current and correct. Agent memory needs the same three qualities, and most memory failures come from losing one of them.

Short-Term vs Long-Term Memory in AI Agents

Agent memory comes in two broad kinds, and they fail in different ways.
  • Short-term memory is the context window: the conversation so far, the current plan and recent tool results. It's accurate while it lasts, but it's limited in size and disappears when the session ends. As it fills up, systems summarise or drop older material, which is where details often go missing.
  • Long-term memory lives outside the model, in files, databases or search indexes, and survives between sessions. It can hold far more, but nothing in it helps until something writes it down, finds it again and keeps it accurate.
Why not send the whole history every time? Cost and speed. In a 2025 paper, the team behind the open-source memory layer Mem0 reported 91% lower p95 latency (the response time that 95% of requests beat) and token savings of more than 90% compared with feeding the model the full conversation. Those are a vendor's figures on one benchmark, but the direction is consistent: past a certain point, rereading everything is slow, expensive and, as long-context research keeps showing, less accurate.

The four types of agent memory

A 2023 Princeton paper, Cognitive Architectures for Language Agents, borrowed four categories from cognitive science that most agent frameworks still use.
Memory typeWhat it holdsExample in a customer service agent
WorkingThe current task and its detailsThe order number the customer just gave
EpisodicPast events and experiencesLast month's refund for this customer was delayed by a courier problem
SemanticFacts about the world, the user or the businessThis customer prefers email and works on Berlin time
ProceduralHow to do things: instructions, skills and codeThe steps for issuing a refund in your billing system
The labels matter because each type needs different handling. Episodic memory grows quickly and goes stale. Semantic memory needs correcting when facts change. Procedural memory, the paper notes, lives in the model's weights and the agent's code, so in practice the people who build the agent set it up and change it.

How AI Agent Memory Works

Every memory system, from ChatGPT's to a custom support agent's, has to do four jobs.

Write: decide what's worth keeping

Either the agent decides during the task, by calling a tool that saves a note, or a background process reads finished conversations and extracts what matters. Anthropic's memory tool takes the first route. Claude checks a memory directory before it starts a task and saves what it learns there as files. The tool runs on the developer's side, so the business decides where those files live.

Store: pick a format you can inspect

Notes can be plain text files, database rows, entries in a vector database (which stores text as lists of numbers so that similar meanings can be found) or a knowledge graph of linked facts. Plain files are underrated. People can read, correct and audit them without special tools.

Retrieve: get the right note back at the right time

There are two routes. Some memory is injected into every session, which is reliable but costs space, so it has to stay small. Claude Code, for instance, loads only the first 200 lines or 25 KB of its MEMORY.md index file at the start of a session. Everything else is searched when needed, which scales better but depends on the agent thinking to look. Stanford's 2023 Generative Agents study ranked memories on three signals, how recent, how important and how relevant each one was, and many systems still use a version of that idea.

Maintain: correct, merge and expire

This is the job most systems skip. Anthropic's researchers describe a process often called dreaming, in which the agent reviews its logs offline to "fix mistakes or produce new notes and representations". OpenAI uses the same word. In June 2026 it began moving ChatGPT to a dreaming-based memory system that updates memories automatically in the background, with a summary page where users can see and edit what it has inferred.
Anthropic describes the result as a set of layers. Raw transcripts sit at the bottom as a searchable log. Above them are notes the agent chose to write, then organised pages such as a wiki for each subject. At the top is a small amount of context loaded into every session.
A four-layer pyramid. From the bottom: Transcripts, a searchable log of every session; Notes, what the agent chose to write down; Wiki pages, organised by subject and linked; and at the narrow top, Injected context, loaded every session, with a note saying keep this layer small because it takes up space in every session. A dashed arrow labelled Dreaming loops from the bottom layer to the top, described as an offline review that checks notes against the transcripts, fixes errors, merges duplicates and promotes what matters.
The top layer is read every time, so it must stay small. Lower layers are searched only when needed.

Agent Memory vs RAG: What's the Difference?

Memory and retrieval-augmented generation (RAG) often share plumbing. Both store text, search it and add the best matches to the model's context, and they may even use the same database. The difference is who writes the material and how often it changes.
RAGAgent memory
Who writes itPeople: your policies, manuals and documentsThe agent, from its own work and conversations
How often it changesWhen someone updates a documentPotentially every session
ScopeUsually shared by every userOften per user, per customer or per project
Typical failureA document is out of date or missingA note is wrong, stale or planted
The last row matters most. A wrong document is a content problem you fix once, as we explained in why you should fix your knowledge base before building a chatbot. A wrong memory was written by the agent itself, often without anyone reading it, and it can be reused session after session. For the trade-offs between retrieval and simply loading more text, see our comparison of advanced RAG and long context windows.
The two can work well together. In one Anthropic experiment, Claude built a wiki from a corpus of about 350,000 files. With search alone, it met 53% of the graded criteria on a set of 25 questions. With search and the wiki together, its score rose from 54% to 61% as the wiki grew, and it used about 20,000 fewer tokens per question. On legal questions about mergers, it scored 87% with a wiki against 73% with search alone. On another legal task, though, the wiki showed no clear gain, a reminder that a memory layer is worth building only when testing shows it helps.

Why AI Agents Forget (and Remember the Wrong Things)

When an agent with memory gets something wrong, the cause is usually one of five problems.

1. The note was never written

An agent has to decide what matters before it knows what's coming. Anthropic treats this as an open question: "how can Claude tell what's salient in the moment, before it knows what's coming next?" Its framing is that every saved note is a small prediction that a future session will need it. Some predictions will be wrong.

2. The note exists but isn't read

Anthropic's CL-Bench tests include a task where Claude answers questions by looking things up in a database, is told the correct answer afterwards and can write a note for next time. A memory store lifted Claude Opus 4.7's score from 0.383 to 0.700. Claude Sonnet 4.6 barely moved, from 0.330 to 0.364. The newer Sonnet 5 read its memory before answering 9 times out of 10, while Sonnet 4.6 did so less than half the time. Once researchers refined its instructions, Sonnet 4.6 reached 0.641. Memory only helps if the agent checks it.

3. Too much was saved

This was Sonnet 3.7's problem in Pokémon. It stored details it could read straight off the screen, such as its money and map position, so its records kept going out of date and it spent more and more effort combing through memories instead of playing. Around step 10,000, it spent 17 steps creating six files and deleting nine others without taking a single action in the game. Fable 5 stored strategy, plans and tested rules instead.
Side-by-side comparison of two memory folders from Claude playing Pokémon Red. On the left, Claude Sonnet 3.7's folder is a large grid of 413 small coloured tiles: 413 files, 225 folders and 977,930 characters, saved everything including state shown on screen, and earned 1 of 8 badges. On the right, Claude Fable 5's folder is a tiny block of 10 tiles inside a dashed outline showing Sonnet 3.7's folder at the same scale: 10 files and 43,406 characters, saved strategy, plans and tested rules, and earned 4 of 8 badges.
One run per model, published by Anthropic in November 2026. Box area shows the total characters each model stored.

4. The note is wrong

A wrong note does real damage because later sessions trust it. In another Pokémon run, Claude recorded the location of a Pokémon Center incorrectly. With the note left uncorrected, it fell back onto an earlier route three times and never got past the mountain it was trying to cross, the same outcome as a run with no memory at all. Anthropic's warning applies to any business agent: a note "that's wrong from the start or that drifts as the environment changes can be read, and trusted, by Claude in a later session".

5. Agents overwrite each other

Shared memory needs the same care as a shared spreadsheet. In one Anthropic test, several agents wrote contract terms into a single shared table at the same time. The table summarised the terms poorly and lost some altogether because agents overwrote one another's work, and a brief built from it included only 41 of 57 facts. When each contract got its own page, written by one agent that quoted the terms word for word, the brief met every criterion.

The Privacy and Security Risks of Agent Memory

Memory turns a passing remark into a stored record. A customer mentions a health condition while asking about a delivery, and a careless memory system files it away for every future conversation. Anthropic's documentation says Claude "usually refuses to write sensitive information to memory files" and recommends adding your own check that strips sensitive data before anything is saved. "Usually" is the word to plan around.
Memory also gives attackers a way to stick around. In 2024, security researcher Johann Rehberger showed that a malicious web page could plant instructions in ChatGPT's long-term memory that sent later conversations to an attacker's server. OpenAI closed the data-leak route in September 2024, although Rehberger noted that untrusted content could still write memories. In 2025 he showed that a hidden instruction in a document could get Google's Gemini to save false memories once the user typed an ordinary "yes".
Attackers don't even need access to the memory store. The MINJA attack, presented at NeurIPS 2025, planted malicious records in an agent's memory using nothing but normal queries, and its authors warn that it "enables any user to influence agent memory". The risk is highest where one memory is shared across many users.
Memory can shape behaviour in quieter ways too. When OpenAI rolled back an overly flattering ChatGPT update in 2025, its follow-up explanation said user memory had made the effect worse in some cases, though it had no evidence that memory increased it across the board.
The basic protections are the ones you'd apply to any database holding personal data:
  • Separate memory by user, customer and project. Claude, for example, keeps a separate memory for each project.
  • Confirm before saving, especially after the agent has read emails, web pages or uploaded files.
  • Validate file paths. Anthropic's docs warn that a path such as /memories/../../secrets.env can reach files outside the memory folder.
  • Let people see, edit and delete what's stored about them, and expire notes nobody uses.
  • Log every write so you can trace where a bad memory came from.
If agents share memory or pass notes to one another, the exposure grows. We covered those leak paths in securing the agent-to-agent conversation.

When Agent Memory Helps and When It Hurts

Memory is a design choice, not a default. It earns its place when work carries over between sessions, and it gets in the way when every case should be judged on its own.
SituationUse memory?Why
Long-running projects, such as coding or research spread over weeksYesSaves re-explaining decisions and dead ends
Repeat customers with stable preferencesYes, per customerFewer repeated questions, more relevant answers
Agents that run the same workflow many timesYes, for tested rulesLessons from past failures carry forward
Decisions that must be consistent and auditable, such as credit or hiring screensUsually noEarlier conversations shouldn't change the outcome for the next applicant
Fast-changing facts, such as prices or stock levelsNoLook them up live, because a remembered price goes stale
Confidential one-off questionsNoUse ChatGPT's temporary chats or Claude's incognito chats, which stay out of memory

How to Design Agent Memory That Stays Useful

If you're building an agent, or buying one with memory switched on, these eight steps cover most of what goes wrong.
  1. Say what memory is for. Anthropic's advice is to tell the model what memory is for, "but not how to use it". When researchers imposed a rule to "record only failures", Opus 5 and Opus 4.7 lost much of the boost memory had given them. Older models benefited more from explicit steering.
  2. Keep the always-loaded layer small. Use a short index that points to detail stored elsewhere. Every injected note takes up space in every session, and a wrong one travels everywhere.
  3. Keep the raw transcripts. Notes are summaries. Keep the originals so notes can be checked against them and rebuilt when they drift.
  4. Store lessons, not live state. Save tested rules and decisions. Look up balances, prices and statuses at the moment you need them.
  5. Give each customer, contract or case its own record, rather than one shared file that several agents edit at once.
  6. Schedule maintenance. Review notes against transcripts, merge duplicates and delete stale entries. Budget compute for it, and check frequently used notes more often.
  7. Control who and what can write. Separate memory by user and account, confirm saves triggered by outside content and strip sensitive data before storage.
  8. Test with memory on and off. Run the same questions across several sessions, with and without memory, as our guide to testing an AI system before launch explains. If memory doesn't measurably help, switch it off.

Conclusion

The model inside an agent forgets everything, every time. What it appears to remember is a set of notes, and those notes are only as good as the decisions about what to write, what to load and what to throw away.
That's why good memory is selective. In Anthropic's tests, the agent with fewer, better-chosen notes made more progress than the one that saved everything, and a run carrying one uncorrected wrong note got no further than a run with no memory at all. Forgetting on purpose is part of the design.
If you're planning an agent that needs to remember customers, projects or past work, or you want to check what an existing one has stored, talk to AI Native Agency.

Frequently Asked Questions

Do AI agents actually have memory?
Not inside the model. Language models don't retain anything from one call to the next. Agents appear to remember because software saves notes, facts or past conversations and adds the relevant ones to the model's input in later sessions.
What is the difference between short-term and long-term memory in AI agents?
Short-term memory is the context window, the text the model can read during the current session. Long-term memory is stored outside the model, in files or databases, and carries over between sessions. The first is accurate but temporary; the second lasts but has to be retrieved and kept current.
Is agent memory the same as RAG?
No, although they often use the same technology. RAG retrieves passages from documents people wrote, such as policies and manuals, and is usually shared by all users. Agent memory is written by the agent from its own experience, is often kept per user and changes constantly.
Does ChatGPT remember everything I tell it?
No. ChatGPT keeps a curated memory drawn from your past chats, and in June 2026 OpenAI began rolling out a system that updates it automatically in the background. You can review and edit what it has stored, delete memories, turn memory off or use a temporary chat, which doesn't use or update memory.
What is memory poisoning in AI?
Memory poisoning plants false or malicious information in an agent's memory so that it influences later sessions. It can arrive through a web page, document or email the agent reads, or through ordinary conversations, as the MINJA research showed. Defences include confirming memory writes, separating memory by user and reviewing stored notes regularly.
How do I make an AI agent forget something?
In consumer tools, open the memory settings and delete the specific memory or clear them all. In a custom agent, you need a delete function that removes the note from every store, including search indexes and any summaries built from it. Build this in from the start, because privacy laws in many countries give people the right to have their personal data erased.
Which tools add memory to AI agents?
Options include Anthropic's memory tool for Claude, open-source memory layers such as Mem0 and Letta, which grew out of the MemGPT research project, and the memory stores built into agent frameworks. Plain files or a database you control can work just as well, as long as you can inspect, correct and delete what's stored.