Understanding Memory: How AI Learns to Remember You

What is AI memory, and why is it not the same as a larger context window? This guide explains four types of memory and the four jobs every memory system performs; examines Generative Agents, MemGPT, Voyager, MemoryBank, and HippoRAG; compares ChatGPT, Claude, and Mem0; and tackles the two hardest questions: what to remember and what to forget.

Large language models have an easily overlooked property: they have no memory by default. Each call starts from a blank slate. The model remembers neither what you just said nor what you discussed last time unless that information is placed into the context window again. When ChatGPT seems to "remember me," that is not an intrinsic model capability; an external memory system is storing and retrieving information on its behalf.

This article explains what that system contains, how it works, and how real products implement it. It is a sequel of sorts to the context engineering article: if context engineering decides what belongs in the window for this step, memory decides what is worth preserving across time and when it should be recalled.

1. First clarify: memory ≠ larger context window

This is the most common misconception. Many people assume that expanding a context window from 8K to 1M tokens solves memory. It does not.

context window is the workbench, and the memory is the warehouse. There are three differences:

• Persistence: context window is volatile and will be cleared at the end of a conversation; the memory must exist across sessions, days, and months.
• Capacity: No matter how large the window is, there is an upper limit, and the more crowded it becomes, the lower the signal-to-noise ratio, the worse the effect (context rot, see "context engineering"); the warehouse is theoretically unlimited.
• Cost: Every token in the window must be recalculated and recharged at every step; the information in the warehouse lies quietly, and only a small part is taken into the window when needed.

Therefore, the core task of a memory system is not to keep all history in the window, but the opposite: move information outside the window, then retrieve the right small piece at the right time. In one sentence:

Memory trades abundant storage outside the window for scarce attention inside it.

This is why an "infinitely long context" cannot replace memory: even if a window could hold a year of conversations, a model would struggle to use all of it effectively, and the cost would be enormous. A warehouse and a workbench serve different purposes.

2. The map given by the human brain: several types of memory

The classification of AI memory almost copies the division of human memory by cognitive science. Remember this picture, and all subsequent architectures will fall into place.

short-term memory (Short-term / Working Memory): The "working memory" used for the current task is equivalent to the computer's RAM. In LLM, it is basically equal to current context window - the ongoing conversation and the return of the tool just obtained. Volatile, limited capacity, fast.

long-term memory (Long-term Memory): Information retained across sessions, equivalent to a hard disk. It is further divided into three types. This distinction is very important:

A useful judgment method: episodic memory answers "what happened", semantic memory answers "what is true", procedural memory answers "what to do". The most solid one for most products today is semantic memory (remember your preferences and facts), followed by episodic memory, and procedural memory is the most difficult and cutting-edge - it means that Agent can "learn a set of practices" from experience and solidify them.

3. Four things a memory system really does

Fancy nouns aside, any memory system does four things in a loop. These four verbs are the backbone of your evaluation of any solution:

1) Writing/Encoding - From this interaction, determine "what is worth remembering" and organize it into a storable memory. The difficulty is choose: remembering everything will pollute the future, missing memory will cause amnesia.

2) Storage - put memory into a carrier outside the window: database, vector library, knowledge graph or pure file. This step determines how to search later.

3) Retrieve - During the next interaction, determine "which items should be remembered this time" and enter them into context window. If you take too much, it will be noise; if you take too little, it will be insufficient.

4) Update/Forget - Memories will become outdated and conflict ("He lived in Shanghai last year and moved to Beijing this year"). The system must be able to correct old memories and eliminate useless memories instead of mindless accumulation.

If you read the article "context engineering", you will find that this is exactly the expansion of the two operations "Write" and "Select" in the time dimension - write = Write, retrieve = Select. Memory is not a new thing, but a branch that grows along the "time axis".

To judge whether a memory system is good or not, don't look at "how much it can store", but whether it does these four steps accurately - especially the Writing trade-off and the Forgetting update, these two steps are the most difficult and can widen the gap the most.

4. Several classic architectures that you must know

There is more than one classic design for memory. The following five are the skeletons for understanding all memory systems. Each one pegs a different key ability. After reading this, you will have a complete "capability map".

1. Generative Agents: memory stream + Reflection

In 2023, Stanford's famous "Generative Agents" (allowing 25 AI villains to live independently in a virtual town) explained the memory architecture clearly for the first time, and it is still a must-read for beginners. It consists of three parts:

• memory stream (Memory Stream): A continuously appended list of experiences recorded in natural language. Everything the villain sees, hears, says, and does is written down in chronological order.
• Retrieval: It is impossible to fit the entire stream into the window, so each time score selection is based on the current situation - the score is a weighted sum of three dimensions: relevance (semantic similarity to the current situation), recency (the longer I don’t think about it, the lower the score, exponentially decaying over time), importance (the score when writing, low for "having breakfast", high for "breaking up with my lover").
• Reflection: The most important step. The Agent regularly reviews recent memories, synthesizes higher-level conclusions (summarizes "I seem to like Klaus next door" from scattered interactions), and then writes them back into the stream as new memories. Reflection upgrades memory from "running account" to "insight".

Remember these three words - memory stream, search by relevance/recency/importance, and reflect. Almost all memory systems later performed addition and subtraction on this skeleton.

2. MemGPT: Treat LLM as an operating system (layered + self-editing)

If Generative Agents gave a paradigm of "retrieval", then MemGPT in 2023 (the subtitle of the paper is called Towards LLMs as Operating Systems, now incorporated into the open source framework Letta) gave a paradigm of "hierarchical management". Its insight is wonderful: context window is like the physical memory (RAM) of the operating system, and the storage outside the window is like a disk; the operating system relies on "paging" to swap in and out to create the illusion of "infinite memory" - LLM can also do it.

The most critical design is: The model has a tool to "edit its own memory" - when the core memory is almost full, it decides which items to move to the archive and which items to transfer back to the core. This changes "memory management" from hard-written rules to the ability of the model itself. This set of "layered + self-editing" ideas has directly affected product-level implementation: For example, Anthropic added a "memory tool" to Claude in September 2025 - giving the model a dedicated file directory, allowing it to add, delete, modify and check by itself, and accumulate knowledge across sessions; combined with "context editing (automatic cleaning of expired tool calls in the window)", official data shows that 84% of tokens were reduced in 100 rounds of web search evaluations, improving compared to the baseline 39%. The same skeleton, productized look.

3. Pin a key point on each of the other three: Voyager, MemoryBank, HippoRAG

• Voyager——The skill library is "procedural memory". ** This autonomous exploration agent in "Minecraft" in 2023 allows the model to generate executable code for each new task and store the run-through programs in natural language description** into the growing "skill library"; next time it encounters a similar task, it can directly retrieve and combine old skills without thinking about it from scratch. This is exactly what the second section says - what is stored is not "facts", but "the ability to do it".
• **MemoryBank - Equip memory with a "forgetting curve". ** It brings the Ebbinghaus forgetting curve of psychology into AI: each memory has a "strength", which is strengthened when it is recalled, and decays if it is not touched for a long time. During retrieval, semantic similarity and this "retention score" are combined. It provides an elegant machine answer to the seventh section "What to forget" - it is not a one-size-fits-all deletion, but a natural forgetfulness and awakening like a human being.
• HippoRAG - map the hippocampus. It borrows the "hippocampal index theory" from neuroscience: treat LLM as the "neocortex", build another knowledge graph as the "hippocampal index", and use an algorithm similar to PageRank to associate on the graph during retrieval. It is the most theoretically ambitious line of picture memory-it is good at connecting scattered clues to answer questions.

Putting the five together is a "capability map" of memory design: Generative Agents is responsible for retrieval and reflection, MemGPT is responsible for hierarchical scheduling, Voyager is responsible for skill accumulation, MemoryBank is responsible for forgetting, and HippoRAG is responsible for association. Whichever ability you lack, just look at the design of that area.

5. What does the memory in the product look like?

Put the architecture into the products you use every day:

ChatGPT: Two layers of memory. One layer is Saved Memories - a list of explicit facts that you can view and delete ("I am a vegetarian"), and the model automatically writes in it; the other layer is Reference Chat History that will be launched in April 2025 - it does not make a list, but implicitly captures patterns from all your past conversations to make the answers more relevant to you. Both layers can be turned off individually in settings, and temporary conversations do not read or write memory. In 2026, OpenAI added a background mechanism to automatically organize memory, making the memory more and more accurate over time.

Claude: We are following the "file-based memory tool + contextual editing" route mentioned in the previous section, which is more controllable by developers - the memory is stored in your own infrastructure, and you have full control of the data.

Gemini: Google has also added memory to Gemini - it can not only remember the personal information you saved in the settings, but also selectively refer to your past conversations to personalize answers. It also supports viewing, editing and closing.

Common Product Philosophy: Memory is not a black box. Allowing users to be able to see, change, and turn things off is becoming a standard feature - because once the memory is remembered incorrectly or something should not be remembered, the collapse of experience and trust will be doubled.

6. Engineering implementation: three mainstream approaches

If you really want to build your own memory system for storage and retrieval, the industry basically has three routes (often mixed):

Route 1: Full-context/Summary The simplest: put all or a summary of the history back into the window. Simple, but it will run into context rot as it grows, which is slow and expensive. Suitable for short-cycle, lightweight scenarios.

Route 2: Vector retrieval (Vector/RAG-style) Convert each memory into a vector and store it in the vector library, and then retrieve the most relevant ones based on semantic similarity. This is currently the most mainstream approach. The open source framework Mem0 is representative: it does not save the original conversation, but uses LLM to extract key facts from the conversation and save them, and only retrieves relevant memories during retrieval. On the long-range conversation memory benchmark LOCOMO commonly used in the industry, Mem0 officially reports that its accuracy is about 26% higher than OpenAI's memory solution, latency is about 91% lower, and token costs are saved by about 90%. The core win is "only feeding a small amount of relevant history, rather than the entire history."

Route 3: Knowledge Graph (Graph) Save memory as an "entity-relationship" diagram (Zhang San - works for - a certain company). Good at handling multi-hop reasoning and relationship changes over time ("He changed jobs" only needs to change one edge). Expressive, but expensive to build and maintain.

In practice, there is no silver bullet. Mature systems often focus on vectors, supplemented by maps, and abstracts.

7. The two most difficult things: what to remember and what to forget

Storage and retrieval are engineering projects, and the real difficulty is two true or false questions:

Problem 1: What to write? Not every sentence is worth remembering. Random memorization will bring about two kinds of disasters: one is noise - writing down all irrelevant trivial matters, which will interfere with future retrieval; the other is memory poisoning - an early wrong conclusion is written into long-term memory, and then retrieved every time, making mistakes along the way, and even becoming an entry point for attacks (someone specializes in "poisoning" memories). So writing must have trade-offs, deduplication, and conflict detection.

Problem 2: What should be forgotten? Memories become outdated and contradictory. The user said "I am in graduate school" last year and graduated this year; what I liked last month is boring this month. A memory bank that only increases but never decreases will sooner or later contradict itself. A good system requires update (overwriting old ones with new memories), attenuation (natural fading out if not used for a long time), and even active forgetting (users request deletion, which is also strictly required for compliance).

In a memory system, storage is the beginner level, retrieval is competence, and forgetting is mastery. Memory that can forget remains alive; otherwise, you do not get an assistant that understands you better over time, but one that carries stale assumptions and grows increasingly stubborn.

8. Developers want to add memory: Don’t reinvent the wheel, look at these solutions first

The good news is that by 2026, "adding memory to the Agent" will no longer need to be written from scratch - a number of mature open source frameworks have encapsulated the above routes. For developers who want to get started, the mainstream options are roughly the following (all can be self-hosted):

Don’t remember the parameters when choosing a model, remember this main line: What ability do you lack most, choose a framework that focuses on that ability - if it’s fast, choose Mem0, if you want to handle timing changes like "Zhang San quits," choose Zep/Graphiti, and if you want a stateful Agent that can manage its own memory, choose Letta.

The most popular new project in 2026 is exactly the opposite answer to the seventh section of "What to remember and what to forget" - MemPalace. It was developed by "Resident Evil" heroine Milla Jovovich and engineer Ben Sigman, using Claude Code, and is open sourced by MIT. It reached more than 20,000 stars two days after its release in April 2026. It started with a complaint: "I'm tired of AI deciding what to remember for me and then throwing out the context I really need."

So MemPalace went in the opposite direction and borrowed the ancient "memory palace" mnemonic technique: save the dialogue word for word without summaries, and then use spatial metaphors to organize it into wings (by people and projects) → rooms (by themes) → drawers (for original text). During retrieval, it can be "partitioned" to search accurately, instead of fishing blindly in a large group of vectors. It uses hybrid retrieval (keyword weighting + temporal proximity), and achieved a recall@5 of 96.6% on the LongMemEval benchmark for pure retrieval. The storage backend is pluggable (default is ChromaDB, SQLite/Qdrant/pgvector is also supported) and can be connected to Claude Code, Cursor, etc.

The value of MemPalace is not only that "a female star wrote a popular open source project", but that it puts the philosophical differences in Section 7 on the table: Should AI automatically cut the memory for you, or should you keep it all and control the structure yourself? ** The former is worry-free, the latter is controllable - there is no standard answer to this, but it reminds you: ** "Let AI decide what to remember" itself is a design decision that requires your own decision.

9. Ready-to-use memory-system checklist

• Do I distinguish between short-term memory (inside the window) and long-term memory (outside the window)? Didn't you think of "a bigger window" as the answer to memory?
• In long-term memory, there are three categories: Semantics/Scenarios/Programs. Which ones do I need? What do you use to save?
• Are there any trade-offs for writing? Or just keep all the conversations mindless? Is there any duplication and conflict detection?
• Should the search be sorted based on relevance/recency/importance scores, or should all be returned to the window?
• Is there an update and forget mechanism? Will old, wrong, expired memories be corrected or eliminated?
• Can users see, modify, delete, and close their own memories (experience + compliance)?
• Is the storage selection correct? Is vector enough? Do I need map to handle relationships and timing?
• Is there any protection against memory pollution? Will the wrong conclusion be biased along the way after being written into long-term memory?
• Have you considered layering like MemGPT, or added reflection like Generative Agents, so that memory can be upgraded from a running account to an insight?

Models will get smarter and smarter, and windows will get bigger and bigger. But as long as the two "attention is limited and information will be out of date" are true, whether can remember you and think of the right things at the right time will always be a system that requires careful design, rather than an ability that is given away by the model.

References

• Packer et al., MemGPT: Towards LLMs as Operating Systems (2023; now an open source framework Letta)
• Park et al., Generative Agents: Interactive Simulacra of Human Behavior (2023, memory stream / Search / Reflection)
• Wang et al., Voyager: An Open-Ended Embodied Agent with Large Language Models (2023, Skill Library / procedural memory)
• Zhong et al., MemoryBank: Enhancing LLMs with Long-Term Memory (2023, Ebbinghaus Forgetting Curve)
• Gutiérrez et al., HippoRAG: Neurobiologically Inspired Long-Term Memory for LLMs (2024, Hippocampal Index/Graphic Memory)
• Mem0, Building Production-Ready AI Agents with Scalable Long-Term Memory (2025, LOCOMO benchmark); open source warehouse mem0ai/mem0
• Open source memory framework: Zep/Graphiti, Letta, Cognee, LangMem
• MemPalace, GitHub repository (2026, Milla Jovovich + Ben Sigman, "memory palace" style); Forbes report
• Anthropic / Claude，Managing context on the Claude Developer Platform（2025，memory tool + context editing）
• OpenAI, Memory and new controls for ChatGPT and Reference chat history (2025)
• Memory in the Age of AI Agents: A Survey (2025, review and paper list)