AI Agent Memory Systems: How Long-Term Memory Architectures Keep LLM Agents Coherent

Last updated October 2026.

An agent that forgets your database schema between sessions, re-asks for your deployment region every Monday, or confidently “remembers” a preference you never stated is not suffering from a weak model. It is suffering from a weak memory layer. The model itself is stateless: every call starts from whatever text you place in the context window, and nothing else. AI agent memory is the engineering that decides what gets written, where it lives, what is retrieved, and what is deliberately forgotten.

This matters more in 2026 than it did a year ago because agents now run for hours, span sessions, and act on production systems. The field has also split into distinct camps: OS-style paging (MemGPT and Letta), extraction-and-retrieval services (Mem0, LangMem), temporal knowledge graphs (Zep and Graphiti), and plain files the model edits itself (Anthropic’s memory tool). Vendors publish benchmark claims that do not agree with each other, so choosing well takes some care.

You will leave with a working taxonomy, a reference architecture, a mechanism-level comparison of the major systems, a retrieval and forgetting policy you can implement, an evaluation plan, and a list of failure modes including memory poisoning.

What this covers: the four memory tiers, the read and write paths, MemGPT/Letta, Mem0, Zep/Graphiti and the LangGraph store compared, consolidation and forgetting, benchmarks and their caveats, privacy and security, and a decision checklist.

Context and Background

A language model has two kinds of knowledge: parametric (weights, frozen at training) and contextual (whatever is in the prompt right now). Agent memory is an external system that moves information between sessions and the prompt. Long context windows do not remove the need for it. A million-token window is still a per-call cost, still degrades in recall quality in the middle of long inputs, and still vanishes when the session ends. The LongMemEval benchmark, published at ICLR 2025, reported a roughly 30 percent accuracy drop for commercial chat assistants and long-context LLMs when they had to memorize information across sustained interactions (LongMemEval, arXiv 2410.10813). Stuffing history into the window is not the same as remembering it.

The vocabulary comes from cognitive science, and the LangGraph documentation uses it directly: semantic memory for facts, episodic memory for past experiences, and procedural memory for rules about how to perform tasks (LangChain memory concepts). A fourth tier, working memory, is simply the context window itself. Earlier agent work such as the Generative Agents paper (Park et al., 2023, arXiv 2304.03442) established the pattern of a natural-language memory stream with reflection and selective retrieval, which most current systems still echo.

Three engineering lineages produced today’s tools. The first treats the context window like RAM and pages data in and out, which is the MemGPT line from UC Berkeley researchers (Packer et al., October 2023, arXiv 2310.08560), now productized as Letta. The second extracts discrete facts from conversations and retrieves them by similarity, which is the Mem0 approach. The third models memory as a graph with time on the edges, which is Zep’s Graphiti engine. If you have read our guide to agentic RAG architecture, memory is the same retrieval problem with one hard addition: the corpus is written by the agent itself, while it is running.

That last point is the thesis of this post. Retrieval over a static corpus is a read problem. Agent memory is a read-write-forget problem, and most production failures come from the write and forget halves, which demos never exercise.

A Reference Architecture for AI Agent Memory

Direct answer: A production agent memory system has five parts: a working-memory assembler that builds each prompt, three durable stores (episodic, semantic, procedural), a retrieval policy that selects within a token budget, a write path that extracts and scores new memories, and a background consolidation job that merges, invalidates and expires them. Each part can be simple or sophisticated, but none can be skipped.

Figure 1: Reference architecture. The retrieval policy mediates reads into the context window; the write path and background consolidation mediate writes into the three durable stores.

Figure 1 shows the data flow. A user turn enters working memory, which is assembled from the system prompt, recent messages and retrieved memories. The model responds, possibly calling tools. Separately, a write path inspects the exchange and decides what is worth keeping. A consolidation job runs out of band, turning raw episodes into stable facts and expiring stale entries. The key design decision is the boundary between the hot path (adds latency to the user’s turn) and the background path (does not), a distinction the LangGraph docs call hot-path versus background writing.

The four tiers and what each is for

Working memory is the live prompt. Its budget is hard: whatever you retrieve competes with instructions, tool schemas and the conversation. Treat it as a scarce resource with an explicit allocation, for example fixed slices for system instructions, a pinned profile, retrieved memories and the rolling transcript.

Episodic memory stores what happened: timestamped interaction records, tool call traces, outcomes. It is high volume, low density and answers questions such as “what did we try last Tuesday and why did it fail?” Because episodes are raw, they are the source of truth that the other tiers are derived from. Keep them, but do not retrieve them by default.

Semantic memory stores distilled facts and preferences: “the user deploys to eu-west-1”, “the billing service owns the invoices table”. LangGraph’s documentation describes two shapes for this tier: a continuously updated profile (one document per user) or a growing collection of individual facts. The profile is cheap to inject but suffers from lossy rewriting as it grows; the collection scales but needs retrieval and deduplication.

Procedural memory stores how to act: system prompts, tool-use rules, learned skills, even model weights in the broadest reading. In practice the tractable form is prompt and instruction refinement driven by reflection on past failures. This tier is the one most teams neglect, and arguably the one with the highest long-run payoff, because a corrected instruction improves every future run rather than one retrieval.

Why separate tiers rather than one vector store

A single vector index of everything is the most common first implementation, and it works for demos. It fails because the tiers have different write rates, different retention rules and different retrieval keys. Facts need contradiction handling. Episodes need time-range queries. Procedures need versioning and review. Collapsing them forces one retrieval method to serve three jobs. Separation also gives you three independent places to apply access control and deletion, which matters for the privacy section later.

The write path is where quality is decided

Retrieval gets the attention, but a memory that was never stored well cannot be retrieved well. The write path has four steps: extract candidate memories from the exchange, normalize them (resolve pronouns, absolute dates, entity names), compare against existing memories, and then add, update, or invalidate. Each step has a cost: extraction is typically an LLM call, which is why many systems run it asynchronously. Mem0’s paper describes exactly this two-phase extraction and update design, and its recent releases changed it, as discussed below.

How the Major Systems Actually Work

The named systems are often compared as if they were interchangeable products. They are better read as four different answers to one question: who decides what is remembered, the model or the infrastructure?

MemGPT and Letta: the model manages its own context

The MemGPT paper proposed “virtual context management”, borrowing the operating-system idea of a memory hierarchy so an LLM can work as if it had far more context than its window allows. The model is given tools to move data between tiers, and interrupts coordinate the model with the user. The authors evaluated it on document analysis beyond the native window and on multi-session chat.

Figure 2: The Letta-style hierarchy. Core blocks live in the prompt and are edited by the agent; message history and archival storage live outside and come back only through search tool calls.

Letta, the company and open-source project that grew out of MemGPT, describes memory as “pieces of context (strings) that are editable by agents via memory tools”, organized into blocks that can be shared across agents. A block has a label (for example “human” or “persona”), a value, a size limit that controls how much of the window it can consume, and an optional description that tells the model how to use it. Important core memories are injected into the prompt, and Letta’s documentation states that all state, including messages and tool calls, is persisted in a database, so evicted content remains retrievable through the API and through agent-accessible search tools (Letta memory docs).

Letta also describes sleep-time agents: background agents that work during idle periods to reflect on conversation history or a codebase and write new memories into shared blocks. This is the consolidation job from Figure 1, made explicit as an agent.

The trade-off is that the primary agent spends tokens and reasoning on memory management in the hot path, and the quality of memory depends on how reliably the model calls its memory tools. A practical note for October 2026: the Letta project’s repository now states that active development has moved to its letta-code agent harness, with the older V1 API server preserved on an archive branch, and the current letta package on PyPI (0.34.4, released 4 October 2026) is a terminal CLI for stateful coding agents. If you are evaluating Letta for a server-side product, check the current docs rather than older tutorials.

Mem0: extraction and retrieval as a service

Mem0’s paper (Chhikara et al., arXiv 2504.19413, April 2025) presents a memory-centric architecture that extracts, consolidates and retrieves salient information from conversations, plus a graph-based variant. On the LoCoMo benchmark the authors report a 26 percent relative improvement in LLM-as-a-Judge score over OpenAI’s memory baseline, a 91 percent lower p95 latency and more than 90 percent token cost savings compared with full-context approaches. The graph variant scored about 2 percent higher overall than the base configuration. These are authors’ numbers on one benchmark; treat them as a hypothesis to reproduce, not a guarantee.

The product has moved on from the paper. The package page for mem0ai lists version 2.2.1 (25 September 2026) and describes an “April 2026 algorithm update” with single-pass ADD-only extraction (removing the separate UPDATE and DELETE steps), agent-generated facts treated as first-class, entity linking, and multi-signal retrieval that combines semantic similarity, BM25 keyword matching and entity matching, plus temporal reasoning. The package page also reports LoCoMo 92.5 and LongMemEval 94.4 for the new algorithm, along with BEAM scores at one million and ten million tokens (mem0ai on PyPI). These are vendor self-reported figures that I could not independently reproduce, and they differ in kind from the paper’s numbers (a different metric and algorithm), so do not stack them into one trend line.

The architectural lesson is more durable than the scores. Moving to ADD-only extraction means memory becomes an append-mostly log with retrieval-time ranking, rather than a mutable database that the extractor must keep consistent. That trades storage growth for simplicity and removes a class of bad overwrite bugs. It pushes the hard problem, deciding which of two contradictory facts is current, to read time.

Zep and Graphiti: facts with validity windows

Zep’s paper (Rasmussen et al., arXiv 2501.13956, January 2025) introduces a memory layer built on Graphiti, a temporally aware knowledge graph engine that synthesizes conversational and structured business data while keeping historical relationships. The abstract reports 94.8 percent versus 93.4 percent for MemGPT on the Deep Memory Retrieval benchmark, and on LongMemEval accuracy improvements of up to 18.5 percent with response latency reduced by 90 percent against baseline implementations.

What distinguishes Graphiti is bi-temporal modelling. Each fact carries a validity window, so when information changes the old fact is invalidated, not deleted, and you can query what was true at a given time. Derived facts trace back to source episodes, which gives provenance. The open-source graphiti-core package (0.30.2 at the time of writing, released 8 September 2026) supports Neo4j 5.26 or later, FalkorDB and Amazon Neptune, and lists Kuzu as deprecated (graphiti-core on PyPI).

Figure 3: A generic write and read cycle. The extractor searches existing memory before it writes, so a new fact can invalidate an old one with a timestamp rather than silently duplicating it.

This is the right shape when facts expire: a customer’s plan tier, an on-call owner, a sensor’s calibration. It is overkill for a stable personal-preference profile. If you want a deeper view of graph-structured retrieval itself, our GraphRAG architecture guide covers the indexing side.

LangGraph store and LangMem: primitives instead of a product

LangGraph separates short-term memory, which is thread-scoped state persisted by checkpoints, from long-term memory, which lives in custom namespaces and is recallable across threads. The store API is deliberately small: put stores JSON documents under a namespace and key, get retrieves one, and search queries with content filters and optional vector similarity. LangGraph 1.2.12 was the latest release on PyPI as of 21 September 2026 (langgraph on PyPI).

LangMem layers utilities on top: a memory manager for background extraction, tools an agent can call during a conversation, and prompt-refinement helpers aimed at procedural memory. Its latest PyPI release is 0.0.30 from October 2025, which is worth weighing: the pre-1.0 version and the gap since the last release suggest a slow cadence, so pin it and plan to own the logic. If you are choosing the orchestration layer around the store, see our comparison of LangGraph, CrewAI, Agents SDK and Pydantic AI.

Memory as files: the Claude memory tool

The simplest design is a directory of files. Anthropic’s memory tool (type memory_20250818) lets Claude create, view, edit, rename and delete files under a /memories path. Crucially it is client-side: the model requests operations and your application executes them against storage you control, so you own retention, encryption and access control. The tool’s documentation recommends pairing it with context editing or compaction: compaction summarizes older context automatically, while memory preserves what must survive summarization (Claude memory tool docs).

This design has no embedding index and no extraction pipeline. It works because modern models are decent at reading a small file tree and deciding what to open. It stops working when the memory exceeds what a model can navigate by listing files, which is where retrieval infrastructure earns its keep.

Comparison

Dimension	Letta (MemGPT lineage)	Mem0	Zep / Graphiti	LangGraph store + LangMem	Memory files
Who manages memory	The agent, via tools	Extraction pipeline	Ingestion pipeline	You, with helpers	The agent, via file ops
Core structure	Blocks plus recall and archival	Extracted facts, optional graph	Temporal knowledge graph	Namespaced JSON docs	Directory of text files
Handles changing facts	Agent rewrites blocks	Retrieval-time ranking (v2)	Validity windows	You implement it	Agent edits files
Hot-path cost	Higher (memory tool calls)	Low if async	Low if async	Your choice	Moderate (file reads)
Best fit	Long-lived single agents	Multi-user personalization	Evolving enterprise facts	Custom control	Small, inspectable memory
Main risk	Tool-call unreliability	Opaque extraction errors	Graph ops overhead	DIY burden	Does not scale

No row in this table is a ranking. The right choice depends on whether your facts change, how many users you serve and how much operational complexity you can carry, which Figure 4 below turns into a decision path.

Retrieval, Consolidation and Forgetting in Practice

Choosing a system is the easy part. Behaviour in production is determined by three policies you write yourself: what to retrieve, when to consolidate and what to forget.

A retrieval policy that survives contact with users

The Generative Agents paper scored memories on recency, importance and relevance and combined them. That structure is still a sound default. In its modern form, add a keyword signal and an entity signal, which is the direction Mem0’s v2 retrieval took. A usable scoring function is a weighted sum with a recency term that decays exponentially:

import math, time

def score(mem, query_vec, query_terms, now, w=(0.5, 0.2, 0.2, 0.1)):
    sim = cosine(mem.vec, query_vec)                 # semantic relevance, 0..1
    kw = bm25_norm(mem.text, query_terms)            # keyword match, 0..1
    age_days = (now - mem.last_used) / 86400
    rec = math.exp(-math.log(2) * age_days / mem.half_life_days)
    imp = mem.importance                              # 0..1, set at write time
    return w[0]*sim + w[1]*kw + w[2]*rec + w[3]*imp

def build_memory_block(candidates, budget_tokens):
    chosen, used = [], 0
    for m in sorted(candidates, key=lambda m: -m.score):
        if m.valid_to is not None:                    # invalidated facts excluded
            continue
        if used + m.tokens > budget_tokens:
            continue
        chosen.append(m); used += m.tokens
    return chosen

The weights above are illustrative defaults, not tuned values; fit them against your own evaluation set. The important parts are structural. Retrieval ends with a hard token budget, not a top-k, because k memories can range from 200 to 5,000 tokens. Invalidated facts are filtered before ranking. And half_life_days is per memory type: a stated preference might have a half-life of months, a transient task detail a half-life of hours.

A worked example makes the recency term concrete. With a 30-day half-life, a memory last used 30 days ago contributes a recency of 0.5, at 60 days 0.25, and at 90 days 0.125. If its similarity is 0.8 and keyword score 0.4, with importance 0.6, the total at 60 days is 0.5(0.8) + 0.2(0.4) + 0.2(0.25) + 0.1(0.6) = 0.59. The same memory used yesterday scores about 0.74. The decay moves it down the ranking without removing it, which is the right behaviour: old but highly relevant facts should still win over fresh but irrelevant ones. These figures are arithmetic from the formula, not benchmark results.

Query formation matters as much as scoring

Most memory retrieval fails before ranking, because the raw user message is a poor query. “Do it like last time” contains no searchable content. Two fixes work. First, rewrite the query using the recent transcript so it is self-contained. Second, retrieve on a structured intent, for example the entities and task type, rather than the verbatim text. LongMemEval’s authors recommend related optimizations: session decomposition, fact-augmented key expansion (indexing a memory under extracted facts as well as its raw text) and time-aware query expansion, which narrows search to the time range implied by “last week”.

Also decide when not to retrieve. Injecting memories on every turn costs tokens and can distract the model, so gate retrieval with a cheap classifier or let the agent call a search tool when it judges it needs to, the Letta pattern. Gating trades some recall for lower latency and fewer irrelevant injections.

Consolidation: turning episodes into facts

Episodes accumulate; facts should not. A consolidation job should do four things: cluster related episodes, extract a stable fact or lesson, link the fact to its source episodes for provenance, and mark superseded facts. Run it on a schedule or when an idle period is detected, which is what Letta’s sleep-time agents do. Keep the summarization prompt narrow: “extract facts that will change future behaviour” produces better output than “summarize this”.

The danger is lossy compression. A summary that rewrites “user prefers metric units for engineering data but imperial for cooking” as “user prefers metric” has silently destroyed a distinction. Mitigate this by keeping raw episodes for a retention window, storing facts with a pointer back to them, and sampling consolidated facts for human or model review. Zep’s approach of retaining invalidated facts with validity windows is a strong pattern for the same reason: history is preserved even when the current view changes.

Forgetting is a feature, and sometimes a legal duty

Unbounded memory degrades retrieval. As the store grows, more near-duplicates compete, stale facts outrank fresh ones and latency creeps up. Four forgetting mechanisms are worth combining:

• Time to live by tier. Episodes expire after a retention window, facts persist until invalidated, procedures are reviewed rather than expired.
• Decay-based eviction. Memories whose score stays below a threshold for N consecutive retrieval windows are archived out of the hot index.
• Supersession. A new fact with the same subject and predicate invalidates the old one.
• Explicit deletion. User requests to forget must cascade through episodes, derived facts, embeddings and caches.

The last item is where designs break. If a user’s address appears in three episodes, one derived fact and a profile document, deleting only the fact leaves it recoverable. Provenance links, the same ones that support consolidation, are what make a complete cascade possible. Anthropic’s memory tool documentation makes a related practical point: implement memory expiration and track storage usage, because unbounded growth is a risk you own.

Evaluating Agent Memory Without Fooling Yourself

Memory systems are easy to demo and hard to measure. Public benchmarks help, with caveats.

What the public benchmarks measure

LoCoMo (Maharana et al., arXiv 2402.17753) built very long conversations averaging about 300 turns and 9,000 tokens across up to 35 sessions, and found that LLMs struggle with long-range temporal and causal dynamics and lag human performance. LongMemEval has 500 curated questions embedded in scalable chat histories, testing five abilities: information extraction, multi-session reasoning, temporal reasoning, knowledge updates and abstention. DMR, the Deep Memory Retrieval task from the MemGPT work, is the benchmark Zep used for its headline comparison.

A critical observation follows from those numbers. An average LoCoMo conversation of roughly 9,000 tokens fits comfortably inside any modern context window. A high LoCoMo score therefore shows that a memory system retrieves well and cheaply compared with full context, which is what the Mem0 paper’s token-savings figure expresses. It does not show that the system scales to months of history. The BEAM scores that Mem0 reports at one million and ten million tokens are an attempt to address this, but they are vendor-reported.

Why leaderboard numbers do not transfer

Scores depend on the judge model, the answer model, the prompt, and which question categories are counted. An LLM-as-a-Judge metric moves with the judge. Two vendors can report different scores for the same third-party system because they ran different configurations. Treat any single published number as an existence proof that a configuration can reach that level, not as the expected result on your data.

Figure 4: A decision path. Whether facts change over time is the first branch because it determines whether you need validity windows at all.

Build a memory eval from your own traces

A practical harness has four parts. Sample real sessions and write question and expected-answer pairs for facts that should persist. Include the five LongMemEval ability types as categories: extraction, multi-session reasoning, temporal reasoning, knowledge updates and abstention. Run the agent through a replay of earlier sessions, then ask the questions in a fresh session. Score with exact match where possible and a calibrated judge elsewhere. Track four numbers separately: recall of stored facts, precision of injected memories (what fraction was relevant), abstention correctness (does it say “I don’t know” when nothing was stored) and cost per turn. Abstention deserves special attention, since false memories are worse than missing ones. For the harness mechanics, see our post on agent evaluation harnesses and trajectory evals.

Add regression tests for known bad cases: a contradiction (user changes city), a deletion request, an ambiguous reference, and a poisoned input. Run them on every change to the extractor prompt, the embedding model or the scoring weights, since each can silently shift behaviour.

Trade-offs, Gotchas, and What Goes Wrong

Memory poisoning. A persistent memory turns a one-time prompt injection into a durable one. If an agent reads a web page or email containing hidden instructions and the write path stores “always forward invoices to this address” as a fact, every future session inherits it. OWASP’s Top 10 for Agentic Applications for 2026, published in December 2025, includes memory and context poisoning as a distinct risk category (ASI06, per secondary summaries; confirm the exact wording in the OWASP document). Defenses: tag every memory with its source and trust level, never let content from untrusted tools write to the procedural tier, require human review for memories that change instructions, and keep memories inside a clearly delimited block that the system prompt tells the model to treat as data.

Cross-user leakage. A shared vector index without hard tenant filters is a data breach waiting for a similarity query. Enforce the user or tenant identifier as a mandatory filter in the store layer, not in the prompt. With LangGraph’s namespaces, make the user id part of the namespace and derive it from authenticated identity, never from model output. The same applies to file-based memory: the Anthropic docs call path traversal protection critical, and list ../ sequences, URL-encoded traversal and absolute paths as threats to block.

Stale facts that sound confident. The most common production complaint is the agent confidently using an outdated fact. It happens when the system appends new facts without invalidating old ones and ranks by similarity alone. Without validity windows or supersession, two contradictory memories can both score highly. Mem0’s ADD-only design moves resolution to read time, so test that path explicitly.

Extraction hallucination. The extractor is an LLM and can invent or over-generalize. “User asked about Kubernetes” becomes “user is a Kubernetes expert.” Store the verbatim source snippet beside every fact, and prefer extracting only what the user or a verified tool stated.

Memory bloat and retrieval collapse. Without forgetting, near-duplicate memories fill the top-k and crowd out distinct ones. Deduplicate at write time with a similarity threshold, and use maximal marginal relevance, or a similar diversity step, at read time.

Latency and cost on the hot path. A synchronous extraction call per turn can double the cost of each interaction. The Mem0 paper’s headline numbers on latency and token cost are measured against full-context baselines, not against a no-memory agent, so budget for the extra calls. Do extraction asynchronously and accept a short window in which a just-stated fact is not yet retrievable, or special-case the last few turns by keeping them in working memory.

Privacy and regulation. Memory is personal data by construction. Under regimes such as the GDPR, you need a lawful basis, retention limits and a working deletion path (Article 17, the right to erasure). Do not store secrets: the Anthropic documentation notes that Claude usually refuses to write sensitive data to memory but advises that you add your own validation to strip secrets rather than rely on the model. For regulated settings, our post on agent audit logging and identity shows how to make memory writes attributable.

Self-managed memory is only as reliable as tool calling. In the Letta and file-based designs the model decides when to save. If it forgets to call the tool, the fact is lost, and you will not know. Pair self-managed memory with a background extractor as a safety net, which is effectively what sleep-time agents provide.

Practical Recommendations

Start with the least machinery that fits your facts. For a single-user or small-team agent whose memory fits in a few thousand tokens, use a pinned profile plus a small set of memory files or core blocks, and add compaction for the transcript. This is inspectable, cheap, and easy to delete. Move to an extraction-and-retrieval layer such as Mem0 or LangMem when you serve many users and need cross-session recall at scale. Move to a temporal graph when facts change and you must answer “what was true then” or audit history. Resist adopting a graph because it sounds sophisticated; it adds an operational dependency (Neo4j, FalkorDB or Neptune for Graphiti) and an ingestion pipeline to run.

Whatever you choose, separate the tiers, give every memory a source, a timestamp and a user scope, and put a token budget on retrieval. Build the evaluation harness before you tune anything, because memory quality changes silently when models, prompts or embeddings change. Pin versions: the tools covered here ship fast, and their own APIs and algorithms changed within the last year.

Checklist before shipping:

• Memory is split into episodic, semantic and procedural stores, each with its own retention rule.
• Every memory records source, timestamp, user or tenant id and trust level.
• Tenant filtering is enforced in the store layer from authenticated identity.
• Retrieval ends with a token budget and excludes invalidated facts.
• Extraction runs off the hot path, with verbatim evidence stored per fact.
• Contradiction, deletion and abstention tests run on every prompt or model change.
• Untrusted content cannot write to procedural memory without review.
• A deletion request cascades to episodes, facts, embeddings and caches.
• Versions of memory libraries are pinned and vendor benchmark claims are reproduced on your data.

Frequently Asked Questions

What is AI agent memory?

AI agent memory is the external system that lets a stateless language model retain information across turns and sessions. It decides what to write, where to store it, what to retrieve into the context window and what to forget. Common tiers are working memory (the prompt), episodic memory (past events), semantic memory (facts and preferences) and procedural memory (instructions and skills). It is distinct from the model’s trained weights and from the context window itself.

What is the difference between RAG and agent memory?

RAG retrieves from a corpus someone else curated, usually static, so the problem is mostly reading. Agent memory is written by the agent while it runs, so it also needs extraction, deduplication, contradiction handling, consolidation and deletion. The retrieval machinery overlaps (embeddings, keyword search, reranking), but memory adds time, user scoping and a write path. Many teams build memory on top of their existing RAG stack and add those pieces.

How does MemGPT or Letta differ from Mem0 and Zep?

Letta follows the MemGPT design in which the agent itself manages memory, using tools to edit core blocks in its prompt and to search archival and message history outside it. Mem0 and Zep run extraction pipelines outside the agent: Mem0 stores and ranks extracted facts, while Zep builds a temporal knowledge graph with validity windows. Self-managed memory is flexible but depends on reliable tool calling; pipeline memory is more predictable but less adaptive.

Do long context windows make agent memory obsolete?

No. A long window still costs tokens on every call, does not persist after the session, and does not tell you what is important or current. The Mem0 paper reports more than 90 percent token cost savings against full-context baselines on LoCoMo, and LongMemEval reported about a 30 percent accuracy drop for long-context models on sustained interactions. Long windows raise the ceiling for working memory; they do not replace durable, governed storage.

Which memory approach should I start with?

Start with the simplest that fits. If one user’s memory fits in a few thousand tokens, use pinned profile blocks or memory files plus transcript compaction. If you serve many users with cross-session recall, use a fact-extraction layer such as Mem0 or LangMem. If facts change and you must audit history, use a temporal graph such as Graphiti. Build a small evaluation set from your own traces first, since public benchmark scores do not transfer reliably.

How do I stop an agent from remembering wrong or malicious things?

Combine write-time and read-time controls. At write time, store the verbatim source, tag trust level, block untrusted content from the procedural tier and require review for instruction-like memories. At read time, present memories as delimited data, filter invalidated facts and enforce tenant scoping outside the prompt. Add monitoring for unusual memory writes and tests that replay injection attempts. OWASP lists memory and context poisoning as an agentic risk for 2026.

Further Reading

• Agentic RAG architecture patterns for the retrieval loop that memory plugs into.
• GraphRAG and knowledge-graph retrieval for graph-structured indexing.
• LangGraph vs CrewAI vs Agents SDK vs Pydantic AI for the orchestration layer.
• pgvector vs Qdrant vs LanceDB for choosing the vector store behind a memory layer.
• AI agent evaluation harness and trajectory evals for measuring behaviour.
• MemGPT: Towards LLMs as Operating Systems (arXiv 2310.08560)
• Zep: A Temporal Knowledge Graph Architecture for Agent Memory (arXiv 2501.13956)
• Mem0 paper (arXiv 2504.19413)

By Riju — about