AI Agent Memory - Mem0, Zep, Letta or LangMem? How to Choose and Implement Memory Across Sessions

AI agent memory is the mechanism that lets a language model retain information beyond a single conversation - because the LLM itself is stateless and forgets everything once the session ends. Which route to take? A rule based on what the agent needs to remember: if a dozen or so fixed fields about the user are enough (name, plan, preferred contact channel), you don't need a framework - a profile table in your database will do. When there are dozens or hundreds of facts and they emerge in conversations, take Mem0 (fastest start, free plan, Apache 2.0 licence). When relationships and how they change over time matter (who with whom, since when), choose Zep / Graphiti. When the agent runs for hours or days and manages its own memory - Letta. When your whole system already runs on LangGraph - LangMem.

Mem0, Zep, Letta or LangMem - or is a profile table enough? A one-paragraph decision rule, a comparison of licences, benchmark results and prices, cost at 1k and 100k users, memory types, Mem0 code, memory in n8n and GDPR compliance.

A customer writes to your chatbot: "the same problem as last time again." The agent replies: "Could you describe the problem?" The customer already knows they are talking to a machine with no memory - and trust evaporates. Meanwhile a human support agent would open the history and say: "I see you reported this two weeks ago, let's check whether the fix worked."

That difference - remembering context between conversations - separates a demo toy from an agent you can deploy in a company. And because a language model inherently remembers nothing between calls, memory has to be designed and built separately. This article shows how: from memory types, through context management and frameworks, to code, n8n and GDPR.

Why an LLM does not remember - the statelessness problem

Every call to a language model is independent. The model has no "state" between requests - it takes text in, returns text out and immediately forgets. The impression that ChatGPT "remembers" a conversation is an illusion: each time, the application sends the entire conversation history so far to the model as part of the prompt. It is not the model remembering - it is the application re-attaching the history.

This mechanism works up to a point, then hits three hard walls:

• Context window limit - even models with 200k-1M token windows have an upper bound; a long conversation or large documents eventually will not fit
• Cost and latency - you pay for every input token on every call; re-attaching the full history to each request means cost grows quadratically with conversation length
• No continuity across sessions - when the user closes the chat and returns tomorrow, the history is empty; without external storage the agent starts from zero

AI agent memory solves all three: instead of re-attaching everything, it stores information externally and retrieves only what is relevant to the current decision. This is exactly the same idea as RAG for company knowledge (I covered it in the article on building a knowledge base) - just applied to interaction history and facts about the user.

The four types of AI agent memory

Research on agent architecture (including "Cognitive Architectures for Language Agents") distinguishes four memory types borrowed from cognitive psychology. Understanding the differences is crucial, because each type needs a different store and a different retrieval strategy - and the most common mistake is dumping everything into one vector database.

• Working memory - the active context window: the current conversation, loaded files, tool results from this session. You manage it like a token budget, not a search problem - through compression and prioritization, not similarity search
• Episodic memory - a record of what the agent did and when: conversation traces, decisions made, action trails. Used for auditing, debugging and learning from history. The key is chronological storage, not similarity search
• Semantic memory - facts about the world, the user and the domain: customer preferences, industry knowledge, company data. RAG was built for this - content-similarity retrieval is the right approach here
• Procedural memory - how to perform a task: learned procedures, reusable skills, routines. In practice stored in system prompts, tool definitions and the agent's code

The most important design rule: do not mix episodic memory (event logs) with semantic memory (facts) in one vector index. Similarity search over event logs degrades retrieval quality for both - a log "user clicked X at 14:32" and a fact "user prefers email contact" require completely different access strategies.

Short-term memory - managing the context window

Short-term memory is managing what fits in the context window during one session. In multi-step agents, tool results quickly take up most of the token budget, so context has to be kept in check: a sliding window of recent turns, a rolling summary of the older part of the conversation, compaction at a token threshold and trimming tool results. For most business uses a rolling summary is enough: the last few exchanges in full plus an updated summary of the rest.

I cover in-session context techniques (write, select, compress, isolate) and a practical context budget in detail in the post on context engineering. Here I focus on what carries knowledge across sessions.

Watch the trap: compression is lossy. Every summary loses detail, and if the agent summarizes summaries, after a few iterations it is left with a vague caricature of the conversation. That is why you should extract important facts (order number, agreements, decisions) into long-term memory as structured data before they go under the compression knife.

Long-term memory - how an agent remembers across sessions

Long-term memory lives outside the context window - in an external database, usually a vector store - and survives closing the chat, restarting the server or the user returning a week later. Its lifecycle has three phases:

1. 1.Write - during or after a conversation the agent extracts important facts and stores them. Key: you do not store the raw transcript, but extracted, structured information ("customer X prefers courier delivery", "project Y has a June 30 deadline")
2. 2.Retrieve - before the agent responds, it searches semantic memory for facts relevant to the current query and injects them into the context window. This is the core RAG pattern: keep knowledge outside, pull in only what is needed
3. 3.Update - when new information contradicts old information, memory must be updated, not just appended. Otherwise you accumulate conflicting facts ("prefers email" and "prefers phone") and the agent will guess

The last phase is the hardest and most often skipped. Good memory frameworks do deduplication and conflict resolution automatically - they detect that a new fact replaces an old one and overwrite it instead of multiplying versions. That is why it is worth reaching for a ready-made framework instead of building memory on a raw vector database: writing and retrieving you can code in an hour, but managing the lifecycle of facts is months of refinement.

Do You Need a Memory Framework at All - the First Decision

The most important question isn't "Mem0 or Zep" but "do I need a memory framework". A framework solves one specific problem: facts that emerge in conversations have to be extracted, stored, updated when they conflict and retrieved only when relevant. If you don't have that problem, a framework just adds cost and another dependency.

A practical rule: start with a profile table and summaries, and add a framework only when you notice important facts from conversations getting lost or contradicting each other.

Mem0 vs Zep vs Letta vs LangMem - Memory Framework Comparison

By 2026 the agent memory market has matured enough that you do not have to build it yourself. Four frameworks dominate, each with a different architectural approach:

The decision rule in practice:

• Start with Mem0 unless you have a strong reason to choose otherwise - lowest entry barrier, the managed SaaS removes graph-database and scaling overhead, and self-hosting is available for privacy requirements
• Choose Zep when understanding relationships and how they change over time is key - who works with whom, how preferences evolve, links between entities; here a temporal graph beats a plain vector store
• Consider Letta when you are building an autonomous agent running for hours or days and need memory as a first-class runtime element, not a library bolted on the side
• Use LangMem if your stack is already LangGraph - you get memory tooling that fits your existing orchestration without adding a new dependency

Numbers worth knowing before you choose (as of October 2026):

An important caveat: the benchmark results come from the authors' own papers (Mem0 and Zep), so treat them as claims, not an independent test. The best benchmark is 30-50 real scenarios of returning users from your own application.

There is no single winner - the choice depends on whether you value deployment speed (Mem0), accuracy and relationships (Zep), control (Letta) or stack consistency (LangMem).

What It Costs at Two Scales

An estimate for an assistant where each active user sends about 20 messages a month, and each message means one memory write and one retrieval:

The conclusion is the same as with vector databases: managed hosting pays off at the start because it saves team time, while at large scale self-hosting wins by an order of magnitude. I calculated extraction cost for GPT-6 Luna ($0.10 per million input tokens) and about 1,500 tokens per extraction. Watch out for a trap in Mem0's plans: the retrieval limit is ten times lower than the write limit, and it's retrievals that grow with every message.

How to implement memory in code - a Mem0 example

The simplest route is Mem0. Below is a customer support agent that writes and retrieves memory per user - just a few lines beyond a normal model call:

from mem0 import Memoryfrom openai import OpenAImemory = Memory()client = OpenAI()def chat(user_id: str, message: str) -> str:    # 1. Retrieve facts relevant to the current query    relevant = memory.search(query=message, user_id=user_id, limit=5)    context = "\n".join(m["memory"] for m in relevant["results"])    # 2. Inject memory into the system prompt    system = (        "You are a customer support assistant. "        "Known facts about the user:\n" + (context or "(none)")    )    resp = client.chat.completions.create(        model="gpt-6-luna",        messages=[            {"role": "system", "content": system},            {"role": "user", "content": message},        ],    )    answer = resp.choices[0].message.content    # 3. Store new facts from this exchange (extraction happens automatically)    memory.add(        messages=[            {"role": "user", "content": message},            {"role": "assistant", "content": answer},        ],        user_id=user_id,    )    return answer

Three things that make a difference here:

• user_id isolates memory - each customer has their own space; Mem0 never mixes two users' facts, which is critical for privacy
• memory.add() does not store the raw conversation - under the hood an LLM extracts facts worth remembering and discards small talk; you do not clutter the database with "hello" and "thanks"
• search() retrieves only the top 5 - you inject a few of the most relevant facts into context, not the whole history; cost and latency stay constant no matter how long the customer has been with you

For self-hosting (Ollama and vLLM) you swap OpenAI for a local Ollama or vLLM endpoint, and configure Mem0 to use a local model for extraction and a local vector store - all memory stays in your infrastructure.

Memory in n8n and no-code tools

Not every deployment needs code. In n8n the AI Agent node has built-in memory options you configure by clicking:

• Simple Memory (window buffer) - keeps the last N messages in instance memory; short-term memory within one workflow, the simplest but volatile (gone after a restart)
• Memory in an external database - connect Postgres, Redis or a vector store as the store; memory survives restarts and works across sessions
• Mem0 / Zep via an HTTP node or a dedicated integration - full long-term memory with fact extraction, without writing code

The key in n8n is the session key - the identifier the agent uses to know whose memory to load. Usually it is a CRM customer ID, a phone number or an email address. Without a stable session key every conversation is anonymous and memory does not work across sessions. This is exactly the same mechanism as user_id in code - just set in the interface.

For simple cases (a FAQ assistant remembering context within one conversation) Simple Memory is enough. For an agent that should recognize a returning customer and their history - you need memory in an external database with a sensible session key.

Memory security and privacy (GDPR)

Agent memory is by definition a store of personal data - preferences, contact history, sometimes sensitive data. That places it squarely under GDPR and requires designing compliance from the start, not after the fact:

• Right to be forgotten (Art. 17) - you must be able to delete a specific user's entire memory on request; this is why isolation by user_id is not just good practice but a requirement - without it you cannot erase one person's data without touching others'
• Data minimization (Art. 5) - store only facts needed for the agent to function; fact extraction (instead of raw transcripts) helps, but configure it so it does not capture sensitive data without a legal basis
• Tenant isolation (multi-tenancy) - in an app serving multiple companies, one client's memory must never leak into another's context; test this deliberately, because an isolation bug is a data breach
• Encryption and data location - for sensitive data consider self-hosting memory (a local vector store + a local extraction model) so data never leaves your infrastructure - this is an argument for self-hosted Mem0/Zep over managed SaaS
• Prompt injection via memory - if the agent writes user-supplied content to memory, an attacker can inject an instruction that executes on the next retrieval; memory is another vector from the article on prompt injection - treat stored content as untrusted data

The practical takeaway: before deploying memory to production, answer the question "how do I delete one user's data on request?" If you do not have a simple answer, the memory architecture is not GDPR-ready.

AI agent memory deployment checklist

1. 1.Separate memory types: working (context), episodic (logs), semantic (facts) - do not dump everything into one vector database
2. 2.For short-term memory start with a rolling summary: last 8-10 exchanges in full + a living summary of older ones
3. 3.Extract important facts into long-term memory as structured data before compression loses them
4. 4.Choose a framework: Mem0 as the default, Zep for relationships over time, Letta for autonomous agents, LangMem for a LangGraph stack
5. 5.Isolate memory by user_id / session key from day one - it is the foundation of privacy and GDPR
6. 6.Configure fact extraction instead of storing raw transcripts - smaller database, lower cost, less personal data
7. 7.Handle update and deduplication: a new fact should overwrite a conflicting old one, not pile up beside it
8. 8.Retrieve only the top-K relevant facts (3-5) - cost and latency stay constant regardless of history
9. 9.For sensitive data consider self-hosting memory (local store + local extraction model)
10. 10.Treat stored user content as untrusted data - memory is a prompt injection vector
11. 11.Implement per-user memory deletion (right to be forgotten) before production
12. 12.Measure quality: does the agent retrieve the right facts? Test on realistic returning-user scenarios

Key takeaways

An LLM is stateless - memory must be built separately, at two levels. Short-term is managing the context window through summaries and compression; long-term is writing facts to an external store and retrieving them with the RAG pattern. Separate the four memory types (working, episodic, semantic, procedural) and do not dump everything into one vector database. Do not build from scratch - Mem0 is the default choice, Zep for relationships over time, Letta for autonomous agents, LangMem for LangGraph. From day one, isolate memory by user_id, extract facts instead of transcripts and plan per-user deletion - because agent memory is a store of personal data under the full GDPR regime and another prompt injection vector.

Memory across sessions is one layer. The other is managing the context window within a single session - covered in the post on context engineering.

---

I help companies design and deploy memory for AI agents and chatbots - from choosing the architecture and framework (Mem0, Zep, Letta), through integration with code or n8n, to GDPR compliance and security. Get in touch - I start with a review of what your agent should remember and whether it needs a framework for that.