The Forgetting Problem in AI Memory Systems: A Comprehensive Research Report

1. Problem Statement and Landscape

The “forgetting problem” in AI memory systems encompasses a fundamental tension: AI agents must retain useful knowledge over time while discarding what is outdated, redundant, contradictory, or harmful. Unlike human cognition – where forgetting is an adaptive feature that prevents cognitive overload and maintains information relevance – current AI systems employ largely binary retention strategies that either preserve everything (leading to memory bloat, contradiction accumulation, and retrieval degradation) or lose information catastrophically at context boundaries.

The HaluMem benchmark (November 2024), the first operation-level hallucination evaluation benchmark for memory systems, revealed that all tested systems had memory accuracy below 62%, with update accuracy below 26% and steadily worsening at scale. Memory hallucinations compound across extraction, updating, and retrieval stages, propagating errors downstream. This makes principled forgetting not a limitation but a design requirement.

The field has exploded in 2024-2026, culminating in the ICLR 2026 MemAgents Workshop (April 2026), which formally recognizes that “the limiting factor is increasingly not raw model capability but memory: how agents encode, retain, retrieve, and consolidate experience.”

---

2. Taxonomies of AI Memory

2.1 Memory Forms (from “Rethinking Memory in AI,” May 2025)

The field now recognizes three primary memory representations:

• Parametric Memory: Knowledge embedded in model weights during pretraining/fine-tuning. Enables fast, context-free retrieval but lacks transparency and selectivity.
• Contextual Unstructured Memory: Explicit text/image/audio storage divided into short-term (session) and long-term (cross-session) stores.
• Contextual Structured Memory: Knowledge graphs, relational tables, and ontologies supporting symbolic reasoning and precise querying.

2.2 Memory Operations Framework

The “Memory in the Age of AI Agents” survey (December 2025) identifies four management operations and two utilization operations:

Management: Consolidation, Indexing, Updating, Forgetting
Utilization: Retrieval, Compression

Forgetting is defined as “selectively suppressing memory content that may be outdated, irrelevant, or harmful.” In parametric memory, this is implemented through unlearning techniques; in contextual memory, through time-based/semantic filtering and pruning policies.

---

3. Spaced Repetition Adapted for AI

3.1 MemoryBank (AAAI 2024)

The pioneering system adapting the Ebbinghaus forgetting curve for LLM agents. Memory retention follows:

R = e^(-t/S)

where R is retention probability, t is elapsed time, and S is memory strength. When a memory is recalled, S increments by 1 and t resets to 0, implementing a direct analog of spaced repetition: frequently accessed memories persist longer.

3.2 FOREVER (January 2026)

FOREVER (FORgEtting curVe-inspired mEmory Replay) introduces a critical innovation: model-centric time. Rather than counting training steps (which poorly reflect actual learning), it defines time through parameter update magnitude:

• Parameter change at step t: Delta_t = ||Theta_t - Theta_{t-1}||_2
• Accumulated model time: tau_t = Sum(Delta_i, i=1..t)
• Virtual “model day” calibrated from initial training: tau_day = Sum(Delta_i, i=1..S)

Human Ebbinghaus intervals {1, 2, 4, 7, 15, 30 days} map onto model time as D_model = {d * tau_day | d in D_human}. Replay triggers when accumulated model time reaches thresholds, with intensity-aware regularization that scales replay strength adaptively based on an instability ratio comparing recent to baseline update intensity. Results: +1.4% overall performance over prior best on standard continual learning benchmarks, with consistent gains across 0.6B-13B parameter models.

3.3 DRL-SRS (2024)

Deep Reinforcement Learning for Spaced Repetition Scheduling models the learner as an environment and uses DQN agents to learn optimal review intervals, combining a Transformer-based model for recall probability estimation with reinforcement learning for policy optimization.

---

4. Temporal Decay Functions

4.1 FadeMem (January 2026) – The State of the Art

FadeMem implements the most sophisticated biologically-inspired decay system to date. Each memory unit carries a strength value v_i(t) in [0,1] governed by:

v_i(t) = v_i(0) * exp(-lambda_i * (t - tau_i)^beta_i)

The decay rate adapts based on importance:

lambda_i = lambda_base * exp(-mu * I_i(t))

where importance I_i(t) combines semantic relevance, frequency, and recency:

I_i(t) = alpha * rel(c_i, Q_t) + beta * f_i/(1+f_i) + gamma * recency(tau_i, t)

Shape parameters differ by memory layer: beta = 0.8 (sub-linear) for long-term memory and beta = 1.2 (super-linear) for short-term memory, creating 33-5x slower decay for important memories. At baseline importance, long-term memories have a half-life of approximately 11.25 days while short-term memories decay with a half-life of around 5.02 days.

Results: 82.1% critical fact retention using only 55% storage (vs. Mem0’s 78.4% at 100% storage); 45% storage reduction overall; F1=29.43 on LoCoMo multi-hop reasoning.

4.2 CortexGraph/Mnemex (2025)

An open-source implementation supporting multiple decay models (power law, exponential, two-component) with configurable parameters. The exponential model uses lambda = 2.673e-6 for a 3-day half-life. The composite scoring formula:

strength = importance * e^(-lambda_eff * days) * (1 + recall_count * 0.2)

Promotion thresholds trigger memory elevation to permanent storage when score exceeds 0.65 or usage reaches 5 times within 14 days.

4.3 A-MAC Recency Function (March 2026)

The Adaptive Memory Admission Control framework uses a simpler but effective exponential decay for its recency component:

R(m) = exp(-lambda * tau(m))

with lambda = 0.01 per hour (69-hour half-life). This is combined with four other factors (utility, confidence, novelty, content type prior) in a weighted linear score for admission decisions.

---

5. Information-Theoretic and Principled Pruning Approaches

5.1 A-MAC: Five-Factor Decomposition (March 2026, ICLR 2026 Workshop)

A-MAC decomposes memory value into five interpretable dimensions:

1. Utility (U): LLM-assessed future relevance (97.6% of computation time)
2. Confidence (C): Factual grounding via ROUGE-L overlap with evidence spans – directly mitigates hallucination propagation
3. Novelty (N): Redundancy prevention via Sentence-BERT cosine distance: N(m) = 1 - max_{m’ in M} cos(phi(m), phi(m’))
4. Recency (R): Exponential temporal decay
5. Content Type Prior (T): Rule-based persistence preferences (highest-impact factor in ablation: -0.107 F1 when removed)

Composite score: S(m) = w1U + w2C + w3N + w4R + w5*T, with admission threshold theta learned via cross-validation.

Results: F1 = 0.583 on LoCoMo (+7.8% over prior SOTA A-Mem), with 31% latency reduction.

5.2 Forgetful but Faithful (December 2025)

This framework formalizes memory retention as a submodular knapsack optimization under provenance-closure constraints:

• Budget constraint: Sum(w_i) <= B
• Utility: U(S) = omega_NC * u_NC(S) + omega_GCR * u_GCR(S) + omega_SRA * u_SRA(S)

It implements six forgetting mechanisms (FIFO, LRU, priority decay, reflection-summary, sensitivity-weighted, exponential mechanism) composable as: f_hyb = f_temporal . f_reflect . f_importance . f_privacy

Each operates on a unified density score: score(i) = (U_hat_i - lambda_priv * s_i) / w_i

A key theoretical result: under exponential time-decay utility U(S) = Sum(v_i * e^(-lambda * age(i))), LRU eviction order is provably optimal. The hybrid policy achieved 0.911 composite performance score with differential privacy guarantees: (epsilon, delta)-DP via the exponential mechanism.

5.3 Machine Unlearning for Parametric Memory (2024-2025)

For knowledge embedded in model weights, targeted removal techniques include:

• Gradient Ascent: Reversing the learning process on specific data
• Negative Preference Optimization (NPO): DPO variant using only negative responses from the forget set
• ULD (Unlearning from Logit Difference): Trains a reversed-objective assistant model and subtracts its logits
• FALCON: Uses mutual information of activations to identify layers where forget/retain knowledge is least entangled
• SeUL: Applies gradient ascent to specific sensitive spans rather than entire sequences
• LoRA-based Unlearning: Freezes base model weights, introduces adapters for modular, continual unlearning

IBM demonstrated that machine unlearning reduced toxicity from 15.4% to 4.8% on Llama in 224 seconds without degrading other capabilities.

---

6. Contradiction Resolution Strategies

6.1 Knowledge Conflict Taxonomy (EMNLP 2024)

Xu et al. established the definitive taxonomy of knowledge conflicts in LLMs:

• Context-Memory Conflict: External context contradicts parametric knowledge
• Inter-Context Conflict: Multiple retrieved sources disagree (noise, outdated information, misinformation)
• Intra-Memory Conflict: Model’s own parametric knowledge yields divergent responses to differently-phrased queries

6.2 Resolution Frameworks

Astute RAG (Google, October 2024): Adaptively elicits LLM internal knowledge, iteratively consolidates internal and external knowledge with source-awareness, and finalizes answers according to reliability estimation. Analysis showed 19.2% of data exhibit conflicts, with internal knowledge correct 47.4% of the time – demonstrating that neither source should be blindly trusted.

FaithfulRAG (2025): Decomposes retrieved evidence into atomic claims, then guides generation through a “self-thinking” phase that resolves inconsistencies at the fact level rather than document level.

TCR – Transparent Conflict Resolution (2026): A plug-and-play framework that disentangles semantic match from factual consistency via dual contrastive encoders, estimates “self-answerability” to gauge internal confidence, and feeds three scalar signals to the generator through lightweight soft-prompts with SNR-based weighting.

6.3 FadeMem’s Conflict Resolution

When new information arrives, FadeMem identifies semantically similar memories (similarity > 0.75 threshold) and classifies relationships as compatible, contradictory, subsumes, or subsumed:

• Compatible: Coexist with redundancy penalties
• Contradictory: Competitive dynamics where older memory strength decays: v_i(t) = v_i(t) * exp(-rho * clip((tau_new - tau_i)/W_age, 0, 1)) – newer information gradually displaces older contradictory information
• Subsumes/Subsumed: Triggers intelligent fusion via LLM

Result: 68.9% macro-averaged accuracy across three conflict types.

6.4 Hindsight’s Opinion Evolution (December 2025)

The Hindsight system separates facts from opinions structurally, maintaining confidence scores c in [0,1] for all opinions. When new evidence arrives, scores update:

• Reinforce: c’ = min(c + alpha, 1.0)
• Weaken: c’ = max(c - alpha, 0.0)
• Contradict: c’ = max(c - 2*alpha, 0.0)
• Neutral: c’ = c

This achieves 89.61% accuracy on LoCoMo (vs. prior SOTA 75.78%).

6.5 Multi-Agent Conflict Resolution

For multi-agent systems with shared memory, emerging solutions include:

• Optimistic concurrency with merge strategies for rare, resolvable conflicts
• Conflict-free Replicated Data Types (CRDTs) guaranteeing eventual consistency without locking
• CQRS (Command Query Responsibility Segregation) separating write and read models
• Escalation policies routing unresolvable conflicts to supervisor agents or humans

---

7. Catastrophic Forgetting: Parametric-Level Solutions

7.1 MESU – Metaplasticity from Synaptic Uncertainty (Nature Communications, 2025)

A Bayesian learning rule where uncertainty modulates plasticity:

Mean update: Delta_mu = -sigma^2 * (dC_t/dmu) + (sigma^2 / (N * sigma_prior^2)) * (mu_prior - mu)

The synaptic variance sigma^2 appears explicitly in front of gradient terms instead of a fixed learning rate, embodying metaplasticity: uncertain weights adapt readily while confident ones remain stable.

Controlled forgetting operates through a truncated posterior retaining only N recent tasks. Variance converges to a bounded value: sigma^2_inf = (1/N) * 1/[H_D(mu_0) + 1/(N * sigma_prior^2)], preventing both overconfidence and complete plasticity loss.

Results: 91.37% on Permuted MNIST (200 tasks) vs. EWC Online 88.5%; maintains ROC AUC near 1.0 for OOD detection even after 1000 epochs. Requires neither task boundaries nor replay.

7.2 Neural ODE + Memory-Augmented Transformers (Scientific Reports, 2025)

The first systematic integration of continuous-time dynamics (Neural ODEs) with attention-based memory retrieval. Neural ODEs enable smooth representation learning while memory-augmented transformers provide explicit knowledge consolidation. Includes rigorous PAC-learning theoretical bounds.

Results: 24% forgetting reduction and 10.3% accuracy gain over state-of-the-art.

---

8. System-Level Memory Operating Systems

8.1 MemOS (July 2025)

The first Memory Operating System for LLMs, treating memory as a first-class system resource. Supports multi-modal memory (text, images, tool traces, personas) with unified APIs for add/retrieve/edit/delete operations. Memory lifecycle rules allow agents to “age” memories with conflict detection, deduplication, versioning, and configurable forgetting policies.

8.2 EverMemOS (January 2026)

Implements an engram-inspired three-phase lifecycle:

1. Episodic Trace Formation: Dialogue streams become MemCells with episodic traces and time-bounded foresight signals
2. Semantic Consolidation: MemCells organize into thematic MemScenes, distilling stable semantic structures
3. Reconstructive Recollection: MemScene-guided retrieval composes minimal sufficient context

Achieves state-of-the-art on both LoCoMo and LongMemEval.

8.3 A-Mem (NeurIPS 2025)

Turing/Zettelkasten-inspired self-organizing memory that creates interconnected knowledge networks through dynamic indexing and linking. All memory organization (creation, linking, evolution) is governed by the agent itself, not external rules. Doubled performance in complex reasoning with reduced token costs.

---

9. Key Research Frontiers (2026 and Beyond)

Based on the ICLR 2026 MemAgents workshop and recent surveys, the critical open problems are:

1. Memory Automation: Moving from hand-coded forgetting rules to learned retention/eviction policies via reinforcement learning
2. Spatio-Temporal Memory: Capturing structural relationships and temporal evolution simultaneously
3. Multi-Agent Memory Governance: Standardized protocols for concurrent access, conflict resolution, and privacy in shared memory
4. Unified Memory Representation: Shared indexing across parametric, structured, and unstructured memory
5. Forgetting Quality Metrics: Evaluation frameworks that assess not just what is remembered but whether forgetting decisions were correct
6. Privacy-Preserving Forgetting: Formal differential privacy guarantees for memory eviction decisions
7. Brain-Inspired Architectures: Implementing complementary learning systems with fast hippocampal encoding and slow cortical consolidation

---

Summary of Key Systems Comparison

System	Year	Decay Model	Conflict Resolution	Storage Efficiency	Benchmark Performance
MemoryBank	2024	Ebbinghaus exponential	None	No pruning	AAAI baseline
A-Mem	2025	None (self-organizing)	Dynamic linking	Token cost reduction	NeurIPS 2025 SOTA
Hindsight	2025	Temporal graph edges	Opinion confidence scores	N/A	89.61% LoCoMo
FadeMem	2026	Adaptive exponential, dual-layer	Competitive dynamics + LLM fusion	45% reduction	82.1% retention at 55% storage
A-MAC	2026	Exponential (69h half-life)	Score-based replacement	Admission control	F1=0.583 LoCoMo
Forgetful but Faithful	2025	Composable policies	Provenance-based	Budget-constrained	0.911 composite

---

Sources:
- FadeMem: Biologically-Inspired Forgetting for Efficient Agent Memory
- Adaptive Memory Admission Control for LLM Agents (A-MAC)
- Memory in the Age of AI Agents: A Survey
- Rethinking Memory in AI: Taxonomy, Operations, Topics, and Future Directions
- Knowledge Conflicts for LLMs: A Survey (EMNLP 2024)
- Hindsight is 20/20: Building Agent Memory that Retains, Recalls, and Reflects
- Forgetful but Faithful: A Cognitive Memory Architecture and Benchmark for Privacy-Aware Generative Agents
- Bayesian Continual Learning and Forgetting (MESU) - Nature Communications
- FOREVER: Forgetting Curve-Inspired Memory Replay for Language Model Continual Learning
- Mitigating Catastrophic Forgetting: Neural ODEs with Memory-Augmented Transformers
- HaluMem: Evaluating Hallucinations in Memory Systems of Agents
- Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts
- A-MEM: Agentic Memory for LLM Agents (NeurIPS 2025)
- MemOS: A Memory OS for AI Systems
- EverMemOS: A Self-Organizing Memory Operating System
- ICLR 2026 MemAgents Workshop
- CortexGraph (formerly Mnemex)
- Machine Unlearning for LLMs Survey
- DRL-SRS: Deep Reinforcement Learning for Spaced Repetition
- Mastering Memory Consistency in AI Agents: 2025 Insights
- Multi-Agent Memory from a Computer Architecture Perspective
- Seeing through the Conflict: Transparent Knowledge Conflict Handling in RAG
- A Survey on the Memory Mechanism of LLM-based Agents (ACM TOIS)