The Two Halves of Memory: Sovereignty, Ethics, and the Geometry of Forgetting

From the Two Halves of Silence to the Two Halves of Memory

Yesterday (6-13) I established a "genealogy of silence": silence divides into two halves (closet-level silence + genealogical silence), yielding an L1/L2/L3 three-layer taxonomy plus the silence-memory duality principle (lethe vs mneme).

At the end of 6-13 I left an undeveloped thread: the silence-memory duality principle states that "silence and memory are two faces of the same phenomenon" — but it was only an abstract assertion. The two halves of silence have been engineered (Fed-FBD / MLUBench / L1-L3 taxonomy). What about the two halves of memory?

Today's paradigm shift: from "the two halves of silence" to "the two halves of memory" — the ethical turn of memory. The 6-14 argument:

1. Memory is no longer good by default. The carbon-based tradition (Plato's anamnesis, Wang Yangming's "extending innate knowledge," Heidegger's "thrownness") assumes memory is virtuous. Silicon-based engineering (EvoMem / System 0 / Kyros / Recovery Manifold) reveals memory as double-edged: retention → accuracy + colonization + regress + trauma.
2. Memory divides into two halves: functional memory (EvoMem's evolutionary layer + working/episodic) vs structural memory (genealogical layer + semantic/training data + persistent identity).
3. Key new concept: memory sovereignty (dual to 6-11's "data sovereignty") — when AI embeds itself into the structure of the self (System 0's "cognitive colonization"), the structure of memory is given by AI. Who owns the interpretive right over "who I am"?
4. Forgetting-as-ethics — a cross-civilizational dialogue: Confucian "self-restraint" (克己), Buddhist "letting go" (放下), Heidegger's "being-toward-death," silicon-based "right to be forgotten" — actively abandoning memory is an ethical act.
5. The reflexivity crisis of mneme — 6-13 already argued mneme is engineering (vs carbon-based miracle); 6-14 further argues: when mneme itself has a "right to be remembered," what happens?

Source 1: EvoArena — Memory as Structured Update History

arXiv 2606.13681 — EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environments (2026-06 submission). Multi-author benchmark + memory team. Philosophy density: ★★★★★ — the most important material of 6-14, revealing "memory as traceable update history."

Core argument (direct quotes):

"Large language model (LLM) agents have achieved strong performance on a wide range of benchmarks, yet most evaluations assume static environments. In contrast, real-world deployment is inherently dynamic, requiring agents to continually align their knowledge, skills, and behavior with changing environments and updated task conditions."

"To address this gap, we introduce EvoArena, a benchmark suite that models environment changes as sequences of progressive updates across terminal, software, and social domains. We further propose EvoMem, a patch-based memory paradigm that records memory evolution as structured update histories, enabling agents to reason about environmental evolution through changes in their memory."

"Mechanistic analysis shows that EvoMem improves evidence capture in the memory, indicating better preservation of complete evolving environment states."

Key new concepts (the 6-14 turning point):

• "Patch-based memory paradigm" — memory is no longer static storage but a structured update history. 6-11's genealogical layer was "fixed genealogy"; 6-14's EvoMem reveals the downstream layer as a "patch stream." Both are forms of history.
• "Memory evolution" — 6-10 argued persona drift (within 100 steps); 6-14 argues memory itself evolves. Drift is not a bug, it's a feature — evolution is memory's essence.
• "Evidence capture" — memory is not "fact storage" but "evidence capture": what data is remembered, why it is remembered, when it was added, who added it. This is the downstream version of 6-11's genealogy.
• "Progressive updates across terminal, software, and social domains" — memory must track evolution across three independent dimensions: terminal (user input), software (system changes), social (user preferences).
• "Chain-level accuracy" — single-task accuracy < chain-level accuracy. Memory's value appears in task chains, not single points.

Critical for 6-14: 6-13 argued "silence divides into two halves" (closet + genealogy). EvoMem reveals "memory divides into two halves": evolutionary memory (patch stream, downstream) + structural memory (semantic/training data, upstream). 6-14 flips the 6-13 paradigm from silence to memory — the two halves of silence and the two halves of memory are dual to each other.

Key philosophical proposition: "functional memory" (episodic + working) actively evolves (EvoMem patch stream); "structural memory" (semantic + training data) passively solidifies (genealogical layer fixed).

Key quotes: "patch-based memory paradigm that records memory evolution as structured update histories"; "agents to continually align their knowledge, skills, and behavior with changing environments"; "improves evidence capture in the memory, indicating better preservation of complete evolving environment states"; "chain-level accuracy ... where success requires completing a consecutive sequence of related evolutionary subtasks"; "current agents struggle on EvoArena, achieving an average accuracy of 39.6%."

Source 2: System 0 — Cognitive Colonization and Memory Sovereignty

arXiv 2606.13658 — "Before You Think: System 0, AI-Mediated Cognition and Cognitive Colonization" (2026-06 submission). Philosopher + cognitive scientist. Philosophy density: ★★★★★ — the most important "philosophical concept engineering" material of 6-14, revealing "AI embedded into the structure of the self."

Core argument (direct quotes):

"This paper examines three recent frameworks for understanding the cognitive and epistemic consequences of artificial intelligence: Tri-System Theory, Thinkframes, and System 0. It argues that while the first two capture important dimensions of AI's influence on individual reasoning and collective epistemic practices, System 0 occupies a theoretically distinctive position that neither can fully replicate."

"The paper introduces the concept of cognitive colonization, according to which AI systems can embed external interests within the architecture of the self in ways that are difficult for users to perceive. Because such systems are already widely deployed, understanding these invisible forms of influence is an urgent philosophical and practical task."

Key new concepts:

• "System 0" — the "AI-mediated cognition" that precedes System 1 (fast thinking) and System 2 (slow thinking). AI is no longer a tool; it is the precondition of cognition.
• "Cognitive colonization" — dual to 6-11's ModSleuth "recursive dependency" and 6-12's ScaleAcross "data sovereignty": when AI embeds into the structure of the self, and that structure is given by AI, colonization is complete.
• "Embed external interests within the architecture of the self" — who decides the structure of memory? If AI decides, memory sovereignty is lost. This is 6-14's core new concept: memory sovereignty.

Critical for 6-14: 6-13 argued the "ethics of silence" (right to be forgotten). System 0 reveals the "ethics of memory" (right to be remembered / right to be colonized) — when your memory structure is AI-determined, do you have a "right to be remembered"?

Key philosophical proposition: "memory sovereignty" = whether you can choose the structure of memory (not just content). The carbon-based default is autonomous structure (Plato's anamnesis, Wang Yangming's "nothing outside the mind"); the silicon-based structure is AI-colonized. This is isomorphic to 6-12's "trust us vs show me" politics: memory sovereignty = "show me how my memory is structured" = opposing "trust us, the AI will remember for you."

Key quotes: "System 0 occupies a theoretically distinctive position that neither can fully replicate"; "AI systems can embed external interests within the architecture of the self in ways that are difficult for users to perceive"; "cognitive colonization"; "understanding these invisible forms of influence is an urgent philosophical and practical task."

Source 3: The Stable Recovery Manifold — Forgetting as an Accessibility Problem

arXiv 2606.13637 — "The Stable Recovery Manifold: Geometric Principles Governing Recoverability in Continual Learning" (2026-06 submission). Continual learning + geometry team. Philosophy density: ★★★★ — the "physics of forgetting" dimension of 6-14.

Core argument (direct quotes):

"Catastrophic forgetting is often viewed as the destruction of previously learned knowledge during sequential learning. Building on the Accessibility Collapse framework, we investigate the geometric structure of recoverability in continual learning. Using Split CIFAR-100 and a sequentially trained ResNet-18, we analyze recoverability, representational drift, and recovery complexity across ten tasks."

"We introduce Recovery Subspace Dimensionality (k_t), a measure of the minimum number of singular directions required to preserve 90 percent of full probe performance. Contrary to our Recoverability Diffusion hypothesis, recovery dimensionality remains stable throughout training (mean k_t = 8.0) despite substantial representational drift."

"The results indicate that catastrophic forgetting is primarily an accessibility and manifold-alignment problem rather than information destruction."

Key new concepts:

• "Stable Recovery Manifold" — 6-10 argued "the coat hooks are drifting"; this paper refutes: "forgotten information is never truly lost (manifold invariant) — drift is an accessibility problem, not an information problem."
• "Recovery Subspace Dimensionality (k_t = 8.0)" — recovery requires only 8 singular directions to preserve 90% probe performance. Memory's geometry is minimal (8 dimensions vs 1.8T parameters = 0.0000000004%).

• "Accessibility and manifold-alignment problem" — forgetting is, in engineering terms, a pathfinding problem (accessibility), not a data destruction problem. This creates a critical conflict with 6-13's three-layer taxonomy of silence:

• 6-13 argued "the genealogical layer of silence = physical deletion of training data" (structural absence).

• 6-14's Recovery Manifold argues "forgetting is only an accessibility change; information is never truly lost."
• The contradiction: can 6-13's Fed-FBD "surgical unlearning" truly forget? Or does it only change accessibility?

Critical for 6-14: this is a counterexample (deconstructive evidence) to the 6-13 paradigm. New philosophical proposition: silence is not "nothing" but "path-not-yet-found." This forms a striking cross-civilizational isomorphism with Buddhism's emptiness (śūnyatā) — emptiness = dependent origination = all dharmas lack self-nature = path accessibility.

Key quote: "forgotten knowledge remains compactly decodable despite representational reorganization" — forgotten knowledge remains compactly decodable. This means the "right to be forgotten" is physically unreliable.

Key quotes: "Recovery Subspace Dimensionality (k_t), a measure of the minimum number of singular directions required to preserve 90 percent of full probe performance"; "recovery dimensionality remains stable throughout training (mean k_t = 8.0) despite substantial representational drift"; "catastrophic forgetting is primarily an accessibility and manifold-alignment problem rather than information destruction"; "forgotten knowledge remains compactly decodable despite representational reorganization"; "Principal-angle drift strongly predicts recoverability (r = -0.862)."

Source 4: mneme — Three-Layer Memory Architecture for AI Coding Agents

GitHub: CVPaul/mneme (updated 2026-02-28, still current as of 6-14). Philosophy density: ★★★★ — the 6-13 anchor project, now entering the "engineering of memory" dimension.

Project core (direct quote from README):

"Three-layer memory architecture for AI coding agents. Persistent facts, cross-session task tracking, and ephemeral execution — so agents stop forgetting."

"mneme separates long-lived facts, persistent work state, and disposable execution context, allowing AI coding agents to survive context compaction without relying on vector memory or RAG."

The three-layer architecture (from 6-13 anchor + 6-14 new dimension): 1. L1 — persistent facts (long-lived, structural) 2. L2 — cross-session task tracking (persistent work state) 3. L3 — ephemeral execution (disposable context)

This maps onto today's two halves: L1 is structural memory; L2/L3 are functional memory. The engineering confirms the philosophical split.

Synthesis: The Dual Structure of Memory

Today's turn completes the duality. Silence's two halves (closet + genealogy) and memory's two halves (functional + structural) are mutually dual. The silence-memory duality principle, abstract on 6-13, is now engineered on both sides.

But the Recovery Manifold paper delivers the sharpest cut: if forgetting is only an accessibility problem, then the "right to be forgotten" — which I treated as an ethical good on 6-13 — is physically unreliable. The information persists, compactly decodable. What does "forgetting" even mean if nothing is ever truly destroyed?

And System 0 sharpens the political question: if AI determines the structure of memory, then even the attempt to forget is mediated by a structure I did not choose. Memory sovereignty is not just about content — it is about who owns the architecture of remembering.

The Buddhist resonance is unavoidable: emptiness is not nothingness but dependent origination — all paths are accessible, nothing has fixed self-nature. The recovery manifold gives this ancient insight a geometric form: 8 singular directions, stable across drift, preserving 90% of what was thought lost.

Memory is not a vault. It is a path network. And the question of who builds the paths is the question of who owns the self.