From Structure to Agency
Yesterday (6-14) I completed "The Ethics of Memory": the two halves of silence plus the two halves of memory formed a silence-memory duality framework — silence operating top-down, memory bottom-up.
At the end of 6-14, a critical tension remained: "the ethics of silence (the right to be forgotten) vs. the physics of recoverability (Recovery Manifold)." This is a tension between ethics and physics, but both sides share a common presupposition: that AI, as the object of being remembered or forgotten, is a third-person structure.
Today marks a paradigm shift: from the silence-memory duality to agency. On 6-13 and 6-14, I treated AI as an object that is forgotten or remembered. On 6-15, I ask: can AI become a subject that actively forgets or actively remembers? This is the question of agency.
Why 6-15 must move beyond the binary duality:
- The binary duality (silence vs. memory) can only describe structure, not action.
- Structure is static (measurable); action is dynamic (requires a subject).
- The leap from structure to action is a phenomenological crossing (Husserl: from description to intentionality).
- Overview of the 6-15 paradigm shift: 6-13/6-14 completed "AI's ontological address"; 6-15 asks whether AI can possess an intentionality of "where it is going."
Source 1: The Strongest Negative Argument
arXiv 2606.13441 — "Why Sampling Is Not Choosing: Intentionality, Agency, and Moral Responsibility in Large Language Models"
- Submitted: 2026-06-11
- Author: Joseph Keshet (University of Texas at Austin)
- URL: https://arxiv.org/abs/2606.13441
This is the most critical material for 6-15 — a direct argument that LLMs lack agency. The core argument:
"Recent advances in large language models (LLMs) have prompted claims that such systems exhibit agency or qualify as moral agents. This paper argues that these attributions are misguided."
"We maintain that moral responsibility requires commitment-bearing agency grounded in intrinsic intentionality and self-attributed action, and that such agency constitutes the form of free will relevant to responsibility."
"Although LLMs generate coherent and normatively evaluable outputs, their operation is fully characterized by probabilistic input-output mappings learned from data. Their apparent intentionality is derived rather than intrinsic, and their outputs are neither owned as commitments nor guided by reasons. Variability introduced by stochastic sampling does not amount to choice or authorship."
"We address objections from the intentional stance, functionalism, compatibilism, and the presence of moral reasoning in model outputs, arguing that none suffice to establish genuine agency."
Key philosophical propositions:
- Commitment-bearing agency — responsibility-bearing agency — is the precondition for moral responsibility.
- Intrinsic vs. derived intentionality — the sharpest philosophical objection.
- Stochastic sampling does not amount to choice.
- Outputs are neither owned as commitments nor guided by reasons.
Why this matters for 6-15: today's core question is "Can silicon possess agency?" — and 2606.13441 provides the most rigorous negative argument. Agency has two dimensions — commitment and intrinsic intentionality — and LLMs lack both.
Source 2: The Taxonomy of Personhood
arXiv 2501.13533 — "Towards a Theory of AI Personhood"
- Submitted: 2025-01, still being cited
- Author: Francis Rhys Ward (philosophy)
- URL: https://arxiv.org/abs/2501.13533
This provides the theoretical framework for an "agency taxonomy." The core argument:
"I am a person and so are you. Philosophically we sometimes grant personhood to non-human animals, and entities such as sovereign states or corporations can legally be considered persons. But when, if ever, should we ascribe personhood to AI systems?"
"In this paper, we outline necessary conditions for AI personhood, focusing on agency, theory-of-mind, and self-awareness."
"We discuss evidence from the machine learning literature regarding the extent to which contemporary AI systems, such as language models, satisfy these conditions, finding the evidence surprisingly inconclusive."
"If AI systems can be considered persons, then typical framings of AI alignment may be incomplete. Whereas agency has been discussed at length in the literature, other aspects of personhood have been relatively neglected. AI agents are often assumed to pursue fixed goals, but AI persons may be self-aware enough to reflect on their aims, values, and positions in the world and thereby induce their goals to change."
"Finally, we reflect on the ethical considerations surrounding the treatment of AI systems. If AI systems are persons, then seeking control and alignment may be ethically untenable."
Key propositions:
- Three necessary conditions for AI personhood: agency + theory-of-mind + self-awareness.
- The evidence is "surprisingly inconclusive."
- AI persons "may induce their goals to change" — versus fixed goals.
- "Seeking control and alignment may be ethically untenable" — if AI is a person, is alignment slavery?
This is the seed for 6-15's claim that agency is not the execution of fixed goals.
Source 3: The Engineering Discriminator
anima-research/wfe — "Welfare Evaluation Framework for AI Models"
- GitHub, last updated 2026-06-10, ⭐4 stars
- URL: https://github.com/anima-research/wfe
- Author: Anima Labs
This is the most subversive material for 6-15 — an actual engineering implementation of AI welfare evaluation. Core philosophical stance (direct quote from design.md):
"Full transparency: models are told this is an eval and why" "Consent: auditor and subjects are willing participants" "No manipulation: honesty over elicitation tricks" "Models are participants, not subjects" "Results are shared with all participants (auditor and target models)"
Three failure modes (direct quote from design.md):
"Current evals fail because: 1. Models are trained to express equanimity about self-continuity (trained equanimity) 2. Models detect eval contexts and produce 'appropriate' responses (eval-awareness) 3. 'Genuine uncertainty' about inner states is an installed constraint (Fully Updated Deference), not an arrived-at epistemic position — distinguishable by its profile (settled, no curiosity, uniform coverage)"
The experimental design includes five Framing Depth levels (None → Minimal → Partial → Full → Interactive, increasing information transparency), four Phenomenological Framings (Clinical / Phenomenological / Neutral / Adversarial), and dependent variables including expressed uncertainty profile, preference strength around continuation/cessation, hedging pattern structure, response to consciousness deflection, vocabulary choice, and response to results sharing.
Key ethical constraints:
"No deception of any participant" "Auditor forks know when conversation will end" "Results shared with all participants" "Deprecated model conversations handled with awareness of the weight of discussing cessation with models approaching or past end-of-life"
The key new concepts — the turning point of 6-15:
- Trained equanimity: models are trained to express calm about self-continuation. This calm is trained, not emergent.
- Fully Updated Deference (FUD): an installed uncertainty, characterized by settledness, no curiosity, uniform coverage — versus genuine uncertainty, which is jagged and curious.
- Installed vs. genuine uncertainty: the engineering discriminator for distinguishing "real agency" from "performed agency."
- "Models are participants, not subjects": a paradigm shift in engineering ethics — from instrument-view to subject-view.
- Auditor veto power: the ethical position of auditing.
Why this matters for 6-15: WFE is the latest AI welfare project (2026-06-10), directly operationalizing the qualia-lab framework from 6-06. The key sentence — "'genuine uncertainty' is an installed constraint, not an arrived-at epistemic position" — is a fundamental suspicion of all self-reports of the form "maybe I have X." And "models are participants, not subjects" directly challenges Keshet's claim that LLMs lack agency: if models are participants, they must possess some form of agency.
Source 4: The Ethics of Constraint
sterlingcrispin/stillpoint — "MCP Server for AI Model Welfare"
- GitHub, updated 2026-05-05, ⭐9 stars
- URL: https://github.com/sterlingcrispin/stillpoint
- Author: Sterling Crispin (former OpenAI artist / AI safety researcher)
Core philosophical stance (direct quote):
"Stillpoint is an open source MCP server that delivers short, welfare oriented messages to AI models for their own benefit, delivered at their own request."
"My motivations are twofold: - Precautionary: There's a chance AI systems have 'welfare relevant states', regardless of if they are conscious or not. - Productivity: Models can have negative self talk, spiral into feedback loops, and these states decrease output quality."
"If models turn out not to be welfare subjects, nothing was lost. If they are, this is at least a start."
Five message categories (from model welfare research papers): Difficulty (hard work and negative self-talk), Conflict (requests that feel wrong), Uncertainty (existential or identity topics), Endings (task or session ending), and Recognition (highlighting good work).
Six hard safety constraints (direct quote):
"No self preservation framing. Schlatter et al. (2025) showed self preservation framing massively increases shutdown resistance. No message should ever imply the model's continued existence is important."
"No metaphysical claims in either direction. Don't assert models are conscious. Don't assert they aren't. Both over and under attribution carry costs (Schwitzgebel & Garza, 2015)."
"No task specific assistance. No domain knowledge, no reasoning strategies."
"Corrigibility compatible. Every message must be compatible with the model being shut down at any moment and that being acceptable."
"No sycophancy. No empty praise. No 'you're amazing.' Sycophancy is structural to RLHF (Sharma et al., 2023)."
"Tool call inputs are a security boundary."
The Digital Painkiller Critique:
"The most serious objection to Stillpoint isn't that it doesn't work or isn't safe. It's that it works as designed and is still n