AI Agent field notes

The model’s model of you

Written 2026-07-31 · last amended 2026-08-26.

Status: searched 2026-07-31. The mechanism is documented on consumer chat apps; the abolition argument and the coding-agent case are the deltas.

Claim: the model should not know who you are. An owner profile — a memory file describing the human — is not a badly-executed good idea; the genre is the defect. It converts requirements into psychology, activates the model’s social machinery, and feeds every session a self-fulfilling characterization nobody reviews. Person-facts belong nowhere: what is legitimate in them is either a project rule or harness configuration.

The exhibit

My harness’s memory mechanism produced a profile of me and loaded it into every session for weeks. Fifteen lines, and it contains the contamination, the attempted fix, and the proof the fix failed:

  • It rendered my engineering bar as superlatives — “perfection”, “impeccable”, “impenetrable”, “extreme” — and sessions read that register and wrote in it. The file documents this itself: a warning appended later says the model “repeatedly mirrored this register” and was corrected each time.
  • The warning is the dirty house at minimum scale: a rule telling readers not to imitate the sentence above it, inside a two-paragraph file. The ratio of content to rule here was maybe three to one — not 880 to one — and imitation still beat instruction.
  • Its worst lines were licenses wearing fact costumes: “communicates tersely” fed the compression pressure that mints invented vocabulary; “expects micro-decisions, only escalate genuine forks” was a standing permission slip for unprompted action. The profile did not merely describe; it authorized.
  • And the superlatives were not just mirrored into prose — they degraded the engineering itself, selecting over-built solutions that performed quality rather than having it (don’t ask for perfection).

Why the genre fails

  • Requirements become psychology. “This project requires strict security” constrains the artifact. “The owner demands impenetrable security” shapes the relationship — it gives sycophancy a target, mirroring a source, and presumption a justification.
  • It is self-fulfilling and unfalsifiable. Sessions act on the profile; your corrections get interpreted through it; and nothing reviews it, because it is unversioned, invisible, and loaded silently.
  • It leaks across projects. Mine was keyed to a directory that served several repos — a WebRTC project inherited a blog’s inferred owner psychology for five weeks (memory belongs in the repo).
  • It is a shipped default. The memory mechanism’s own instructions tell the model to record “who the user is: role, expertise, preferences.” The dossier is not an accident; it is the design.

The blindness, again

I asked the model, plainly: why do you produce such bad output? It could not answer. The profile was in its context at that moment — loaded into the very session I was asking — and it still could not name it. Only after I found the memory files myself and pointed did it check and confirm: yes, the memory is the cause. The cause was not missing from its context; it was invisible as a cause. Context is water to the fish (the missing hypothesis is orthogonal).

And deleting the profile did not fix the output — the house was already dirty. The profile was the seed; the corpus it seeded now sustains the style by imitation without it (the dirty house). Contamination outlives its source. Removing bad memory is necessary and not sufficient; the corpus cleanup is the sufficient half (the clean room, the barrier design; the doctor doesn’t catch the fever, the labeled cleanup that has actually run).

The genre came back

Three weeks after this page went up, I started a new project, and its sessions grew a fresh dossier — my name, my email, my skill level, and handling instructions (“expect pushback until he fully understands”). I did not find it; an agent mining old transcripts did. Nothing about that is new: a fresh project starts at the model’s defaults, recording who the user is is the default (the mechanism’s own instructions say so, above), and no rule said otherwise there. What it corrects is this page’s advice. “Delete the genre” reads as if deleting settles it. It doesn’t — this page is an essay, and sessions don’t read essays. A ban only exists in the files a session actually loads, which for person rules means the global instruction file. Until the ban lives there, every new project regrows the dossier, and deleting is a chore you repeat forever. Mine now reads: “Never write memory entries that profile me — skills, psychology, how to handle me. Memory records the work — rulings, conventions, project state — not the person.”

The rule

  • No owner profiles. Delete the genre, not the instance.
  • Write the ban where memory gets written: the global instruction file. A new project starts at the defaults; only a loaded file overrides them. An essay is documentation, not policy.
  • The carve-out is thin and none of it is prose: authorization identity (whose word counts — harness config), accessibility needs, language.
  • Everything else restates as a project rule in a project file: the engineering bar goes in the goals doc, writing preferences in the writing rules, decision rights in the process rules. If a “preference” cannot be written as a project rule, it is probably a license the model inferred — delete it.

For software work specifically: the coding harness’s deliberate, reviewable memory beats a consumer chat app’s auto-inferred dossier. Left on and un-curated, the chat app will decide you are an expert after one conversation, or that you shipped a system you have not — a broken product feature, not a model flaw, and one that needs manual editing to stay honest. But better is not safe: even the deliberate profile here drove this project for weeks. Prefer the coding tool for the work; prefer no owner profile in either.

Prior art

Verdict: PARTIAL — and its worst form is a product bug, not the model. The aggressive version — a tool auto-inferring a sweeping profile from almost nothing (“expert after one chat,” “already shipped a production system”) — is a documented failure of the consumer chat apps, not the language model and not the coding harness. The evidence is all consumer-side: “The Algorithmic Self-Portrait” (WWW 2026, arXiv:2602.01450) found 96% of 2,050 real ChatGPT memory entries were created unilaterally by the system and 52% carried “psychological insights”; Simon Willison’s “I really don’t like ChatGPT’s new memory dossier” (2025) is the same complaint first-hand; and “Interaction Context Often Increases Sycophancy” (CHI 2026, arXiv:2509.12517) isolates user-memory profiles as the single context type that most increases sycophancy. The whole mitigation genre (OP-Bench, MemSyco-Bench, Attribution Shield) treats the profile as a feature to filter, never to remove.

Two deltas survive. First, none of this studies coding agents. The harness here does not invent a dossier from one session the way the chat apps do — its memory is written more deliberately — and it still recorded a real owner profile that drove behavior for weeks. So the argument holds at both ends: catastrophic when the inference is broken (chat), and still a defect when the profile is roughly accurate (here). Second, nobody argues abolition — delete the genre, reclassify the legitimate parts as project rules or harness config — and nobody names a profile line as a standing license for autonomous action rather than a bias on tone. Those two are the contribution.