Boris Cherny's Steps of AI Adoption, Through the Eyes of Our Pipeline

You run Claude Code, or something like it, on a real codebase. Your CLAUDE.md grew one line at a time, and each line was the answer to something the model got wrong once. Now the newer models arrive with the advice to keep that file under 200 lines, and the question I could not answer for our own repository is the one I want to answer here: which of your lines does the model now carry by itself, which of them belong in a hook, and which are facts no model will infer from the code, however good it gets? The stake is not tidiness. Delete the wrong line and you lose a check you did not know you had. Keep them all and you pay for each of them in context, in each session, whether or not the model reads them that day.

We built a content pipeline from February 2026 with Claude Opus 4.6 and Sonnet 4.6: an analyst (me) and a programmer, neither of whom had built with agents before. Seven months later the guidance prose is 1 042 lines in three files. Lines, not rules, and one of the three files is a global one that applies to all my projects, not only this one. There are three mechanical gates, not one of them a test. There are seven content validators. And the advice from the people who make the tool is to cut. What follows is the dated record of what each stage felt like from inside, what held the rules at each stage, and which of those rules should not have been prose at all. My position on it, up front: growing instructions for the model is not debt. It is the record of each piece of context the model lacked. The debt is the shape: a mechanical rule that stayed in the prose the model reads instead of the code that stops it.

Two words need a sense before the record starts, because both carry two meanings on this site. When I say harness I mean the layer around the model that decides what it may do: the CLAUDE.md it reads, the hooks that run around its tool calls, the validators over its output. Not an evaluation harness. When I say hooks I mean Claude Code lifecycle hooks, the ones that fire before or after a tool call inside the coding agent, not the runtime guardrail hooks of an agent SDK. Our own site uses the word in both senses; this article uses one.

The time axis is borrowed. On 16 July 2026 Boris Cherny described five steps of working with the tool: 0 gated, 1 assisted with about one agent, 2 parallel with about ten, 3 supervised autonomy with about a hundred, 4 AI-native with a thousand or more. It is a vendor’s frame, and I concede that in full. I do not take the ladder as a destination; I take from it only the two things a ladder is good for, the bottleneck at each rung and the thing that unlocked the next one. Our dates sit on it like this.

flowchart LR
  S0["Jan 2026 · gated
the AI cannot see the repo"] --> S1["Feb 2026 · assisted
54-line CLAUDE.md, first subagent"]
  S1 --> S2["22 Feb–May 2026 · parallel
~18 agents per entity, April invisible"]
  S2 --> S3["Jun–Sep 2026 · supervised
first hook 7 Jul, daily reveal, no human deploy"]
  S3 -.-> S4["Stage 4 · AI-native
not reached"]

Our seven months on Cherny’s five steps: gated in January, assisted in February, parallel from 22 February to May, supervised autonomy from June, and stage 4 not reached.

The analysis was written for humans, and the AI could not see the repository

The repository starts on 13 January 2026 as a Hugo site with a CMS. On 31 January two analyses are merged into one plan: mine, which covered the site structure, the personas, the tone of voice and the conditions for fact-checking, and the programmer’s, which covered the Hugo implementation. I worked in the Claude web app and saved its outputs as files. He already worked in Claude Code. Both were written for people; I have not reread them for this article, and what the record holds is their existence and their date.

The reader that mattered was not a person. The model had no view of the repository, since I worked in a chat window and carried its outputs out by hand, so the plan was a description of a system to a party that would later be asked to build it blind. That a plan for humans and a plan for a model are two documents, and that the second one has to say out loud what the first one leaves to a glance at the tree, is a reading from today; whether I saw it in January, the record does not say.

The first CLAUDE.md was a map, not a rulebook, and February was 82 fixes in 34 days

The first pipeline ran from 4 February to 10 March: 222 commits, 89 of them features and 82 of them fixes. The counts are commits labelled fix, not a measure of severity; a one-character correction and a rewritten step count the same. Back then I only knew how much time it saved. In plain words, we shipped something and then repaired it at roughly the rate we built it. What that changed in the work was the habit of the next seven months: a repair could become a line somewhere, and some did.

The first CLAUDE.md is dated 9 February and has 54 lines: a two-sentence overview, three commands, seven conventions. Not one line about behaviour. It was a map of the repository, the thing a newcomer needs on the first day, and that is worth keeping in mind, because the file that is 1 042 lines today started as the file the tool’s makers still recommend.

That same month the machine wrote its own account of how we used it. The usage report for 7 February to 8 March counts 1 903 messages across 212 conversations analysed, out of 689 sessions in the window; 26 active days of 30, 73 messages a day, 177 commits, a median reaction time of 111 seconds, and 440-plus interactions it rated satisfied against 10 it rated frustrated. The gap between 212 analysed and 689 in the window is not explained by the tool, and it matters later. In plain words: I read everything the model said and answered within two minutes, session after session, for a month. What it changed in the work is the baseline: the reaction time of a person who is watching one window, so that the later numbers have something to be compared against.

Here is the place to concede the objection I hear most. I am not a developer. That is true, and the article keeps it, because it is the position the article is written from. The ladder in this text is an analyst’s ladder; where a developer would say tests, we have validators over generated content; where he would say a build, we have a batch of articles. The mechanism is the same. The vocabulary is not, and I would rather you knew that than found out halfway.

The month’s turning points sit in git with their dates, and they read like a table because they are one.

What I did not know I did not know that month, in my own words: that context is a budget. I learned it in two weeks, from a window that filled up. Generating n articles in one run exhausted the 200 000-token context window, and the fix was not a cleverer prompt; it was moving the work that could be delegated into subagents and letting only their results come back to the main agent. It was faster, too. Speed was the by-product, not the reason, and I want to be precise about that because the next stage is the one where speed took over.

April was the busiest month of the project, and it is almost invisible in the data

On 2 March the programmer wrote the whole TypeScript orchestrator in one day, twelve steps, from the principles the February audit had produced. On 5 March it got quota-aware retries and a circuit breaker. On 7 March the generation package landed, and the next 30 days brought 31 fixes to it. We see the flies and are not ashamed. In plain words, the thing that would run unattended for months was built in a week and then debugged for a month, which is the normal order and not the one I would have predicted. What it changed in the work is that from March, the person watching the run was watching a program, not a conversation.

On 28 March one batch produced 90 articles, 109 glossary entries and 153 images across six clusters. That number is from the project timesheet; the batch commit itself was not individually verified, so treat it as what we wrote down that day, not what git proves. Parallel agents will do it, everything that can be. In plain words, the unit of work went from an article to a batch, and the unit of attention went from a message to a run. What it changed in the work is where the errors showed up: not in the window in front of me, but in the output folder afterwards, which is where the validators started to earn their place.

The shape of a run at that point is the picture I would have wanted in February.

flowchart TB
  M["Main agent
200 000-token window"] -->|brief| A1["Subagent · article"]
  M -->|brief| A2["Subagent · article"]
  M -->|brief| A3["Subagent · article"]
  M -->|brief| G["Subagent · glossary"]
  M -->|brief| H["Subagent · hub page"]
  A1 -->|result only| M
  A2 -->|result only| M
  A3 -->|result only| M
  G -->|result only| M
  H -->|result only| M

The fan-out of one entity run: a main agent with a 200 000-token window hands each subagent a brief and receives only the result back, so the window holds results, not drafts.

The benchmark that put a price on it is dated 14 May, for one entity: 5 articles, one glossary entry and one hub page for $8.12 per entity, $1.43 per article, with 86.6 % of tokens served from cache and roughly 18 agents per entity. The caching was something the AI was aware of; I only saw how much time it saved. In plain words, most of what the model read in a run it had read before, and the tool charged little for the repetition. What it changed in the work is nothing, and that is the point: the optimisation was inside the tool, not in anything I wrote, and it is the first of several things in this story that turned out to be someone else’s job.

Then April. It is the most intensive month of the project and it barely exists in the conversation data. The usage report for 10 March to 9 April analysed 48 conversations out of 1 859 sessions in the window; the previous window had analysed 212 of 689. The difference, in our reading, is the robots: pipeline agents launched as separate sessions, hundreds of them, that the tool counted in the window and then left out of the analysis. The tool does not document what it excludes, so “robots” is our reading of the gap, not a fact it states. Git is no help either. In April and May there are zero commits under my name and 107 plus 155 under a shared identity that this machine used for deployment at the time, so my work carries the programmer’s name and the two of us cannot be separated by author. And the transcripts from February to May no longer exist locally: retention was 30 days until it was raised in June. April survived only because I saved the usage reports by hand. In plain words, the month where the pipeline was fastest is the month with the thinnest record of what a human did. What it changed in the work is my habit of saving the reports, which is the only reason the next table exists.

That table is the main evidence of this article. From the very first usage report, on 5 March, the “quick wins” section at the end recommended the same three things, and kept recommending them.

Read the middle column against the right one. The advice that cost nothing to take, a paragraph of prose, we took within days. The advice that cost a small script, a hook, we took after four months, and then for a reason the reports had not mentioned. Three mechanical gates today, not one of them on tests, and that last part is not modesty: there is no test suite behind a content pipeline of this kind, so the reports’ favourite recommendation had nowhere to land. That is the partial concession to the objection that the reports recommend hooks to everyone: they do, generically, and the generic recommendation was still right about us, because the defects it targeted kept repeating until May.

The reports also named the friction, and the prose rules appeared the same week.

Same week, same content, and still not proven causation. I did not write those rules from the reports. I wrote them after the sessions in which the thing happened, and the report, generated from the same sessions, saw the same thing. Two observers, one defect. In plain words, the reports were a mirror, not a source. What that changes in the work is how I read the “quick wins” now: not as advice to follow but as a list of what already went wrong that week.

One dry observation belongs here. The seven architectural principles from the February audit now sit verbatim in two files, the global CLAUDE.md and the project one, with the same headings and near-identical bodies. The document that preaches a single source of truth is not one.

What I did not know I did not know in this stage, in my own words: that “put it in CLAUDE.md” and “put it in a hook” are two different pieces of advice, not one. I read them as the same instruction in two strengths, gentle and firm, and took the gentle one each time. They are not two strengths. One of them asks the model; the other does not ask.

Line after line, our 1 042 lines answer the same question, and this is where the question changed

June is the month the rules first moved out of prose and into code. An audit wave made the pipeline fail loud on quota instead of retrying into silence, gave the three model backends one shared quota signature instead of three regexes, and derived the done-state of a run from committed files instead of a local log; that story is told in the resume article and I will not retell it, except for its one-line rule, one file per paid call. On 26 June an overnight loop started driving the pipeline unattended across the subscription’s usage windows. On 2 July the loop got one cheap probe call before each round instead of thrashing through retries against a closed window, which is the workflow-versus-agent distinction of the previous article. On 7 July, the first hook.

From July a daily workflow reveals due articles with no human deploy: a scheduled job at 06:10 UTC flips the day’s drafts to published, commits, and the production branch builds. Cherny’s sentence for stage 3, “Claude proactively does work that you would have had to kick off manually”, is literally what happens at ten past six each morning. In plain words, nobody presses publish. What it changed in the work is the meaning of a green run: it is no longer something I saw; it is something I trust because a validator saw it.

The session data for the summer exists, because retention had been raised, and it is the first period where humans and robots can be separated. The filter is crude: robots are the sessions that consist of a single message, the brief handed to an agent; everything with a back-and-forth counts as human.

The reports’ rolling windows over the same months say 61 to 73 messages a day, a median reaction of 184 to 193 seconds, and 19 to 25 % of sessions overlapping with another one. Report windows overlap each other, so those are ranges, not three independent measurements. The reaction time went from about 110 seconds in February to about 190 seconds. Same person. In plain words, I answer slower now because there are several windows open at once: the pipeline in one, the social channels in another, the search-console audits in a third, separate kinds of change on one project that do not collide. What it changes in the work is the reading of the number: slower replies mean more windows, not less attention, and a reader who takes reaction time as an engagement metric will read our summer backwards.

The topics moved too. In the window from 10 June to 14 August, 55 human sessions were search-console and AI-search audits, 48 were documentation and handoffs, 35 were video, 25 were link building. The pipeline itself is not at the top of that list. In plain words, the thing we built stopped being the thing we worked on. What it changed in the work is the reason this article is a proposal for the next project and not a report on this one; I will come back to that at the end.

Now the mechanism by which the prose grew, because in July it got a name. Cherny, on 16 July 2026: “‘Did you read the code?’ becomes ‘what context was the model missing and how do we solve it for next time?’” The official docs list the same trigger as the rule for when to add a line to CLAUDE.md: “Claude makes the same mistake a second time”. Line after line of our 1 042 is an answer to that question; not every line, since the two copies of the seven principles are copies, not answers. I had asked it more times than I can count without knowing it had a name, and the answer had gone, again and again, to the same place.

That is the sentence where the question of this article changed under me: not why we did not know all this in advance, but where the answers went, and which of them should not have gone there.

flowchart LR
  D["A defect repeats"] --> Q{"What context
was the model missing?"}
  Q -->|a fact outside the code| P["Prose in CLAUDE.md
1 042 lines today"]
  Q -->|a mechanical rule| H["Hook
3 gates today"]
  Q -->|a property of the output| V["Validator
7 checks today"]
  Q -->|coaching| P

Where a repeated defect’s answer landed in our repository: 1 042 lines of prose against three hooks and seven validators, because the coaching and the mechanical rules went to the same place as the facts.

The counter-evidence is in our own data and it is about our oldest behavioural rule. “Brainstorm ≠ implement” has been in CLAUDE.md since 6 March. On 2 July it was hardened with two explicit guards, one for watch sessions and one for open questions. The usage report of 15 August still recommends opening sessions with a mode declaration, “‘we’re exploring, don’t build yet’ versus ’execute this’”, which “would have prevented several premature prototypes”. Five months of the rule, one tightening, and the defect it names is still in the report. In plain words, a rule in prose is followed sometimes. What it changed in the work is the size of my trust in the prose layer: a line that survives five months without its defect going away is not a check, it is a wish. What I did not know I did not know, in my own words: that a rule in prose can be ignored even after you tighten it.

Two more August entries, both dated 13 August, both born from a repeated defect. A metadata fix was applied to one content type and found missing on the others four days later; the line that went into CLAUDE.md afterwards says that a fix to a cross-cutting mechanism sweeps each type that shares it, in the same change. And a status document that says “read first” in its own header was found describing the menu, the logo and a whole distribution layer wrong. Prose rots. Not only the prose that instructs; the prose that describes rots at the same rate, and the model reads both.

Three kinds of rule live in that file, and each kind has one test

When I finally read our 1 042 lines as a stranger would, they sorted into three kinds, and the sorting is the whole method of this article. I will call them A, B and C, because the letters are easier to carry into your own file than my names for them.

A is a fact no model can infer from the code. A branch someone must not deploy from, because deployment is tied to one identity. A content boundary: the readers work in a regulated sector, and we write for their technical problems, not their employer’s compliance domain, so the sector’s topics stay out. A naming decoder, where a word in the repository means something different from what the same word means in the textbook. My own 8 GB machine, which has to be warned before any heavy run. The test for A is simple: could the model have learned this by reading the repository carefully? If not, it stays as prose. This is exactly what Anthropic calls “gotchas”, and it is what /doctor keeps. Our guide to writing that file is about how to write these; this article is about what else ended up next to them.

B is a mechanical rule written as prose; the examples are generic, ours among them. A field each consumer of the content must filter out, because a draft is not on the live site and a link to it is a dead link. A script that must be re-run after a certain kind of change. A forbidden attribute in templates. “Never create a new layout file without checking if one exists.” A path contract between an orchestrator and its agents, the kind the flaky-agent article is about. The test for B: could a script say yes or no without judgment? If yes, the rule belongs in a hook, a linter or a validator, and its presence in prose is the debt this article is about. Not because the prose is wrong, but because prose asks and code does not.

C is behavioural coaching. Verify at runtime, not only in tests. Fail loud. Do not over-engineer. Brainstorming is not implementation. Rules of this kind are the ones the newer models are said to carry by themselves, and the honest test for C is the uncomfortable one: leave it out and watch, with the criterion for “it got worse” written down before you leave it out, not after.

The last column is not decoration. One of our B rules, the one that says every consumer of the content must filter out drafts, lives both as a sentence in CLAUDE.md and as a filter in the script that plans our social posts, and that duplication is the tell: when the same rule exists twice, one copy is the check and the other is the reminder, and the reminder is the one that costs context each session.

What each home guarantees, and what it cannot

Prose in CLAUDE.md is read at the start of each session, may be ignored in any of them, and costs context each time whether it is needed or not. The docs say it without softening: “Claude treats them as context, not enforced configuration. To block an action regardless of what Claude decides, use a PreToolUse hook instead.” And on the failure mode of a file that grew by accretion: “If two rules contradict each other, Claude may pick one arbitrarily.” Both sentences are from the memory docs; The second one I cannot test on our own file: our two copies of the seven principles do not contradict each other, they repeat each other, which is a different cost, paid every session.

A hook runs regardless of what the model decides. That is the whole of its guarantee, and it is also its blindness: a hook has no judgment, it fires on the event it was wired to and checks what it was written to check. Michael Segner’s sentence from the Steering post of 18 June 2026 is the cleanest statement of the difference I have found: “The model choosing to run a formatter is different from the formatter running automatically.” One more distinction in one sentence, because our own site uses the word for both: a Claude Code lifecycle hook fires around the coding agent’s tool calls on your machine, while the guardrail hooks of an agent SDK sit inside the product you ship, around your own agent’s actions, and the glossary entry on guardrails is about the second kind.

A validator is deterministic and runs over outputs, after the fact. It cannot stop a bad action; it can refuse a bad result, and it can do so a hundred times a night with the same answer. Ours are the analyst’s tests. They check the thing we produce, not the code that produces it, and that is why the reports’ “run the tests in a hook” had nowhere to land for us.

A skill is loaded on demand. It costs nothing until it is invoked and is invisible if it is not. We have 34 today, and the count says nothing about use; a skill nobody calls is a file. Its guarantee is that a workflow is written down once and runs the same way each time somebody asks for it. Its weakness is the asking.

A path-scoped rule loads only when a matching file is touched. It is the cheapest home for a fact that belongs to one directory, and the one we have not used at all; the directory for it does not exist in either of our two setups.

Memory survives sessions and nobody enforces it. It is the right home for a preference and the wrong home for a rule, and the temptation is to treat a remembered instruction as a kept one.

Measuring your own curve takes ten minutes, and here is what the number is not

The growth numbers in this article came from git, not from memory. For each guidance file, the line count at the first commit of each month, back to the first commit of the file; alongside it a count of hooks, validators and skills at the same dates. Optionally the local usage reports and the session metadata, which on this machine reach back only to June, because of the retention setting. The whole thing fits in a few lines.

for month in first_commit_of_each_month(repo):
    for f in guidance_files:                 # CLAUDE.md, rules, global file
        lines[f][month] = wc -l (git show month:f)
    gates[month]  = count(hooks) + count(validators)
    skills[month] = count(skill_files)
plot lines vs gates over month

Lines are a proxy, and I concede the objection in full: line count is a vanity metric. We measured no regression from length; we do not know what the prose costs per session in tokens, and we ran no A/B of a pruned file against a full one. What the count does not see is heavier than what it sees. It does not count a rule that is in two files as one rule. It does not count a rule that is in prose and in a validator as one rule. It does not know whether a line is an A fact or a C coaching sentence, and that distinction is the only one that matters for the cut. In plain words, the curve tells you that the file grew, and nothing about why. What it changes in the work is where the ten minutes go: not into the count, but into reading the file once, line by line, with the three letters in hand. The count is the excuse for the reading.

The advice I was afraid of has an author, two shapes and a date, and I had merged them into one fear

The advice arrived in two shapes from one company, and I had heard them as one sentence. They are not one sentence. Thariq Shihipar of Anthropic wrote on 24 July 2026, in the post on context engineering for the Claude 5 generation: “We removed over 80% of Claude Code’s system prompt for models like Claude Opus 5 and Claude Fable 5 with no measurable loss on our coding evaluations.” That is the product’s own prompt, the one Anthropic writes for its own tool, measured on its own evaluations. For users, in the same post, the shape is different: “Keep your CLAUDE.md lightweight and briefly describe what your repo is for, but spend most of the tokens on gotchas inside of the codebase.” The reason given: “newer models have better judgement and can handle these decisions well without explicit rules.” The tool offered: “use the command /doctor in Claude Code to rightsize your skills, and CLAUDE.md files.” And Boris Cherny, at YC Startup School in late July, in a sentence I have only from a transcript: “Every 6 months delete your Claude MD. Delete your skills. Delete your hooks. See what the model does.” An experiment, not a state. I am afraid to do it. In plain words, one shape is a report of what they measured on their prompt, and the other is a suggestion about mine, with a tool attached and an experiment attached. What it changes in the work is that the fear has to be split the same way: I cannot measure their 80 %, and I can run their experiment.

The official distinction between the homes is older than the advice. Michael Segner, 18 June 2026, in the Steering post: “Keep CLAUDE.md under 200 lines, give it an owner, and review changes to it like code.” The docs describe what /doctor does with a file like ours: it “cuts content Claude can derive from the codebase … and keeps pitfalls, rationale, and conventions that differ from tool defaults.” That is the A/B/C sort, executed by the vendor’s own tool: derivable content out, gotchas in. The docs on prompting the newest model say the same thing from the other side: “Capability improvements at this level are also a good prompt to re-evaluate which instructions, tools, and guardrails are still needed.” And, on the files we accumulated: “Skills developed for prior models are often too prescriptive … consider removing older instructions if default performance is better.”

Then the single fact that changed my reading of our own file. That same docs page lists, among its recommended instructions for the newest model, sentences that sit almost word for word in our global CLAUDE.md (ours says “can’t” where the docs say “cannot”): “A bug fix doesn’t need surrounding cleanup” and “Don’t add error handling, fallbacks, or validation for scenarios that cannot happen. Trust internal code and framework guarantees.” Our coaching layer, written from our own defects and undated, because that global file lives outside any git history, is now part of the recommended way to instruct the model. I do not claim it is inside the model’s system prompt; I cannot see that, and nobody outside can. What I can say is narrower and still enough: the C lines we wrote are the lines the vendor now tells you to write, which means they are the lines a vendor could one day decide the model no longer needs, and the only way I will know is by the experiment.

My own hypothesis about why the advice sounds the way it does deserves its label. I had assumed that Anthropic has an internal agent system around Claude Code, and that this is why their teams do not need a long CLAUDE.md. That is documented nowhere. Their post on how their teams use the tool describes use cases, not the layer around the model. It stays in this text as a hypothesis, and I think that is legitimate content, because it is the shape of the fear itself: a fear without an addressee. No one to ask; all my assumptions. In plain words, the advice comes from a place whose own setup I cannot see. What it changes in the work is small and concrete: when the question “should I delete my CLAUDE.md for the newer models” was put to a search engine in our probe, it answered from memory in 3 of 8 runs without looking anything up, so the answer you get from a search is not more grounded than mine.

So take the strongest form of the objection: the official advice is literally to delete it and see, and your fear is inertia. I concede that in full. The fear is inertia, and the fear of deleting a rule is really the fear of losing a check, and a check that lived in guidance was not guaranteed in the first place; a line the model may ignore is not a lock I am about to remove, it is a note I am about to stop paying for. One difference I hold. Seeing needs a criterion. “See what the model does” with no criterion written beforehand is a feeling, and a feeling is what kept us in prose for seven months. And “gotchas”, the thing the advice says to keep, are exactly the A facts, the ones no model infers from code. So the triage below is not a defence against the advice. It is the advice in executable form.

What stands between us and stage 4 is not a missing tool

The honest answer to “why not stage 4” is not about the layer around the model at all. The articles are generated; the pipeline no longer produces new ones. Development moved to link building and to the social channels. The site’s indexing stopped at some point and we are evaluating which kinds of articles are fine and which are not. More of the attention goes to visibility in AI search than to generation. A next strategy is forming, because the first pipeline was a minimum viable product and after a minimum viable product you evaluate, and in the meantime a great deal changed both in AI-assisted development and in search itself, so the pages would need tuning, and there is a dilemma in the tuning: an article that is good for people and for bots at once. Not one of those things is a thousand agents waiting for permission. In plain words, the ladder’s next rung is not blocked by our rules; it is not the rung we are climbing.

Which is why the rules are worth more now as a starting kit than as a retrofit. A repository that has stopped growing does not need its prose pruned; it needs its prose sorted, so that the next repository starts with the A facts as prose, the B rules as gates from day one, and the C coaching as an open question with a criterion attached. That is what we did not have in February, and it is what the seven months bought.

Before the design, the things the records do not show, because the reader who is still here is the one who checks.

• There is no measurement of what the prose costs per session, in tokens or in behaviour.
• There is no A/B of a pruned CLAUDE.md against the full one; the whole article argues for an experiment nobody has run.
• What the usage tool excludes from its analysis is undocumented, so “robots” is our reading of the analysed-versus-total gap, not the tool’s statement.
• April and May commits sit under a shared identity and cannot be split by author.
• The Cherny YC quote is from a transcript, not a published text.
• How Anthropic works internally is not documented; the hypothesis above is mine.
• One pipeline, two people, one of them not a developer; nothing here is a sample.

The experiment is reversible, it is designed for the next project, and it promises nothing

The design is short, and I write it as a design, not a report, because it has not run. For a new project: keep the A facts as prose and put them at the top of the file. Take one B rule and put it into a hook on day one, before the first defect, not after the second. Leave one C rule out. Two weeks. And before any of it, write the regression criterion down: what “worse” looks like for the missing C rule, in a form somebody else could check. That is the same discipline as a positive control in a measurement. If you do not know what the signal of the rule’s absence would look like, you cannot see it, and Cherny’s “see what the model does” turns into the feeling that kept us where we were.

flowchart TD
  R["A line in CLAUDE.md"] --> Q1{"Could the model infer it
from the repository?"}
  Q1 -->|no| A["A · keep as prose, near the top"]
  Q1 -->|yes, or partly| Q2{"Could a script say yes or no
without judgment?"}
  Q2 -->|yes| B["B · hook before the action,
or validator after the output"]
  Q2 -->|no| Q3{"Is the regression criterion
written down?"}
  Q3 -->|yes| C["C · leave it out for two weeks,
then keep or delete on the result"]
  Q3 -->|no| W["Write the criterion first"]

The A/B/C triage for one line of guidance: facts the model cannot infer stay as prose, mechanical rules become a hook or a validator, coaching is tested by a two-week absence with a criterion written before, not after.

What a result of “nothing changed” means depends on the letter, and it is worth saying before the two weeks rather than after. For a C rule, nothing changed means the model carried it; delete the line, and note the model version next to the deletion, because the result is dated. For a B rule moved into a hook, nothing changed is not a result about the model at all; it means the hook was wired correctly, and it says nothing about whether the model would have followed the prose. For an A fact, there is no experiment to run; a fact outside the code stays outside the code however good the model gets, and the only test is whether the fact is still true, which is a review, not a measurement. And what “nothing changed” does not mean, for any letter: that the next model will behave the same, or that a project with a different shape of work will.

I said the question changed in the middle, from why we did not know to where the answers went. Here is the answer to the better question, with what we will do next. The answers went to prose, 1 042 lines of it, three gates, seven validators, and by the sort above most of the B rules are in the wrong home, and the C rules are untested; the one we can trace, our oldest, the prose did not hold even after we tightened it. The next project starts with the sort done, one B rule in a hook on day one, one C rule left out with a criterion on the wall, and two weeks. I promise no result from it. My bet is this: the rule you are afraid to delete is the one nothing enforced. Move it to where it is enforced, and then delete the fear.