AI Learning Series, Lesson 3: Documentation, Rules, and Skills

AI Learning Series · Lesson 3 of 4

Every fresh AI session starts with no memory of your repository. If the conventions only live in someone's head — or in a doc nobody re-reads — the model breaks them quietly, and the break looks like working code until it doesn't.

Status: outline. The on-screen walkthrough hasn't been recorded yet — this post is the plan it will follow, and it gets the real walkthrough once the lesson airs.

What You'll Be Able to Do

• Give a repository three distinct things: documentation it can navigate, rules it must follow, and skills it can reuse
• Sort any piece of guidance into the right one of the three
• Turn a written rule into a check that fails when the rule is broken

Builds on lessons 1 and 2: lesson 2's standing context was one blob of instructions. This lesson splits it into layers with different jobs, so each session loads only what it needs — lesson 1's context-cost idea, applied at repository scale.

Out of scope here: measuring or cutting spend (lesson 4), and building a full application. The on-screen build is deliberately small — the subject is the structure around it.

The Three Layers

Documentation — Findable, One Owner per Fact

• An index, one clearly owned fact per topic, and a change-impact table
• Docs stay findable and don't silently go stale when something they describe changes

Rules — Short, Always Loaded, With Reasons

• The always-loaded file states the don'ts — and cites why, not just what
• Anything longer belongs in a linked doc, not the file every session pays to load

Skills — Procedures Loaded Only When Needed

• A skill is a reusable procedure the assistant loads when a specific, repeatable task comes up
• The worked-example repository has no project-level skills directory today, so the skill is built live — not staged for the demo

The Worked Example

The lesson runs against a real repository in day-to-day use, not a toy built for the lesson. Five patterns, each doing a distinct job:

Entry-Point File — Read First

• One file says what the project is, what to do, and what not to do — before anything else
• When more than one AI agent works in the repo, it's mirrored to each agent's expected entry point, so no agent reads a stale or partial copy

Test-Enforced Rule — Stronger Than Prose

• A rule a test checks beats a rule that lives in a paragraph someone might skip
• This blog does it for translations: category and date must stay byte-identical across languages, and npm run check:translations fails if they don't

Decision Queue — Authorized Before It Starts

• Work gets written down and approved before it begins
• No agent infers scope from a vague chat message

Change-Impact Index — "If You Change X…"

• A table that says which docs go stale when a given file changes
• The connection gets looked up, not rediscovered after something breaks

Definition of Done — Varies by Change

• What counts as verified depends on the kind of change
• A doc edit needs different proof than a schema migration

# Rules File — Skeleton
## What this project is
One paragraph. Link to README for everything else.

## Never
- Edit generated or vendored directories (why: overwritten on update)
- Commit secrets or expose them via public env vars (why: shipped to the browser)
- Mark work done without the check for its change type (why: see Definition of Done)

## Where things live
- Architecture → docs/ARCHITECTURE.md
- Change impact → docs/CHANGE_IMPACT.md
- Definition of done → docs/DONE.md

On Screen

• The failure mode first — a fresh session makes a change that violates a convention nobody wrote down where the model could find it
• Documentation structure — index, one owner per fact, change-impact table
• Rules — what belongs in the always-loaded file vs. a linked doc
• Enforcement — one written rule becomes a test, broken on purpose, then fixed
• A skill, built live — for a task that actually repeats, then compared against running without it
• Sorting drill — real guidance sorted into docs, rules, or skills, including one sorted wrong on purpose and corrected

Which Layer Does It Belong In?

# Sorting Guidance
Is it a fact about how the system works?          → Documentation
Is it always true, and does breaking it hurt?     → Rules (with the why)
Is it a procedure you repeat for a specific task? → Skill
Can a test check it?                              → Also write the test

Verify Before You Rely On It

• Every file and path in the worked example still exists and still says what's claimed — repos drift, and this lesson leans on real paths
• What the demonstrated tool calls a rules file and a skill, where each lives, and how each loads
• Whether skills load on demand or always in that tool — the live skills segment depends on it
• Whether the rules-file size guidance still matches the provider's current advice
• That the live-built skill is one that actually gets reused afterward, not a demo prop

Common Questions

"Can't I just put everything in the rules file?" You can, and every session pays for it. Long rules files get skimmed by people and diluted for models. Keep rules short; link out for the rest.

"How do I know a rule is actually followed?" Write a check that fails when it isn't. A rule without a check is a suggestion.

"When does a prompt become a skill?" The third time you paste the same procedure. Save it once and load it when the task comes up.

Next: Lesson 4

Lesson 4 is last on purpose. It uses everything the first three lessons teach — model choice, prompt and context quality, and repository structure — as the levers for cutting what an AI workflow costs.