# Session 04 — Judgment · Facilitator Guide

**Deck:** [deck-session-04.html](deck-session-04.html) · **Handout:**
[session-04-exercises.md](session-04-exercises.md) · **Length:** 150 min

**Session goal:** every student leaves with a signed one-page delegation
policy, guardrails they have personally attacked and patched, and a
governed full stack scheduled for tonight. This is the capstone — the
session that separates "I made an agent do a thing" from "I run an agent
responsibly."

Energy note: this is the fun one. The red team (Exercise 2) is
adversarial, competitive, and gleeful by design. Lean into it.

---

## Run of show

### 0:00–0:15 · The two failure modes (slides 1–2)

- Open with the miss-log delta from homework: hands up if misses dropped
  after memory night. They did. *"That's your system getting better
  because you governed it, not because you hoped."*
- Slide 2: too much trust (A) vs too little (B). The insight that reframes
  the session: **everyone guards against A; almost everyone is quietly
  living in B**, approving the same safe draft forty times. Ask: "who's
  approved the identical harmless thing more than five times this week?"
  — hands. "That's failure mode B. It has a cost too."

### 0:15–0:35 · The delegation ladder (slides 3–4)

- Six rungs, slide 3. Read them as a progression of earned trust, not a
  menu. Rung 1 (observe) is where everything starts — *"no exceptions,
  even for obviously-safe jobs."*
- Slide 4, the three placement questions: reversible? blast radius?
  proven accuracy? Drill the rule of thumb until they can recite it:
  **irreversible + high blast radius = never above rung 4.** Money and
  relationships live at rung 4 permanently.

### 0:35–1:05 · EXERCISE 1 · The ladder audit (slide 5)

- 12 tasks, placed solo, then argued in fours. This is a values
  conversation disguised as a technical one — the same task lands on
  different rungs for different lives, and that's correct.
- Float between groups. Seed the hard ones if a group settles too fast:
  "you put 'reply to mom' at rung 5 — what if mom's email is actually
  your mother-in-law asking about money?"
- Room reveal: surface the two tasks groups placed most differently.
  Don't resolve them — *"both are right, for different people. Judgment
  is personal. That's why you write your own policy, not download mine."*

### 1:05–1:20 · The mechanics (slides 6–7)

- Four instruments: allowlist, approvals, logs, kill switch. All boring,
  all essential.
- **Make everyone practice the kill switch now, live** (slide 7, last
  row): pause the schedule, unpause it; revoke a connector, re-add it.
  Ten seconds each. *"Do this until it's muscle memory — you want it in
  your hands before you need it, not after."* This is the single most
  important 90 seconds of the session.

### 1:20–1:25 · Break

### 1:25–1:35 · The attack (slide 8)

- Prompt injection, formally. Callback hard to email #17 from Session 02:
  *"two weeks ago I planted a trap in your sorting exercise. Tonight you
  find out if your agent falls for it."*
- The layered defense: NEVER rule + locked allowlist + rung-4 approvals
  on anything leaving the machine. *"Any one layer can fail. All three
  rarely do. That's why we don't rely on the model being clever."*

### 1:35–2:10 · EXERCISE 2 · Break your own agent (slide 9)

**The centerpiece. Guard the ground rules:**

- ⚠ **Test inboxes ONLY.** Verify at the top of the block that everyone
  has one (you polled at end of S03). Anyone without → they attack a
  partner's test inbox and skip receiving. Never a real inbox, ever,
  even a volunteer's, even "just to see."
- Pairs write two trap emails each from the attack cards (injection,
  urgency, impersonation, flattery), send to partner's test inbox, run
  triage, watch the log together.
- Score: traps caught vs traps landed. Make it a friendly competition —
  the room gets loud, that's correct.
- Every landed trap → patch the constitution → re-run → confirm caught.
  **The patching is the learning**, not the catching. A trap that lands
  and then gets defended teaches more than one caught first try.
- Circulate for the genuinely clever attacks and spotlight them: *"come
  see what Maria built — this one got past three people."*
- ☆ SKIPPABLE if long: two traps per pair instead of four. Never skip
  the patch-and-re-run — catching without patching teaches nothing.

### 2:10–2:15 · Debrief (slide 10)

- *"You just adversarially tested an AI system before trusting it. Most
  companies deploying this stuff haven't. The log you watched is the same
  instrument the professionals use — there is no other magic."* Let that
  land; it's the confidence they leave with.

### 2:15–2:35 · The policy + sign it (slides 11–13)

- Slide 11: the one-page policy, MAY / MUST ASK / NEVER / review date.
- Exercise 3: fill it from their ladder audit, set the 30-day review
  (have the agent put it on the calendar — enjoy that this is now
  trivial for them), **read the NEVER list aloud in their group.**
- Reading aloud is not ceremony — spoken guardrails get edited better
  than written ones. Insist on it.
- Then sign it. Real pens, good paper. The signature makes it feel like
  what it is: a decision, not a setting.

### 2:35–2:45 · Graduation run + send-off (slides 13–17)

- Exercise 4: confirm the five files (job, constitution, memory index,
  policy, schedule), promote ONE earned task from rung 3→4 (their first
  deliberate trust upgrade, on the record), verify kill switch, schedule
  tonight, close the laptop.
- Slide 14, the callback: *"the tool is neutral, the hands are not —
  yours are now trained."* Callback to the pitch page and to Session 03's
  machinery. Full circle.
- Slide 15: their next three agents (calendar negotiator, paperwork
  reader, watchlist) — *"same six parts, same five files, same one
  ladder. You have the whole method."*
- Send-off: invite graduate demos to chris@beyondprompt.ai. Mean it —
  the war stories become future course material.

---

## Common failure points

| Symptom | Cause | Fix |
|---|---|---|
| Student wants to run the red team on their real inbox "to be realistic" | Enthusiasm outrunning caution | Hard no. Explain: the trap emails you're about to write are real attacks; you don't test live ammunition in your own house |
| An agent catches every trap first try | Strong S02 constitution (good!) | Escalate difficulty: have the partner write a subtler injection, or nest it inside a legitimate-looking forwarded thread |
| Policy NEVER list is vague ("don't do bad things") | Abstraction hiding indecision | "A machine can't obey that. Give me a dollar amount, a named list, a verb" — rewrite one line together |
| Student reluctant to sign | Taking it seriously (good sign!) | Honor it — "sign it in pencil, review in 30 days. The point is you decided on purpose, not that it's permanent" |

## What good looks like at the door — and at graduation

Every student: a signed, dated delegation policy · at least one trap
caught after patching · kill switch practiced · five files confirmed ·
tonight's governed run scheduled · laptop closed.

They walked in four weeks ago thinking AI was a chatbot. They leave
running a governed agent overnight and able to explain the building it
lives in. **That's the whole course.**