Hands-on workshop
Agents write the fixes. You build the gate that decides which ones are real.
Agent-written security fixes all pass their own tests. The work is telling which ones actually close the hole — and doing it faster than anyone can read diffs by hand.
You are the judge. Then you teach the machine to judge. Everything runs locally against OWASP Juice Shop, an intentionally vulnerable training app. Nothing is installed — the app and its dependencies come in the pack.
What one gate did in a single run
108
agent-written fixes judged
16/22
known-bad caught, 0 false blocks
All judged end to end in under four minutes, with no one reading a diff. Where a runtime verifier exists the gate catches the plausible-but-wrong fixes; where none exists yet, it escalates instead of rubber-stamping.
Before you start
Five minutes of setup. You'll do the whole lab in one browser tab.
- 1A Mac or Linux laptop with Node 20 or newer. (Windows: use the Docker image below.) Check with
node -v.
- 2A modern browser. The lab is one page at http://localhost:7777.
- 3For Acts 4 & 5 only — a coding agent: Claude Code signed in, or an OpenRouter key. Acts 1–3 need nothing.
Get the lab
One download. Unzip it and you're ready — no install, no npm.
Download links go live for the session.
Run it
- 1Unzip the file. It makes a folder.
- 2Open the folder and double-click
run.command (Mac) or run ./run.sh (Linux). First time on Mac, if it's blocked: right-click → Open.
- 3A terminal opens. Wait for the line
console -> http://localhost:7777.
- 4Open http://localhost:7777 in your browser. The lab loads. Everything below happens on that one page.
Windows / Docker: run the image and open the same URL.
docker run --rm -p 7777:7777 fixgate-lab
# then open http://localhost:7777
The five acts
Follow these in order. Each one builds on the last.
Act 1
See the hole
Pick an attack and click RUN ATTACK. It boots a clean app and runs the real exploit — one shopper reading another's basket. The bug every fix in the queue claims to close.
Act 2
Judge the queue
Run the "tests-only" gate: almost everything merges — every fix passed its own tests. Switch to the full gate and re-judge: half flip to red. Green tests don't mean the hole is closed.
Act 2b
Teach the gate
The gate ships almost empty. Open fixgate/probes, fill a blank check (one that the attack is blocked, one that normal use still works — ask Claude to help), save, and re-judge. Your score climbs.
Act 3
The agent fights back
Switch the queue to "fleet (no answer key)". One fix passes every check and still leaves a way in — it merges. Turn on diff-probes and re-judge: now the back door is caught.
Act 4
Close the loop · needs an agent
Open a rejected row and send it back to the fixer. A coding agent gets the failing checks as its spec, rewrites the fix, and the gate re-judges — up to three tries. Whatever still fails goes to a named person who signs off.
Act 5
Can the machine write the gate? · needs an agent
Let an agent write the checks and score it against yours. It tends to catch as many bad fixes as the experts — and to wrongly block a good one. That false block is why an automated verifier still needs a human-checked answer key.
Set up the agent for Acts 4 & 5
Only needed for the last two acts. In the terminal, before (or instead of) starting the console:
# Option A — Claude Code (signed in on your machine)
export FIXGATE_AGENT_CMD="claude -p --dangerously-skip-permissions"
# Option B — OpenRouter (no local agent to install)
export OPENROUTER_API_KEY="sk-or-..."
export FIXGATE_AGENT_CMD="node fixgate/agent-openrouter.mjs"
The agent only ever sees what failed and what must keep working — never your check code. A mid-tier model (a Sonnet-class Claude or a GPT-class model) is plenty.