blog / mcp
A headless Scratch VM over MCP: 112 tools for AI agents to build and publish Scratch projects
scratch-unified-mcp is the first standalone headless Scratch VM exposed over the Model Context Protocol — 112 tools behind one stdio command that let an AI agent load a .sb3, press the green flag, click sprites, step frames, read every live thread, and watch variables change. No browser attached. Everything assertable from a tool call.
There were already Scratch tool ecosystems — but each one covered a slice. One talked to the Scratch website. One edited blocks with a headless VM. One converted Python to blocks. None covered the whole loop, and switching between three servers with three naming schemes killed momentum. scratch-unified-mcp merges all three behind a single transport: python3 -m scratch_unified.
Why a headless VM is the piece that was missing
Driving Scratch through the website API only gets you so far: you can read a project and write a project, but you can't run it and see what actually happens. The piece that didn't exist anywhere else is a headless TurboWarp scratch-vm you can drive and inspect over MCP — load a project, run it, and get back an ordered timeline of what fired: say bubbles, broadcasts, questions and answers, errors.
The VM tool surface
The sb3_* family (60 tools, proxied to a lazily-spawned Node sidecar running interpreter-mode TurboWarp) is where the magic lives:
sb3_vm_load/sb3_vm_green_flag— load a project, press the flag (clears bubbles, questions, errors)sb3_vm_run— advance N seconds or frames, paced or flat-out, returning state plus an ordered event timelinesb3_vm_state— full snapshot: targets (x/y/vars/lists/costume), monitors, bubbles, question, thread count, errorssb3_vm_threads— every live thread: target, clone flag, starting hat, stack depth, status, kill flagsb3_vm_step_frame— exactly one frame, with before/after counts and the deltasb3_vm_input— keys, mouse position and clicks in stage coordinates,askanswers — including a click shim sowhen this sprite clickedactually fires headlesssb3_vm_seed— deterministic PRNG (mulberry32), so the same seed gives the same runsb3_vm_watch— poll-and-diff variable watcher, per-clone capablesb3_vm_pen_png— pen raster as PNG base64;sb3_vm_mix_wav— an offline audio mix as WAV
The gaps are patched, not hidden
Running a VM with no renderer means upstream touching always returns false, JIT mode trips false pick random alarms, and clicks never land. The project patches these instead of pretending they don't exist: a distance-based touching fallback, a sprite-click shim, broadcast logging via wrapped startHats, interpreter mode, and a seeded RNG where scratch-vm has none. Deterministic in, deterministic out.
Getting started
Point any MCP client at it:
git clone https://github.com/blessed0x/scratch-unified-mcp
cd scratch-unified-mcp && pip install -e .
# optional: clone the three upstream reference repos (see README)
// MCP client config
{ "mcpServers": { "scratch-unified": {
"command": "python3",
"args": ["-m", "scratch_unified"],
"cwd": "/path/to/scratch-unified-mcp",
"timeout": 600000
} } }The verification suite backs it up: 34 offline checks on the tool surface, 41 runtime checks against a live headless VM playtest, and a 42-check static gate on the game generator that regenerates the demo .sb3. Full per-tool reference lives in docs/IDENTIFIERS.md.
Where this lands
An agent loop that can build a Scratch project, run it headlessly, read the screens, and iterate until the tests pass — then publish. That's the whole loop, and it's open source on github.com/blessed0x/scratch-unified-mcp. AI agents get real tools; Scratch gets a test runner it never had.