This timer has no API. So we gave it one.

01

Meet the rig

Drag it to turn it and pull it apart. Pick a layer to see what it does.

02

Watch it press buttons

03

Try it yourself

Two replays, one pipeline. Same brain, different machine. Press play, watch the finger work, and be the human on the risky step. The timer is the real rig device; the incubator is a demo profile showing the same code driving a second machine.

Replays run from fixed scripts in this page, built from the real fixtures and a recorded risk score. The full stack — planner, vision, scorer, finger — runs in the repo below.

04

How it works

1. Eyes. A webcam watches the panel. SAM 3.1 finds the buttons and the screens; Muse Spark reads what the screens say, voting across reads.
2. Brain. Muse Spark plans the presses from the live screen read, then the plan is checked in code against that same read before anything moves.
3. Finger. An ESP32 drives a servo that presses the real button. Two fingers on the rig: H and ST/SP.
4. Gut check. Muse Spark Open scores each risky step with a tier and a one-line reason. It can only add, never remove an approval.
5. The call. A human holds approve. Steps that turn something on always need the hold, whatever the score says.

web app  →  backend (HTTP + WebSocket)  →  MCP tools: get_controls, press, read_screen, get_frame, stop
                          ↑                              ↓ press holds the approval gate
              Muse Spark (plan) + SAM 3.1 (find)      ESP32 + servo → machine
              Muse Spark Open on Baseten (risk)       webcam → screen reads

05

Which Meta model does what

Two real risk cards, one per replay

06

Safety

Every rule below lives in the code, not just on this page.

• The approval gate lives inside the press tool, so the agent can’t skip it. (backend/mcp_server.py)
• Steps that turn on output always need hold-to-approve, whatever the risk score. The backend refuses a tap. (backend/api.py)
• The risk scorer can only add a tier and a reason, never remove an approval. If it fails, the step scores high. (risk/scorer.py)
• Reads are voted: two reads agree, or a third breaks the tie. A low-confidence read pauses the job instead of pressing blind.
• Press budget: 30 presses per job. Going over is refused, nothing pressed.
• Plans that don’t add up get refused with a plain reason. (backend/planner.py checks the plan against the screen)
• A press that didn’t register gets retried — never on an approval step, where a double-press would undo the job.
• Buttons with no finger are never planned. STOP is always on screen and lifts the finger right away, even mid-hold.

07

The numbers

Only numbers with a source are here. Everything else says so. Full table, sources and check dates: docs/costs.md.

Replace the machine, or give it an API

08

Built during the hackathon

• Pam (@PamAntonio): the idea, the first contract, the first planner and device profiles, the ESP32 firmware, the serial protocol, and the rig, including the press angles.
• Bhavi (@Bhavishaahuja): the MCP server, the planner hookup and later planner work, the backend runner, vision, the risk scorer, the web app, calibration, and the live camera.

Everything in this repo was written during the hackathon, from Oct 27, except the very first commit: the README was created on Oct 26 (b06e0dc, Pam). web/ started from create-next-app.

We used Claude Code as a coding assistant. The Co-Authored-By trailers in the commit history show which commits. This site was built with Muse Code. Commit history

09

Run it yourself

No hardware needed. This runs the backend on FakeHardware plus the web app. You need your own Meta Model API key and Baseten key in a repo-root .env file (MODEL_API_KEY and BASETEN_API_KEY).

python3 -m venv .venv
.venv/bin/python -m pip install -r backend/requirements.txt
FAKE_CONFIDENCE=high FAKE_LOW_READ=2 FAKE_MISS_PRESS=btn_h .venv/bin/python backend/api.py

cd web
npm install
NEXT_PUBLIC_USE_MOCK=false NEXT_PUBLIC_BACKEND_URL=http://localhost:8000 NEXT_PUBLIC_DEVICE_ID=timer-01 npm run dev

python -m venv .venv
.venv\Scripts\python -m pip install -r backend\requirements.txt
set FAKE_CONFIDENCE=high
set FAKE_LOW_READ=2
set FAKE_MISS_PRESS=btn_h
.venv\Scripts\python backend\api.py

cd web
npm install
set NEXT_PUBLIC_USE_MOCK=false
set NEXT_PUBLIC_BACKEND_URL=http://localhost:8000
set NEXT_PUBLIC_DEVICE_ID=timer-01
npm run dev

Open http://localhost:3000. The fake flags replay the low read and the missed press from the timer replay above. Restart the backend before every take, each flag fires once per start. Full flags and rig setup are in the repo README.