Run the refinery locally or on AWS.
Refine runs on your machine or on AWS Batch. Test’s checkpoint intake and simulation are available for development. The physical Gate A benchmark is pending.
Install and open the dashboard Built
Install uv, Python 3.12, Node 22 or later, and ffmpeg. Run these commands from the repository root.
uv venv -p 3.12 .venv
uv pip install --python .venv/bin/python -e '.[gpu]'
npm --prefix site ci
CUPEL_DATA_DIR="$PWD/data" npm --prefix site run devOpen localhost:4173/dashboard, add videos with a consent record ID, and start a run. Footage stays in your data/ folder, which git ignores.
cupel run --local Built
The dashboard writes run.json and sources.json for each run. A minimal run.json contains {"scorer":"vjepa"}. Each source needs a path, a consent record ID and a source ID. Use an opaque session_id to group recordings that must stay together in an evaluation split.
node cli/cupel.ts run --local data/runs/<run>
node cli/cupel.ts run --local data/runs/<run> --from <stage>status.json records progress and log.txt captures stage output. After fixing a failure, rerun starting at the failed stage.
Run the same pipeline on Spot Built
Use the CDK storage and Batch stacks in backend/ with worker images configured. Set your AWS CLI credentials and region for that account. Put s3:// source URLs and consent record IDs in sources.json.
export CUPEL_ENV=dev
export CUPEL_BUCKET='<DataBucketName>'
node cli/cupel.ts run --batch data/runs/<run> --shards 4
node cli/cupel.ts run status data/runs/<run>CPU stages run on Fargate Spot. GPU stages run on EC2 Spot. A status refresh copies the report and manifest to the run folder and records the completed export inventory for the dashboard. Videos and datasets stay in S3.
The nine stages
Cheap stages run first so they discard the most footage before any GPU work. Each stage reads and writes the clip manifest, a parquet file keyed by clip ID that carries the source file, timestamps, consent record ID, score version and every stage's output.
| # | Stage | What it does |
|---|---|---|
| 1 | clip | Cuts video into 3 s clips at 640×360, 30 fps. Drops a final remainder under 2 s. |
| 2 | filter | Drops blurry, dark, shaky and idle clips, and clips with no hands. Blurs faces and screens in every frame. |
| 3 | embed | Runs frozen V-JEPA 2 on 16 frames of each clip. Stores vectors in parquet with a FAISS index. |
| 4 | dedup | Removes near-duplicates and caps clips per cluster. |
| 5 | score | Scores each clip by V-JEPA's prediction error and keeps the middle band. Swappable for a simple-filter baseline. |
| 6 | caption | Finds task steps and captions each one with Qwen3-VL-2B, run locally. |
| 7 | actions | Extracts 3D hand pose per frame with MediaPipe. |
| 8 | report | Writes the coverage report: task clusters, hours per cluster, gaps. |
| 9 | export | Writes WebDataset shards, one reel of every kept clip, and a LeRobot dataset with one episode per clip. |
What a run leaves behind
| Path | Contents |
|---|---|
manifest.parquet | One row per clip, with every stage's output |
clips/ | Blurred clips only |
report.json | Funnel, coverage and timing |
export/ | WebDataset shards, reel.mp4 and lerobot/, a LeRobot v2.1 dataset |
Register checkpoints and run simulation Built
Register a checkpoint URI and the robot it targets. The ledger keeps the reference and its ancestry. For LeRobot or openpi, a Python callback can watch the training output and register each saved step.
node cli/cupel.ts test push <checkpoint> --uri <uri> --robot <robot>
.venv/bin/python -m cupel.callback <output-dir> --robot <robot> --framework lerobot --onceFor openpi, use --framework openpi. Keep the callback’s receipt in the training output folder so a restart can resume with the same checkpoint ancestry. For remote registration, set CUPEL_API_TOKEN to a workspace write token and configure the checkpoint’s S3 bucket in Cloud settings. Sync the training output to that S3 location before registration.
.venv/bin/python -m cupel.callback <output-dir> --robot <robot> --framework lerobot \
--uri-prefix s3://<workspace-bucket>/<training-prefix> --api-url https://<workspace-host>For a local PushT simulation, install the sim extra in a supported environment. With a configured Batch sim worker, submit using --batch and collect the run using the simulation ID it prints.
node cli/cupel.ts test sim <checkpoint> --env pusht --episodes 20 --seed 0
node cli/cupel.ts test sim <checkpoint> --env pusht --episodes 20 --seed 0 --batch
node cli/cupel.ts test results <sim-id>Train the three Gate A arms
Start with a Gate A experiment that preregisters --training-steps and --training-seed, and a shared SO-101 demonstration dataset. The Batch command reads the training values from that plan and uses one demo snapshot and worker revision for all arms. See EVALUATION.md in the repository for the study setup.
node cli/cupel.ts train gate-a <experiment> --demos <s3-demo-prefix> --batch
node cli/cupel.ts train status <experiment>Physical evaluation and pilots
Gate A requires a blinded SO-101 study with 240 physical trials. The real-robot result, customer pilots, fitted world models and production fleet deployment are planned. The current Track record shows the study gates.