Guides

Runs

A run simulates one ready trace on a list of GPU configs, with a count for each. Every count is its own sim, and every sim is its own sandbox.

Launch a run in the UI#

Open a ready trace and choose New run.
Add a row per config and set its count, from 1 to 64. The trace's suggested_sim_config is preselected. Without one, the first compatible config is.
Choose Launch 6 simulations. The button counts the sims for you. The run screen opens and updates live.

Configs for a different architecture than the trace's trace_gpu_arch are disabled. An H100 trace contains Hopper-only instructions, so it runs only on sm_90 configs such as SM90_H100 and SM90_H200. To model another architecture, trace on that GPU.

Launch a run over the API#

Create a run takes the trace id and the entries.

bash

curl -X POST "$CASCADE_URL/api/runs" \
  -H "Authorization: Bearer $CASCADE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"trace_id": "demo", "entries": [{"gpu_config": "SM90_H100", "count": 4}, {"gpu_config": "SM90_H200", "count": 2}]}'

The app checks the request before creating anything.

The trace must exist (404), be ready, and not be a sample (409).
Every gpu_config must be one List GPU configs returns, with the trace's architecture (422).
entries can't be empty and every count is 1 to 64 (422).

The response is the new run with every sim pending or launching. Launching happens in the background. Concepts walks through it.

Follow a run#

The run screen shows a card per sim with its state, how long since its last heartbeat, kernels done out of the trace's kernel count, the kernel it's on, its latest metrics, and the last lines of its log. Over the API, read Get a run or subscribe to Stream run events.

The run's own state comes from its sims.

sim states	run state
`running`, `succeeded`	`active`
`succeeded`, `succeeded`	`succeeded`
`succeeded`, `failed`	`partial`
`failed`, `failed`	`failed`
`cancelled`, `failed`	`cancelled`
`cancelled`, `succeeded`	`partial`
`cancelled`, `cancelled`	`cancelled`

Cancel a run#

On the run screen, choose Cancel run and then Confirm cancel, or call Cancel a run. Every sim that hasn't finished becomes cancelled, the app stops its sandbox, and the simulator's next heartbeat gets cancel: true. Sims that already finished keep their results. Cancelling twice is harmless.

When a sim goes quiet#

A sim with no heartbeat for 60 seconds shows as lost. It goes back to running when it reports again. A sim still lost after 600 seconds becomes failed and its sandbox is stopped. Both times scale with CASCADE_HEARTBEAT_TIMEOUT_S.

After a restart#

The app keeps everything in SQLite at $CASCADE_DATA_DIR/cascade.db. When it starts again, it launches any sims still pending. Sims on Sail run in their own sandboxes, so they keep going and their heartbeats land again once the app is back.