Getting started
Introduction
Cascade takes a trace of an ML workload captured on one real GPU and simulates it on other GPU configs, many copies at once. You trace once, then ask how the same kernels run on an H100, an H200 or a dozen replicas of each.
How it fits together#
Cascade has three parts. Each part runs in a different place, and they talk over S3 and plain HTTP.
- Profiler. Runs your workload, such as a
vLLMmodel, on a Modal H100 under Accel-Sim's NVBit tracer. It uploads each kernel's trace to the Cascade app while tracing is still running. - Cascade app. One Sail sandbox that receives traces over S3, gets them ready to simulate, launches runs, and shows everything in a web UI and an HTTP API.
- Simulators. Each simulator is a fork of the app's sandbox, so it starts with the trace already on disk. It runs Accel-Sim against one GPU config and reports progress back to the app.
Every workspace includes a sample run. Profiling a workload takes about 30 minutes, so each app opens with a finished one marked Sample: Alessio's facebook/opt-125m trace from an H100 and its Accel-Sim run on SM90_H100, with real step timing to compare against. Its results are real, recorded before your workspace existed, and you can't rerun it because its 415 MB of instruction traces aren't bundled. See samples/opt125m-h100-e2e. Set CASCADE_SAMPLES=false to start without it.
Worked example: OPT-125m#
The same eager workload generates 32 tokens from Hello, my name is, with four repeats (one warm-up). H100 and A100 each have their own native traces and real unprofiled baseline; the H200 prediction replays the H100 traces and has no measured H200 baseline.
| Hardware / config | Real mean E2E | Simulated mean E2E | Error |
|---|---|---|---|
| H100 / SM90_H100 | 254.45 ms | 262.14 ms | +3.02% |
| H200 / SM90_H200 | — | 262.16 ms | — (counterfactual) |
| A100 / SM80_A100 | 222.08 ms | 229.58 ms | +3.38% |
The A100 result is the complete Modal SM80_A100 reference, not the partial Sail attempt. Small request errors retain measured eager host time and are not equivalent to small GPU-model errors. See the full sample analysis for timings, coverage and limitations.
To ask about a different prompt length, output length or concurrent arrival pattern, use the request model and scheduler. The Request simulator UI and /api/scenarios build estimates from a finished sim; /v1 estimates each request alone. Predictions outside the six-token prefill / decode-context 7–37 calibration range are unvalidated extrapolations, not measured results.
Where to go next#
cascade/models.py.