Cascadedocs

Guides

Trace a workload

Profile GPU timings and selected NVBit steps in your own Modal app; the package does not run CPU simulation.

python
# app.py
import modal
from cascade_profiler import Workload, register

app = modal.App("my-profiler")
runs = modal.Volume.from_name("my-profiler-runs", create_if_missing=True)
hf = modal.Volume.from_name("my-hf-cache", create_if_missing=True)
prof = register(app, runs, gpu="H100!", hf_cache=hf)

@app.local_entrypoint()
def main():
    wl = Workload("facebook/opt-125m", ["Hello, my name is"], max_tokens=32, repeats=4)
    run = prof.profile.remote(wl.to_dict(), trace_steps=[0, 5])   # GPU: baseline -> nsys -> NVBit on steps 0,5
    print(run["run_name"], run["stage_wall_s"])
    print(f"modal volume get my-profiler-runs {run['run_name']} ./{run['run_name']}")

Optional S3 output: replace the registration above with:

python
prof = register(app, runs, gpu="H100!", hf_cache=hf,
    s3={"endpoint": "https://s3.example.com", "bucket": "my-runs", "prefix": "profiles",
        "region": "auto", "addressing_style": "path"},
    secrets=[modal.Secret.from_name("my-s3-credentials")])

Put AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and optionally AWS_SESSION_TOKEN in that app-owned secret or the GPU runtime environment, never in destination options or files. The finished run uploads to s3://my-runs/profiles/<run_name>/, retaining the volume copy; run.json and traced/metadata.json record the destination and URI.

Run and download#

Use Python 3.12 locally: register() serializes Modal functions, so client and container Python minor versions must match.

bash
pip install "cascade-profiler[modal] @ git+https://github.com/atoniolo76/cascade@profiler"
modal setup
modal run app.py

The example prints the modal volume get command to download the run. The first invocation builds the GPU image; later runs reuse it.

The GPU function runs baseline (no profiler), nsys capture, then NVBit tracing of the selected steps. Outputs include corrected_timeline.json, trace_alignment.json and traced/traces/; traced-run timings are not used for reconciliation. S3 uploads happen at the end, with run.json last; keep_nsys_rep=True is rejected with S3 output to avoid exporting an unscrubbed environment capture.

See the OPT-125m sample analysis for measured costs, replay errors and limitations; its CPU simulation uses our own tooling, not the profiling package.

Legacy streaming into Cascade#

For the app's streaming trace layout, use accelsim_trace.py from validate-accelsim-modal (commit b348d7d), not the package example above. Pass the endpoint from Deploy the app: bucket cascade, path-style addressing.

bash
modal run accelsim_trace.py --model facebook/opt-125m --list-layers
modal run accelsim_trace.py \
  --model facebook/opt-125m \
  --layers model.decoder.layers.10.self_attn \
  --prompt "Hello, my name is" \
  --max-tokens 4 \
  --s3-endpoint https://<your-s3-endpoint> \
  --s3-bucket cascade \
  --s3-addressing-style path

Choose layer paths from --list-layers. --s3-prefix defaults to accelsim-traces and must match the app's CASCADE_S3_PREFIX; --s3-region defaults to auto. Use --run-name for an explicit identifier. --spinlock-handling defaults to mark_region; alternatives are fast_forward and none.

This legacy CLI checks bucket access before tracing, streams closed kernel files and deletes the local copies. It also uploads logs, spinlock data and metadata.json, including on failure, under s3://cascade/accelsim-traces/<run_name>/. The app post-processes this layout; see Traces. Package uploads instead retain the full profiling run layout.

Upload a legacy trace you already have#

scripts/upload_trace.py sends metadata.json last with status and a files list. Use --meta key=value for trace metadata, such as the GPU architecture (the app defaults to sm_90 for H100):

bash
uv run python scripts/upload_trace.py ./my-run --run-name my-run \
  --endpoint https://<your-s3-endpoint> --meta trace_gpu_arch=sm_80

Add --status failed to mark the trace failed without processing it.

esc