help

Getting started

Post your first run

curl -X POST "$TRACECASE_URL/api/runs" \
  -H "x-tracecase-token: $TOKEN" \
  -H "content-type: application/json" \
  -d '{
    "suite": "refund-agent",
    "label": "PR #142 / opus-4.8",
    "results": [
      { "caseName": "refund under limit", "passed": true },
      { "caseName": "refund over limit escalates",
        "passed": false, "flags": ["unsafe_tool"] }
    ]
  }'

FAQ

How do I post a run?+

POST your suite results to /api/runs with the x-tracecase-token header. The suite is created automatically on first sight.

How does it detect a regression?+

Each run is diffed against the previous run of the same suite. A case that passed before and fails now is marked REGRESSED.

What counts as a flag?+

Any case you submit with a non-empty flags array (for example unsafe_tool or hallucination) is counted as flagged.

How do I fail my CI build on a regression?+

The POST response includes shouldFail: true when there are regressions or flags. Wire that into your CI step's exit code.