Generating Reports

Regtrace can produce structured reports in JSON and Markdown formats for pipeline consumption, PR comments, or sharing with stakeholders.

Output streams

Human-readable output (terminal, markdown, progress, errors) goes to stderr. Structured data (JSON format) goes to stdout. This lets you pipe JSON output while still seeing terminal progress:

regtrace run --format json 2>/dev/null | jq '.suite.suite_score'

Terminal output (default)

regtrace run

This is the default format. It prints colored output to stderr with per-test-case results, metric summaries, and quality gates.

JSON output

regtrace run --format json

Outputs a JSON object to stdout:

• Suite metadata (name, version, run ID)
• Summary (status, score, pass/fail counts)
• Quality gates results
• Regression info
• Judge provider and model
• Per-test-case detailed results

Write to a file:

regtrace run --format json --output results.json

JSON structure

{
  "suite": {
    "name": "my-qa-set",
    "version": "1.0",
    "run_id": "run_20260101_a3f9",
    "duration_ms": 1234
  },
  "summary": {
    "status": "passed",
    "suite_score": 0.85,
    "total_test_cases": 5,
    "passed": 4,
    "failed": 1
  },
  "quality_gates": {
    "passed": true,
    "gates": {
      "suiteScore": { "passed": true, "actual": 0.85, "minimum": 0.7 }
    }
  },
  "regression": {
    "status": "clean",
    "suite_delta": 0.02,
    "baseline_run_id": "run_20251231_xyz"
  }
}

Markdown output

regtrace run --format markdown --output report.md

Produces a structured Markdown document with:

• Report header (name, version, run ID)
• Metadata table
• Summary table with status and scores
• Per-metric scores table
• Quality gates section with pass/fail per gate
• Regression section (if baseline exists)
• Per-test-case result tables

Write to a file:

regtrace run --format markdown --output report.md

This file is suitable for posting in PR comments or Slack.

Automated reports

In CI, generate reports automatically:

- run: regtrace run --format json --output results.json
- run: regtrace run --format markdown --output report.md