Reports & Diagnostics

What every run produces

reports/
  latest.html           # newest run, human-readable
  latest.json           # newest run, full data
  latest-summary.json   # newest run, compact summary
  latest-junit.xml      # newest run, CI format
  runs/
    <run-id>/
      report.html
      report.json
      summary.json
      junit.xml
      artifacts/
        screenshots/
        traces/

The reports directory. latest. always points at the newest run.*

File	Audience	Use it for
report.html	Humans	Debugging failures: screenshots, errors, stack traces, source frames.
report.json	Scripts	Full run data including artifacts, traces, and smart evidence.
summary.json	CI / dashboards	Compact pass/fail/flaky counts, duration, and report paths.
junit.xml	CI test publishers	GitHub, Jenkins, GitLab, Azure DevOps test result views.

Reading the HTML report

The HTML report is built to answer “what happened?” without rerunning anything. For each failed test you get:

• Failure diagnostics with a likely cause and suggested next actions.
• The inline failure screenshot, what the page looked like at the moment of failure.
• The error message and stack trace.
• A source code frame highlighting the failing line of your test.
• The trace timeline, when the run used --trace.
• Smart browser evidence, when the run used --smart-report.

On local machines, a failed run automatically starts a small report server and opens the failed report in your browser. Disable it with --no-open-report-on-failure, or pin the port with --report-port 9323. CI never opens it.

Trace: a step-by-step record

orbittest run tests/login.test.js --trace

With trace on, the report gains a Trace Timeline: every orbit.* step with its status, duration, page URL and title, failed-step details, and links to step screenshots. Raw trace files land under reports/runs/<run-id>/artifacts/traces/.

Trace answers the question “what was the test doing right before it failed?”, which is usually the fastest path to the bug. In CI, the default is trace: "on-failure", so you pay the capture cost only when it matters.

Smart Report: browser evidence

orbittest run tests/login.test.js --smart-report

Smart Report watches the browser itself while your test runs. It collects console errors, page JavaScript exceptions, failed network requests, slow requests (threshold configurable via smartReportSlowRequestMs), recent navigation, and the current URL. This usually settles the key question on a red build: did the app break, or did the test?

One behavior to know: when Smart Report sees a clear application failure, an unauthorized response, a failed API call, a client-side crash, it marks the test failed even if your script did not assert at that point. The reasoning: a test that clicks through a broken page and reports green is lying to you.

Step mode: live debugging

orbittest run tests/login.test.js --step

Step mode opens the Orbit Inspector in OrbitTest’s managed browser. It runs with one worker, disables timeouts, pauses before each orbit.* action, and leaves the browser open at the end. Every click shows its red coordinate dot, so you can confirm OrbitTest is clicking exactly where you expect.

• Step runs the next highlighted action, then pauses again.
• Resume continues without pausing.
• Stop ends the debug run.

Rule of thumb: use --step to watch and control a run live; use --trace when you want evidence after the fact, especially from CI.

Reading reports from scripts, and cleaning up

const fs = require("fs");
const summary = JSON.parse(fs.readFileSync("reports/latest-summary.json", "utf8"));

console.log(`Status: ${summary.status}`);
console.log(`Passed: ${summary.summary.passed}, Failed: ${summary.summary.failed}, Flaky: ${summary.summary.flaky}`);

if (summary.status !== "passed") process.exit(1);

summary.json is small and stable, ideal for custom pipeline gates.

Reports accumulate. The cleanup command keeps the latest run, the last 10 passed and last 30 failed runs, and removes anything older than 30 days, all adjustable:

orbittest clean-reports --dry-run   # always preview first
orbittest clean-reports
orbittest clean-reports --passed 20 --failed 50 --max-age-days 60

Frequently asked questions

Why keep more failed runs than passed runs by default?

Failed runs are the ones you investigate later, the evidence is the whole point. Passed runs mostly confirm a green state you already trust.

Do traces slow tests down?

Capture adds some overhead per step. That is why CI defaults to on-failure capture, and why you enable --trace locally only when investigating.

Can I publish the HTML report from CI?

Yes, upload the reports/ directory as a pipeline artifact. The report is self-contained static HTML. The CI/CD page has a complete GitHub Actions example.

Related pages

• CI/CD Integration — CI mode, retries and flaky detection, fail-fast, sharding across jobs, and a complete GitHub Actions workflow.
• CLI Reference — Every OrbitTest command and flag: run, init, ui, forge, devices, doctor, and clean-reports, with recommended npm scripts.
• Configuration — Every orbittest.config.js option explained: workers, retries, timeouts, browser display, CI behavior, and named environments.
• Troubleshooting — Fixes for unauthorized devices, missing devices, blocked Companion installs, black mirror screens, and empty log panels.