Visual Automation

When you need pixels, not DOM

Some applications render everything onto a canvas: games, WebGL scenes, maps, charts, drawing tools, remote desktops. There are no buttons in the DOM to click and no text nodes to read, the entire UI is pixels. OrbitTest’s visual automation API exists for exactly these cases.

It is marked experimental: the API is stable, but edge-case coverage is still growing. Use it where the DOM genuinely cannot tell you what the user sees, and prefer normal locators everywhere else.

await orbit.open("https://www.pinthing.com/");

expect(await orbit.exists(orbit.css("canvas"))).toBe(true);

// Make rendering deterministic before visual checks
await orbit.evaluate(() => window.stopClock && window.stopClock());

const changed = await orbit.visual.changed(async () => {
  await orbit.evaluate(() => window.pinthing.down());
});

expect(changed).toBe(true);

Assert that an action visibly changed the canvas.

Coordinate actions

await orbit.mouse.click(680, 450);
await orbit.mouse.drag({ x: 680, y: 450 }, { x: 760, y: 360 });
await orbit.mouse.wheel(0, -500);

Raw mouse input at page coordinates: click, drag, scroll.

Coordinates are page pixels. Keep the browser window size consistent between runs (CI is headless and stable; local windows vary), and derive coordinates from known anchors when you can rather than hard-coding magic numbers everywhere.

Visual checks

// Wait until the screen stops changing
await orbit.visual.waitForStable();

// Save what the user sees
await orbit.visual.snapshot("reports/canvas-state.png");

// Check one pixel with tolerance
await orbit.visual.expectPixel({ x: 30, y: 30 }, "#ff0000", { tolerance: 5 });

// Assert an action causes visible change
await orbit.visual.expectChanged(async () => {
  await orbit.mouse.click(680, 450);
});

// Find and click by color
const point = await orbit.visual.findColor("#df1f1f", { tolerance: 70 });
await orbit.visual.clickColor("#df1f1f", { tolerance: 70 });

// Read a pixel programmatically
const pixel = await orbit.visual.pixel({ x: 100, y: 100 });

The visual toolkit: stability waits, snapshots, pixel assertions, color search.

Making visual tests reliable

Pixels are sensitive to everything: fonts, antialiasing, device pixel ratio, GPU drivers, browser versions, animation timing. Visual tests stay reliable when you control those variables:

Always waitForStable() before snapshots or pixel checks, never assert mid-animation.
Use tolerance on every color comparison. Exact-match pixel assertions break on the first antialiasing difference.
Freeze time-driven rendering in setup (the stopClock pattern above) so frames are deterministic.
Pin the environment in CI: same runner image, same browser version, same viewport.
Assert small regions or single anchor pixels rather than whole-screen comparisons; the smaller the assertion, the more precise the failure.

And keep a sense of proportion: if the DOM can answer the question (hasText, exists), use the DOM. Visual checks are the tool for the parts of the screen the DOM cannot describe, and excellent debugging evidence, snapshots show exactly what the user saw at the moment of failure.

Frequently asked questions

toMatchScreenshot() is the mobile visual-comparison assertion from the Android provider. The browser-side visual API on this page is finer-grained: pixels, colors, regions, and change detection.

My pixel check passes locally and fails in CI. Why?

Different rendering environments: fonts, DPR, GPU. Pin the CI image and browser version, raise the tolerance, and check a more stable anchor pixel, usually solid-color UI rather than text edges.

Can visual and DOM assertions mix in one test?

Yes, and they should. Use DOM assertions for everything the DOM knows, and visual checks only for the rendered-pixels part of the flow.

Working with Elements — Clicking, typing, reading text with three text readers, waiting for state, and iterating collections with orbit.all().
Reports & Diagnostics — HTML, JSON, and JUnit reports, trace timelines, Smart Report browser evidence, step mode, and report cleanup.
Locators — Target elements by visible text first, then by role, test ID, CSS, or XPath when needed, and which selectors to avoid.
Mobile Test API — The complete orbittest/mobile reference: app lifecycle, gestures, selectors, typing, waits, device state, and evidence capture.