Testsprite CLI

TestSprite CLI is the command-line client for TestSprite, a cloud testing platform that runs end-to-end tests against a live frontend or backend as a user would, rather than against mocks. The CLI is built for coding agents: an agent describes a behaviour to verify, the platform runs it in a real browser or against a real API, and any failure comes back as one self-consistent bundle the agent can act on. People drive the same commands from a terminal or CI.

testsprite setup stores and verifies an API key and installs a verification-loop skill into a coding agent such as Claude Code, Cursor, Cline, Windsurf, Antigravity, Codex, Kiro, or GitHub Copilot, so the agent knows how to create, run, and triage tests on its own. The intended loop is create → run → failure get → fix → rerun: when the agent changes code, it either creates a test for new behaviour or reruns existing tests, and every passing test is kept in a durable suite that grows with the project.

Tests are described in a plain-language JSON plan file with action and assertion steps and no browser code; test create --plan-template emits a schema-correct skeleton pinned to the installed CLI version, and test scaffold and test lint author and validate plans offline. test failure get returns the failing step, its neighbours, screenshots, DOM snapshots, the test source, a root-cause hypothesis, and a recommended fix target, all sharing a single snapshotId; the CLI refuses to stitch data from two different runs. test diff compares two runs by verdict, failure kind, per-step status flips, and code-version drift, and test flaky replays a test several times to report a stability score.

Runs support --wait, --output json, predictable exit codes, and a --dry-run that exercises the full code path with canned data. On GitHub Actions, waited runs annotate the PR checks tab with one ::error:: per failure and append a results table to the job summary; --report junit writes a JUnit XML sidecar. Further command groups manage projects and backend-test credentials, named cross-project test lists with per-project environment pins, cron-based schedules with pause and auto-pause states, and local agent skill installation. Node.js 20.19+, 22.13+, or 24+ is required.

Features

• Live-app testing: runs in the cloud against real browsers and real APIs, not mocks
• Agent setup: testsprite setup verifies the key and installs a skill for Claude, Codex, Cursor, Cline, Antigravity, Kiro, Windsurf, or Copilot
• Plain-language plans: JSON plan files of action and assertion steps, with --plan-template, scaffold, and lint for offline authoring
• Failure bundle: one snapshot-consistent package of failing step, screenshots, DOM, source, root-cause hypothesis, and fix target
• Verification loop: create, run, failure get, fix, rerun, with every pass banked into a durable suite
• Run diffing and flake checks: test diff compares runs; test flaky reports a stability score
• Scriptable output: stable --output json, documented exit codes, and an offline --dry-run
• CI annotations: PR check annotations and job-summary tables on GitHub Actions, plus JUnit and summary-file outputs
• Test lists and schedules: cross-project collections with environment pins and 5-field cron schedules
• Project and auth management: project CRUD, static credentials, and auto-refresh login for backend tests
• Diagnostics: doctor checks CLI and Node versions, profile, endpoint, credentials, connectivity, and agent skill