Find the flaky tests in your CI
Upload the JUnit XML of each test run. See which tests flake, which failures are new, and which tests are slow.
- 1 curlin your CI sends the JUnit XML of a run
- 2 kindsof failure: new, and known flakes
- 200commits kept for each branch
- 20repos for each person, free
What you get
Flaky tests, ranked
Every test that both passed and failed on a commit, with its flake rate and when it last flaked.
New failures first
For each commit, the tests that failed and were never flaky, apart from the known flakes. Review the first list.
A comment for the pull request
The upload answers with ready Markdown. One line in your workflow posts it.
Read more in the docsThe slowest tests
The tests with the highest mean duration over the last 20 commits.
History of each test
A bar for each of the last 30 commits: passed, failed, flaky or skipped.
Any runner
Vitest, Playwright, Jest, pytest, Maven and others write JUnit XML. Tests reads it in your upload, with no install.
Read more in the docs
Upload in 3 steps
- Sign in, add your repo (an owner and a name, such as
acme/widgets) and make an upload token. The token starts withtst_and is shown once. Save it as a secret of your CI namedTESTS_TOKEN. - Make the report with your test runner: JUnit XML.
- Send it after your tests, even when they fail, with one curl.
gzip -c junit.xml | curl --fail-with-body \
-X POST "https://tests.utilarium.app/api/v1/upload?commit=$(git rev-parse HEAD)&branch=main&flag=unit" \
-H "Authorization: Bearer $TESTS_TOKEN" \
-H "Content-Type: application/xml" \
-H "Content-Encoding: gzip" \
--data-binary @-# .github/workflows/tests.yml
name: tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
- run: npm ci
- run: npx vitest run --reporter=default --reporter=junit --outputFile.junit=junit.xml
continue-on-error: true
- name: Upload test results
run: |
gzip -c junit.xml | curl --fail-with-body -X POST \
"https://tests.utilarium.app/api/v1/upload?commit=${{ github.event.pull_request.head.sha || github.sha }}&branch=${{ github.head_ref || github.ref_name }}&run=${{ github.run_id }}-${{ github.run_attempt }}&flag=vitest" \
-H "Authorization: Bearer ${{ secrets.TESTS_TOKEN }}" \
-H "Content-Encoding: gzip" \
--data-binary @- | tee upload.json
env:
TESTS_TOKEN: ${{ secrets.TESTS_TOKEN }}A second upload for the same commit, flag and run replaces the first, so a retried job is safe. The docs show Playwright and the pull request comment.
See your first flaky test in minutes
Add a repo, make a token and paste one curl into your CI. A test that passes after a retry shows up in the Flaky list.