Pro feature

Rank tests by real failure history

Out of the box, Testward already tells you which tests cover a change. Connect your CI and it will also rank them by how often they actually fail — so you run the flaky, fail-prone ones first.

How it works

After your tests run in CI, one step uploads the JUnit results to Testward. There's no API key to manage — the upload is authenticated by your workflow's GitHub Actions OIDC token, which GitHub signs and we verify. Testward stores pass/fail outcomes per test and uses them to rank the tests relevant to each PR.

Setup (one workflow step)

Make your test job emit JUnit XML (almost every framework can — --reporters=junit, pytest --junitxml, go test … | go-junit-report, etc.), then add this step. The two highlighted lines are all that's specific to Testward:

# In your CI workflow (.github/workflows/ci.yml)
jobs:
  test:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      id-token: write        # ← lets the job request an OIDC token
    steps:
      - uses: actions/checkout@v4
      # ... install deps, then run tests producing junit.xml ...
      - name: Run tests
        run: npm test -- --reporters=default --reporters=jest-junit
        # writes ./junit.xml

      - name: Upload results to Testward
        if: always()         # ← upload even when tests fail
        run: |
          TOKEN=$(curl -sH "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
            "$ACTIONS_ID_TOKEN_REQUEST_URL&audience=diffsense" | jq -r '.value')
          curl -sf -X POST https://diffsense-api.fly.dev/api/test-results \
            -H "Authorization: Bearer $TOKEN" \
            -H "Content-Type: application/xml" \
            --data-binary @junit.xml

That's it. After a few CI runs, Testward will have enough history to start ranking. The "Tests that cover this change" section in your PR comments becomes "Run these tests first," annotated with each test's failure rate.

Notes

• Requires a Pro or Team plan on the repo's owner. Uploads from Free repos are rejected.
• For the best file-level mapping, use a JUnit reporter that includes the file attribute (jest-junit, pytest, vitest, and most modern reporters do).
• We store pass/fail/skip outcomes and timing — not your test code or source.
• History older than 90 days is pruned automatically.

← Back to Testward · vs other reviewers