Skip to main content

Contributing Tests

This page describes when to add a platform test, how to graduate exploratory work into the suite, and what makes a good platform-level assertion.

When to Add a Platform Test​

Add a platform test when you want to protect a user-facing behaviour across the full integration stack — not when you want to test logic inside a single service.

Good candidates:

  • A new API endpoint on the BFF that exercises Ranger permissions
  • A workflow that spans multiple services (e.g. workspace provisioning → Trino access)
  • A known failure mode that was caught manually and should never regress

Not good candidates:

  • Logic that lives entirely inside one service (write a unit test there instead)
  • Infrastructure state (use Terraform outputs or Cluster Agent checks)
  • [Add other anti-patterns as the team discovers them]

The Notebook-to-Test Pipeline​

The notebooks/ directory in the platform-tests repo is the starting point for experimental work. Use a notebook to explore an API, figure out the right assertions, and verify behaviour manually.

Once the behaviour is understood and stable, graduate the notebook to a test file in tests/bff/ or tests/setup/ as appropriate.

A test is ready to graduate when:

  1. It can run non-interactively without manual steps
  2. Its assertions are deterministic — the same environment always produces the same result
  3. It cleans up after itself, or its side effects are idempotent

[Add any project-specific graduation checklist items here.]

What a Good Platform Test Asserts​

Platform tests should assert on outcomes, not implementation details.

Prefer:

  • response.status_code == 200
  • result["rows"] contains expected values
  • response.status_code == 403 when a restricted user attempts a forbidden action

Avoid:

  • Asserting on internal service logs or database state
  • Asserting on response times (use load tests for that)
  • Asserting on field names or response shapes that are likely to change — [unless they are part of the public API contract]

Marking a Test as Pending​

If a capability should be tested but isn't ready yet, add a stub file in tests/bff/ with a comment explaining what is blocking it. Also update the coverage map in Our Approach.

This keeps the gap visible without silently skipping work.

Go Deeper​