Our order API had a healthy-looking suite: about 1,400 tests, 87 percent line coverage, green on every merge. In one quarter it still let through three production incidents. A field rename broke the mobile app, a new report endpoint hit a 40-second p99 at month-end traffic, and a researcher showed that changing an order id in the URL returned another customer's order.

None of these were gaps in the number of tests. They were gaps in the kinds of tests. Almost all of ours were functional tests that call an endpoint with valid input and check the response. This post is the map we built afterwards: each API test type, the specific failures it catches, what it misses, and a small example. It deliberately skips how to balance a whole test portfolio; the focus is on what each type is for.

The map

Test type Question it answers Catches Misses
Functional Does the endpoint do what the story says? Logic errors, wrong status codes Consumer expectations, scale, abuse
Contract Do provider and consumers still agree on the shape? Renamed or removed fields, type changes, breaking enums Business logic, performance
Schema-based fuzzing Does the API honour its own spec for all inputs? 500s on edge inputs, spec drift, missing validation Logic that is valid per schema but wrong
Integration Does it work with the real database, queue and third parties? Migrations, transactions, serialization, timeouts Load behaviour, consumers
Load and performance Does it hold latency and error targets under realistic traffic? Slow queries, pool exhaustion, lock contention Correctness of results
Security Can a caller do or see what they should not? Broken object-level authorization, auth bypass, mass assignment, injection Anything not modelled as an attack

Functional tests: the baseline

Functional tests are the ones most teams already have: send a request, assert on status, body and side effects. They are fast and specific. Their blind spot is that the same team writes the endpoint and the test, so they encode the provider's assumptions. If we rename total to totalAmount and update the test in the same pull request, everything stays green.

A useful upgrade is negative cases: missing fields, wrong types, values at boundaries, unknown enum values. Most of our 500s in production came from inputs nobody wrote a test for.

Contract tests: catching breaking changes for consumers

Contract testing turns consumer expectations into something the provider must pass. With consumer-driven contracts (Pact is the best-known tool), each consumer writes a test against a mock provider; the interactions are saved as a pact file, and the provider's CI replays them against the real service.

A consumer test in JavaScript with Pact JS:

import { PactV4, MatchersV3 } from "@pact-foundation/pact";
const { like, integer } = MatchersV3;

const provider = new PactV4({ consumer: "mobile-app", provider: "order-api" });

test("gets an order", () => {
  return provider
    .addInteraction()
    .given("order 42 exists for user 7")
    .uponReceiving("a request for order 42")
    .withRequest("GET", "/orders/42", (b) => b.headers({ Authorization: like("Bearer token") }))
    .willRespondWith(200, (b) =>
      b.jsonBody({ id: integer(42), status: like("paid"), total: like(19.99) })
    )
    .executeTest(async (mock) => {
      const order = await fetchOrder(mock.url, 42);
      expect(order.total).toBe(19.99);
    });
});

On the provider side, a verification job fetches pacts from the broker and fails the build if total disappears. The rename incident would have been blocked before merge. The cost is process: consumers must publish contracts, and provider states (order 42 exists for user 7) need setup code.

If your consumers are external and will not write pacts, provider-driven contracts are the alternative: publish an OpenAPI document as the contract and diff each change against the previous version in CI with a breaking-change detector such as oasdiff.

Schema-based fuzzing: let the spec generate the tests

If you have an OpenAPI document, a property-based tool can generate thousands of requests from it, including edge cases humans skip: empty strings, huge integers, unicode, missing optional fields, wrong content types. Schemathesis is the open-source tool we use:

schemathesis run https://staging.example.com/openapi.json \
  --header "Authorization: Bearer $STAGING_TOKEN" \
  --checks all \
  --max-examples 200

It flags server errors, responses that do not match the documented schema, and undocumented status codes. On its first run against our API it found 31 distinct 500s and 12 places where the implementation and the spec disagreed. That second category matters because generated clients and contract tooling trust the spec.

Run it against a disposable environment: it creates data and will happily call DELETE endpoints.

Integration tests: the real dependencies

Integration tests run the API against real infrastructure: the actual database engine and version, the message broker, and either real third-party sandboxes or faithful fakes. With Testcontainers, the database in the test is the same engine as production, started per test run:

import { PostgreSqlContainer } from "@testcontainers/postgresql";

let container;
beforeAll(async () => {
  container = await new PostgreSqlContainer("postgres:16").start();
  process.env.DATABASE_URL = container.getConnectionUri();
  await runMigrations();
}, 60_000);

afterAll(async () => container.stop());

These tests catch what mocks hide: a migration that fails on real data types, a transaction isolation assumption, JSON serialization of decimals, a queue message that does not match the consumer's schema, and timeouts against a slow sandbox. They are slower, so we run them on merge rather than on every commit, and keep them focused on boundaries rather than re-testing business rules.

Load and performance tests: behaviour under traffic

A functional test proves an endpoint is correct for one request. Load tests prove it stays within latency and error targets when many requests arrive together. We use k6, with thresholds that fail the run:

import http from "k6/http";
import { check } from "k6";

export const options = {
  scenarios: {
    month_end: {
      executor: "ramping-arrival-rate",
      startRate: 20, timeUnit: "1s",
      preAllocatedVUs: 100, maxVUs: 400,
      stages: [
        { target: 150, duration: "3m" },
        { target: 150, duration: "10m" },
      ],
    },
  },
  thresholds: {
    http_req_failed: ["rate<0.01"],
    "http_req_duration{endpoint:report}": ["p(99)<2000"],
  },
};

export default function () {
  const res = http.get("https://staging.example.com/reports/monthly?account=1001", {
    headers: { Authorization: `Bearer ${__ENV.TOKEN}` },
    tags: { endpoint: "report" },
  });
  check(res, { "status 200": (r) => r.status === 200 });
}

The arrival-rate executor matters: it keeps sending requests at the target rate even when responses slow down, which is how real traffic behaves. A fixed number of virtual users waiting on each response hides overload, because the load drops as the system slows. Our report endpoint passed a 50-user test and failed this one at 90 requests per second with connection pool exhaustion.

Load tests need production-like data volumes. Ours ran against a seeded database with 2 percent of production rows, which is why the slow query never showed up before the incident.

Security tests: what an attacker would try

The OWASP API Security Top 10 (2023 edition) is the most practical checklist for API-specific risks. The top item, Broken Object Level Authorization (BOLA), is exactly our incident: the endpoint checked that the caller was logged in, not that the order belonged to them.

BOLA is easy to test once you think of it, and it is never caught by functional tests written from the owner's point of view:

test("user cannot read another user's order", async () => {
  const alice = await login("alice@example.com");
  const bob = await login("bob@example.com");
  const order = await createOrder(alice);

  const res = await api.get(`/orders/${order.id}`, { token: bob });
  expect(res.status).toBe(404); // not 403, to avoid confirming the id exists
});

We now generate this test for every route with an id in the path. Other security checks in our suite:

  • Broken authentication: expired, malformed and wrong-audience tokens are rejected with 401.
  • Broken object property level authorization (mass assignment): a PATCH with role or price in the body does not change them for non-admin callers.
  • Unrestricted resource consumption: page size, upload size and query complexity limits are enforced.
  • Injection: schema fuzzing plus a DAST scan with OWASP ZAP's API scan against staging each night.

Where each type runs

Stage Types
Every commit Functional, consumer contract tests, BOLA and auth checks
Every merge Provider contract verification, integration with Testcontainers, OpenAPI breaking-change diff
Nightly Schemathesis, ZAP API scan
Before releases with data or query changes k6 load scenarios against production-sized data

Key takeaways

  • Count test types, not tests: each type catches failures the others cannot.
  • Contract tests (consumer-driven or OpenAPI diffs) stop breaking changes that functional tests happily update along with the code.
  • Schema-based fuzzing finds 500s and spec drift you would never write by hand.
  • Integration tests with real dependencies catch migration, serialization and timeout bugs that mocks hide.
  • Load tests need open-model arrival rates and production-sized data to mean anything.
  • Test authorization from the attacker's seat: every id in a path deserves a cross-user test.

References