In this post I’ll walk through how I designed a parallel-safe API test automation framework in Java using REST Assured, TestNG, and Extent Reports, running against the public restful-booker practice API. I’ll cover the architecture, the thread-safety decisions that make parallel execution boring (in a good way), a debugging story involving HTTP 418, and the “known-defect” strategy behind those deliberately failing tests.
What does the framework actually do?
It’s a full regression suite for a bookings REST API: authentication, create, read, update, delete, and bad-input handling — 36 tests in total. Every run produces a self-contained HTML report showing each request and response, plus a separate bug report documenting six real defects found in the API, each mapped to the exact test that reproduces it.
The stack is intentionally boring: Java 17, Maven, REST Assured for HTTP, TestNG as the runner, Jackson for serialization, and JSON Schema validation for response contracts. Boring stacks are debuggable stacks.
How should you structure an API automation framework?
I split the code by what changes together:
- Test data builders construct every payload in one place — “a valid booking,” “a booking with a negative price.” Tests never build their own JSON.
- An API client exposes one method per endpoint. It owns how to call the API — URLs, headers, auth, retries — and returns the raw response.
- Shared specifications hold everything common to every request (base URL, headers, timeouts) and reusable response expectations.
- Tests own only the assertions: what a correct answer looks like.
The rule that makes this work: the client never asserts. Different tests have opposite opinions about the same endpoint — a happy-path test expects 200, a validation test hitting the same method expects a 4xx. The moment assertions leak into the client, it has to know every caller’s expectations, and reuse dies. Keep the “given/when” in the client and the “then” in the test.
The payoff shows up on change day. When the API adds a field, you touch the model, the data builder, and the schema file — the tests barely move.
How do you make REST Assured safe for parallel execution?
This is where most frameworks quietly break. REST Assured offers convenient static globals — RestAssured.baseURI, RestAssured.config — and nearly every tutorial uses them. They’re also mutable shared state: any thread can write them while another thread reads. That’s a race condition waiting for the day someone flips on parallel="methods" in the TestNG suite XML.
Two decisions make my suite parallel-safe:
1. One immutable request specification. All shared configuration lives in a single RequestSpecification, built once in a static initializer and never modified again. An object that never changes after construction is safe to share across any number of threads with zero locking. Immutability is the cheapest thread-safety you can buy.
2. A ThreadLocal for report tracking. The one value that genuinely differs per test — “which report entry do this test’s HTTP logs belong to?” — lives in a ThreadLocal. A TestNG listener sets it when a test starts; a REST Assured filter reads it to log every request and response into the right entry; the listener removes it when the test ends. That remove() call matters more than people realize: Maven Surefire reuses pooled threads, so a stale value can leak into the next test scheduled on the same thread.
The result: the same 36 tests run on 2 threads in ~48 seconds instead of ~100, with identical results and zero cross-contaminated logs. I verified it with timestamps, not vibes.
The bug that pretended to be rate limiting
Early runs failed intermittently with HTTP 418 — “I’m a Teapot.” My first theory was rate limiting. Retries didn’t help, and the failures weren’t load-correlated, so I stopped guessing and diffed the exact request REST Assured sent against an identical curl that worked.
One difference. REST Assured’s .accept(ContentType.JSON) shortcut doesn’t send what you think — it sends:
Accept: application/json, application/javascript, text/javascript, text/jsonThis API’s bot filter flags the javascript entries as automated traffic and replies 418. Hand-writing a plain Accept: application/json header fixed it completely.
Two lessons worth stealing. First, when a failure looks random, compare the actual bytes on the wire against a known-good request before blaming the environment. Second, never paper over a symptom with retries until you’ve found the cause. I do keep a retry in the client — but it’s scoped to exactly two transient status codes (429 and 503), capped at three attempts, and wraps only the HTTP call, never the assertions. A blanket retry policy is how real regressions get buried; Martin Fowler’s essay on non-deterministic tests is still the best writing on why.
Why do 16 tests fail on purpose?
The target API has real bugs: it accepts negative prices, accepts checkout dates before checkin, returns 200 OK for wrong credentials, and crashes with a 500 on missing fields. When your tests find bugs like these, you have three options:
- Assert the buggy behavior so the suite goes green. Worst option — green now means nothing.
- Skip the tests. The signal disappears, and nobody ever re-checks.
- Assert what a correct API should do, and let the tests fail.
I chose the third. Each failing test carries a known-defect tag, a comment naming its bug ID, and an entry in the bug report with runnable reproduction steps. The failure is the bug report, re-confirmed on every run — and the design self-heals: the day a bug is fixed, its test flips green with zero code changes.
The exact failure count becomes a contract: 16 failures mapping one-to-one onto documented bugs. A seventeenth failure is instantly visible as a real regression. In CI, you’d enforce this by splitting into two jobs — everything except the known-defect group as the merge gate (must be 100% green), and the known-defect group as an informational run that alerts when anything changes.
Assertion precision: how strict is too strict?
One design rule I’d defend in any code review: assert as precisely as the requirement is precise — and no more. For “reject bad input” tests, my response spec accepts any 4xx, because whether an API returns 400 or 422 is a design choice, and pinning one makes tests fail when the API is behaving correctly. But where the correct code is unambiguous — 404 for a missing resource, per RFC 9110’s status code semantics — I pin it exactly.
The same philosophy applies to coverage. Two query-string filters on this API behaved in ways that matched no reasonable reading of the docs, so I left them untested and documented the gap. A test that asserts a guess is worse than no test: it produces either false failures or false confidence.
FAQ
Q: Why use both JSON Schema validation and field-level assertions? They catch different failures. Schema validation checks the contract — fields, types, structure — and catches drift like a renamed field. Field assertions check values — did the API echo back what I sent. Happy-path responses get both.
Q: Is ThreadLocal really necessary for Extent Reports in parallel runs?
Yes. With parallel="methods", two tests log HTTP traffic simultaneously through shared filters. Without per-thread tracking, one test’s requests land in another test’s report entry — and always call remove() when the test ends, because runners reuse threads.
Q: Should generated test reports be committed to git? Generally no — publish them as CI artifacts or to GitHub Pages instead. Repos should hold source, not build output.
Key takeaways
- Separate how to call (client) from what correct looks like (tests).
- Prefer one immutable shared spec over REST Assured’s mutable static globals.
ThreadLocal+ listener + filter is the pattern for parallel-safe reporting.- Diff the wire-level request against
curlbefore calling anything “flaky.” - Retry only diagnosed transient failures — narrowly, with a cap.
- Tests that fail on purpose, mapped to documented bugs, turn a red build into a tracked contract.
If you want to try any of this, restful-booker is free, intentionally buggy, and perfect for it. The bugs you find are the feature.
