A load test that mostly measured itself
k6 setup() login requests landed in the same http_req_duration metric as the GETs under test, so bcrypt cost inflated p95 latency.
The safety net had a hole
The test, the audit log, the scanner, the CI run — covering less than I assumed it did.
Building the k6 setup, I added a setup() phase logging in 20 accounts and distributing tokens across VUs. The first real run looked wrong immediately: p95 of 229ms for simple GETs on a stack under no meaningful load.
Those login POSTs each do a real bcrypt compare — slow by design, that's the point — and they were landing in the same http_req_duration metric as the GETs I actually cared about. With few total requests in an early run, 20 slow logins drag the aggregate percentile well above real endpoint latency.
Tagged setup logins and test requests separately and scoped the thresholds to the test tag. Any load script with a non-trivial setup phase needs its own tags, because most tools bucket everything into one histogram by default, and a surprisingly bad first result is worth checking for this before blaming the system.