fix(tests/stress): eliminate 0-request flakes in locust runs

Three independent issues conspired to make stress tests record 0 requests:

1. Every virtual user did /auth/login in on_start. With 1000 users in a
   spike window, bcrypt-bound logins never finished and on_start failed
   for all users — aggregated requests stayed at 0. Pre-fetch a single
   admin token in the fixture (cached per-host) and pass it via
   DECNET_STRESS_TOKEN so locust users skip the login storm.

2. Locust exits non-zero on any request failure by default, causing
   run_locust to throw away an otherwise valid stats CSV. Pass
   --exit-code-on-error 0 so per-test assertions are the only fail gate.

3. test_stress_sustained ran two locust subprocesses against the same
   uvicorn. Phase 1's keep-alive connections wedged phase 2 into 0
   recorded requests ~2/3 of the time. Refactored stress_server into a
   start_stress_server() context manager and gave each phase its own
   uvicorn.

Stable 3/3 on full suite, 3/3 on test_stress_sustained alone.
This commit is contained in:
2026-04-28 13:01:11 -04:00
parent 681931d9bb
commit cc6abf7256
3 changed files with 124 additions and 47 deletions

View File

@@ -60,20 +60,21 @@ class DecnetUser(HttpUser):
raise RuntimeError(f"Login failed after {_MAX_LOGIN_RETRIES} retries (last status: {resp.status_code})")
def on_start(self):
token, must_change = self._login_with_retry()
# Only pay the change-password + re-login cost on the very first run
# against a fresh DB. Every run after that, must_change_password is
# already False — skip it or the login path becomes a bcrypt storm.
if must_change:
self.client.post(
"/api/v1/auth/change-password",
json={"old_password": ADMIN_PASS, "new_password": ADMIN_PASS},
headers={"Authorization": f"Bearer {token}"},
)
token, _ = self._login_with_retry()
self.token = token
# Prefer the fixture-supplied token: 1000 simultaneous bcrypt logins
# never finish inside a spike window, leaving aggregated requests at 0.
preset = os.environ.get("DECNET_STRESS_TOKEN")
if preset:
self.token = preset
else:
token, must_change = self._login_with_retry()
if must_change:
self.client.post(
"/api/v1/auth/change-password",
json={"old_password": ADMIN_PASS, "new_password": ADMIN_PASS},
headers={"Authorization": f"Bearer {token}"},
)
token, _ = self._login_with_retry()
self.token = token
self.client.headers.update({"Authorization": f"Bearer {self.token}"})
# --- Read-hot paths (high weight) ---