Files
solelog/docs/superpowers/specs/2026-07-22-per-domain-test-safety-net-design.md
2026-07-22 11:12:13 +02:00

6.8 KiB

Design — Per-domain test safety net

Date: 2026-07-22 Status: Approved (brainstorming) → ready for implementation plan Goal: Establish a per-domain test safety net across all three workspaces so future changes can't silently break existing behavior, and shake out latent bugs in the process.

Decisions (from brainstorming)

  1. Deliverable = full safety net. Coverage tooling + per-domain gap-fill (edge / error / authz / regression cases) + one full-lifecycle integration test per domain + a living "guaranteed behaviors" checklist doc.
  2. Scope = all three layers, equal depth. apps/api, apps/worker, apps/admin.
  3. Bug handling = log in Plane + fix trivial inline. Every finding becomes a Plane work item; small/safe fixes land inline in the same pass; risky / behavior-changing ones get their own ticket and are deferred (flagged as such).
  4. No hard coverage threshold gate now — measure and report; revisit a CI gate later.

Current baseline (2026-07-22)

Workspace Test files Tests Status
apps/api 15 103 ✅ green
apps/worker 8 30 ✅ green
apps/admin 11 50 ✅ green
  • No @vitest/coverage-v8 in any workspace — untested paths are currently guesswork.
  • API harness: real file DB (file:./.tmp/test.db), migrated once per run, serial (fileParallelism: false). Helpers in apps/api/test/helpers.ts: createTestUser, authToken, bearer, seedActivity. The DB is reset once per run, not per file, so domain state accumulates across files — this already caused a "test bleed" bug (tests had to scope queries to avoid each other's rows). This fragility is fixed as part of this work.
  • Client harness: jsdom + Testing Library, per-workspace src/test/setup.ts.

Work

1. Coverage tooling (first — turns guesswork into a measured gap list)

  • Add @vitest/coverage-v8 (dev) + a test:coverage script to all three workspaces.
  • Config per workspace: include src/**; exclude tests, config files, and generated / non-logic files (src/db/schema.ts, *.d.ts, main.tsx, vite-env.d.ts).
  • Record a baseline coverage % per domain so every gap filled is measured, not guessed.
  • No hard threshold gate now (arbitrary + flaky). Measure, report, revisit later.

2. Fix the API harness fragility (prerequisite for reliable domain isolation)

  • Add a resetDb() helper (truncate work_sessions, activities, and the better-auth tables session, account, user, verification) usable in beforeEach for domain suites, so each domain's tests are isolated instead of inheriting accumulated state.
  • Migration still runs once per run (top-level, before the DB file is opened/locked — the existing Windows EBUSY constraint in setup.ts stays respected).
  • Domain suites that adopt resetDb() can then assert exact counts without dodging other suites' rows. Existing suites are migrated to it where it removes a scoping workaround.

3. Per-domain audit + fill (equal depth across all three layers)

For each domain: enumerate guaranteed behaviors (happy path + edges) → diff against existing tests + coverage → add missing edge / error / authz / regression tests → add one full-lifecycle integration test (§4) → record the domain's behavior checklist (§6).

API (7 domains):

  • Auth & /me — sign-in/out, bearer token, /api/me role field, CORS.
  • Activities — CRUD, admin-gated writes, reorder, insole_types subset handling.
  • Work-sessions (worker) — start / pause / resume / stop / discard, active list, history, worked-time-excludes-paused, per-user scoping (a worker never sees another's).
  • Worker export / CSV — /api/export, buildSessionsCsv byte-shape.
  • Admin sessions — cross-user views, active view, manual create/edit, stop/discard, users roster.
  • Reports + admin export — /api/admin/report aggregation & filters; /api/admin/export all-users CSV.
  • Admin users — create / set role / reset password / deactivate / reactivate + lockout guards (no self-deactivate/demote, last-active-admin invariant, banned-admin gate).

Worker (5 domains): Auth (context / storage / login) · Stopwatch lifecycle + server reconcile · History · Account / logout · lib (api/ApiError, stopwatch math).

Admin (6 domains): Auth (admin-only gate) · Live view · Sessions + SessionForm · Reports + date-range · Users · Activities.

4. Lifecycle integration tests (where interaction bugs hide)

One per domain, walking the real workflow. Anchor examples:

  • Worker session (API): seed activity → start → pause → resume → stop → assert worked-time excludes paused → appears in history → CSV row correct.
  • Admin ↔ worker (API): worker starts → admin stops it → session lands in report + export with correct totals.
  • User lifecycle (API): admin creates worker → worker signs in → admin deactivates → sign-in blocked and live bearer token revoked → reactivate → sign-in works again.
  • Stopwatch reconcile (worker): running locally → admin stops server-side → next poll clears the stuck timer (the reconcile path in Stopwatch.tsx).

5. Bugs → Plane

  • New parent work item: "Testing & hardening — per-domain safety net" (project SoleLog, id 1956d3d3-15ed-4850-93d8-96d5fb17f213).
  • One child per domain (audit+fill task), plus one child per bug found.
  • Trivial / safe fixes land inline in the same pass; risky / behavior-changing fixes get their own ticket and stay deferred, flagged explicitly.

6. Deliverables

  • Coverage wired in all three workspaces; baseline recorded; all suites green.
  • Gap tests + one lifecycle test per domain.
  • docs/ doc: per-domain "guaranteed behaviors" checklist — the living definition-of-done future work checks against.
  • Plane epic + per-domain tasks + logged bugs.

7. Execution vehicle

Large, parallelizable sweep → the implementation plan is structured to run as a Workflow: fan out one agent per domain (audit + fill), verify, then synthesize — rather than a serial slog. Bugs found during a domain's pass are reported structurally and triaged per §5.

Out of scope

  • A CI coverage-threshold gate (deferred; measure first).
  • E2E browser automation (Playwright/Cypress) — the client integration tests stay at the Testing-Library level.
  • Load/performance testing.
  • Any Phase 5 deploy work (hosting is already handled on the maintainer's machine).

Non-negotiables (not simplified away)

  • Authz / role-scoping paths get explicit tests (worker-can't-see-others, admin-only writes, lockout guards) — these are trust boundaries.
  • Error/409/401/403 paths are covered, not just happy paths.
  • No test asserts against another test's accumulated rows (harness isolation, §2).