docs(spec): per-domain test safety net design
This commit is contained in:
Bas van Rossem
@@ -0,0 +1,129 @@
|
||||
# Design — Per-domain test safety net
|
||||
|
||||
**Date:** 2026-07-22
|
||||
**Status:** Approved (brainstorming) → ready for implementation plan
|
||||
**Goal:** Establish a per-domain test safety net across all three workspaces so future
|
||||
changes can't silently break existing behavior, and shake out latent bugs in the process.
|
||||
|
||||
## Decisions (from brainstorming)
|
||||
|
||||
1. **Deliverable = full safety net.** Coverage tooling + per-domain gap-fill (edge / error /
|
||||
authz / regression cases) + one full-lifecycle integration test per domain + a living
|
||||
"guaranteed behaviors" checklist doc.
|
||||
2. **Scope = all three layers, equal depth.** `apps/api`, `apps/worker`, `apps/admin`.
|
||||
3. **Bug handling = log in Plane + fix trivial inline.** Every finding becomes a Plane work
|
||||
item; small/safe fixes land inline in the same pass; risky / behavior-changing ones get
|
||||
their own ticket and are deferred (flagged as such).
|
||||
4. **No hard coverage threshold gate** now — measure and report; revisit a CI gate later.
|
||||
|
||||
## Current baseline (2026-07-22)
|
||||
|
||||
| Workspace | Test files | Tests | Status |
|
||||
| ----------- | ---------- | ----- | ------- |
|
||||
| `apps/api` | 15 | 103 | ✅ green |
|
||||
| `apps/worker` | 8 | 30 | ✅ green |
|
||||
| `apps/admin` | 11 | 50 | ✅ green |
|
||||
|
||||
- No `@vitest/coverage-v8` in any workspace — untested paths are currently guesswork.
|
||||
- **API harness:** real file DB (`file:./.tmp/test.db`), migrated once per run, serial
|
||||
(`fileParallelism: false`). Helpers in `apps/api/test/helpers.ts`: `createTestUser`,
|
||||
`authToken`, `bearer`, `seedActivity`. The DB is reset **once per run, not per file**, so
|
||||
domain state accumulates across files — this already caused a "test bleed" bug (tests had
|
||||
to scope queries to avoid each other's rows). This fragility is fixed as part of this work.
|
||||
- **Client harness:** jsdom + Testing Library, per-workspace `src/test/setup.ts`.
|
||||
|
||||
## Work
|
||||
|
||||
### 1. Coverage tooling (first — turns guesswork into a measured gap list)
|
||||
|
||||
- Add `@vitest/coverage-v8` (dev) + a `test:coverage` script to all three workspaces.
|
||||
- Config per workspace: include `src/**`; exclude tests, config files, and generated /
|
||||
non-logic files (`src/db/schema.ts`, `*.d.ts`, `main.tsx`, `vite-env.d.ts`).
|
||||
- Record a **baseline coverage % per domain** so every gap filled is measured, not guessed.
|
||||
- **No hard threshold gate** now (arbitrary + flaky). Measure, report, revisit later.
|
||||
|
||||
### 2. Fix the API harness fragility (prerequisite for reliable domain isolation)
|
||||
|
||||
- Add a `resetDb()` helper (truncate `work_sessions`, `activities`, and the better-auth
|
||||
tables `session`, `account`, `user`, `verification`) usable in `beforeEach` for domain
|
||||
suites, so each domain's tests are isolated instead of inheriting accumulated state.
|
||||
- Migration still runs once per run (top-level, before the DB file is opened/locked — the
|
||||
existing Windows EBUSY constraint in `setup.ts` stays respected).
|
||||
- Domain suites that adopt `resetDb()` can then assert exact counts without dodging other
|
||||
suites' rows. Existing suites are migrated to it where it removes a scoping workaround.
|
||||
|
||||
### 3. Per-domain audit + fill (equal depth across all three layers)
|
||||
|
||||
For **each** domain: enumerate guaranteed behaviors (happy path + edges) → diff against
|
||||
existing tests + coverage → add missing **edge / error / authz / regression** tests → add
|
||||
**one full-lifecycle integration test** (§4) → record the domain's behavior checklist (§6).
|
||||
|
||||
**API (7 domains):**
|
||||
|
||||
- **Auth & `/me`** — sign-in/out, bearer token, `/api/me` role field, CORS.
|
||||
- **Activities** — CRUD, admin-gated writes, reorder, `insole_types` subset handling.
|
||||
- **Work-sessions (worker)** — start / pause / resume / stop / discard, active list,
|
||||
history, worked-time-excludes-paused, per-user scoping (a worker never sees another's).
|
||||
- **Worker export / CSV** — `/api/export`, `buildSessionsCsv` byte-shape.
|
||||
- **Admin sessions** — cross-user views, active view, manual create/edit, stop/discard,
|
||||
users roster.
|
||||
- **Reports + admin export** — `/api/admin/report` aggregation & filters; `/api/admin/export`
|
||||
all-users CSV.
|
||||
- **Admin users** — create / set role / reset password / deactivate / reactivate + lockout
|
||||
guards (no self-deactivate/demote, last-active-admin invariant, banned-admin gate).
|
||||
|
||||
**Worker (5 domains):** Auth (context / storage / login) · Stopwatch lifecycle + server
|
||||
reconcile · History · Account / logout · lib (`api`/`ApiError`, `stopwatch` math).
|
||||
|
||||
**Admin (6 domains):** Auth (admin-only gate) · Live view · Sessions + SessionForm ·
|
||||
Reports + date-range · Users · Activities.
|
||||
|
||||
### 4. Lifecycle integration tests (where interaction bugs hide)
|
||||
|
||||
One per domain, walking the real workflow. Anchor examples:
|
||||
|
||||
- **Worker session (API):** seed activity → start → pause → resume → stop → assert
|
||||
worked-time excludes paused → appears in history → CSV row correct.
|
||||
- **Admin ↔ worker (API):** worker starts → admin stops it → session lands in report +
|
||||
export with correct totals.
|
||||
- **User lifecycle (API):** admin creates worker → worker signs in → admin deactivates →
|
||||
sign-in blocked **and** live bearer token revoked → reactivate → sign-in works again.
|
||||
- **Stopwatch reconcile (worker):** running locally → admin stops server-side → next poll
|
||||
clears the stuck timer (the reconcile path in `Stopwatch.tsx`).
|
||||
|
||||
### 5. Bugs → Plane
|
||||
|
||||
- New parent work item: **"Testing & hardening — per-domain safety net"** (project SoleLog,
|
||||
id `1956d3d3-15ed-4850-93d8-96d5fb17f213`).
|
||||
- One child per domain (audit+fill task), plus one child per bug found.
|
||||
- Trivial / safe fixes land inline in the same pass; risky / behavior-changing fixes get
|
||||
their own ticket and stay deferred, flagged explicitly.
|
||||
|
||||
### 6. Deliverables
|
||||
|
||||
- Coverage wired in all three workspaces; baseline recorded; all suites green.
|
||||
- Gap tests + one lifecycle test per domain.
|
||||
- `docs/` doc: **per-domain "guaranteed behaviors" checklist** — the living
|
||||
definition-of-done future work checks against.
|
||||
- Plane epic + per-domain tasks + logged bugs.
|
||||
|
||||
### 7. Execution vehicle
|
||||
|
||||
Large, parallelizable sweep → the implementation plan is structured to **run as a Workflow**:
|
||||
fan out one agent per domain (audit + fill), verify, then synthesize — rather than a serial
|
||||
slog. Bugs found during a domain's pass are reported structurally and triaged per §5.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- A CI coverage-threshold gate (deferred; measure first).
|
||||
- E2E browser automation (Playwright/Cypress) — the client integration tests stay at the
|
||||
Testing-Library level.
|
||||
- Load/performance testing.
|
||||
- Any Phase 5 deploy work (hosting is already handled on the maintainer's machine).
|
||||
|
||||
## Non-negotiables (not simplified away)
|
||||
|
||||
- Authz / role-scoping paths get explicit tests (worker-can't-see-others, admin-only writes,
|
||||
lockout guards) — these are trust boundaries.
|
||||
- Error/409/401/403 paths are covered, not just happy paths.
|
||||
- No test asserts against another test's accumulated rows (harness isolation, §2).
|
||||
Reference in New Issue
Block a user