test-report-2026-05-17.md — Run summary for the 2026-05-17 session: 64 passed, 3 flaky, 2 skipped, 61 failed (all in pre-existing specs with legacy selectors). Includes per-spec results, root causes for every failure category, and recommended fixes. e2e-test-framework.md — Developer reference covering stack, directory layout, playwright.config.ts rationale (workers: 2, why bcrypt floods the server), shared helpers, selector conventions (PO form has no htmlFor bindings), mobile viewport pattern, and future improvements including auth state sharing. e2e-test-plan.md — Feature coverage matrix mapping all 21 user story groups to their spec files and roles, individual test case tables, regression trigger checklist by code area, gap analysis, and planned CI configuration. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
309 lines
9.8 KiB
Markdown
309 lines
9.8 KiB
Markdown
# PPMS — E2E Test Framework Reference
|
|
|
|
This document describes the Playwright-based end-to-end test framework for the
|
|
PPMS portal: its stack, directory layout, configuration, shared utilities, and
|
|
the conventions every spec must follow.
|
|
|
|
---
|
|
|
|
## Stack
|
|
|
|
| Layer | Tool | Version |
|
|
|---|---|---|
|
|
| Test runner | `@playwright/test` | 1.60 |
|
|
| Browser | Chromium (headless) | bundled with Playwright |
|
|
| Language | TypeScript | inherits from app `tsconfig.json` |
|
|
| Package manager | pnpm | same as portal app |
|
|
| App server | Next.js 15 dev server (`pnpm dev`) | auto-started by Playwright config |
|
|
|
|
---
|
|
|
|
## Directory Layout
|
|
|
|
```
|
|
App/pelagia-portal/
|
|
├── playwright.config.ts # Root config — workers, retries, baseURL, webServer
|
|
└── tests/
|
|
├── e2e/
|
|
│ ├── helpers/
|
|
│ │ ├── login.ts # Shared login(), createDraftPo(), submitPo(), USERS
|
|
│ │ └── auth.js # Legacy plain-JS login helper (pre-existing)
|
|
│ ├── dashboard/
|
|
│ │ └── po-status-badges.js
|
|
│ ├── inventory/
|
|
│ │ ├── items-tags.spec.ts
|
|
│ │ └── cart-icon.spec.ts
|
|
│ ├── mobile/
|
|
│ │ ├── desktop-required.spec.ts
|
|
│ │ ├── manager-approvals.spec.ts
|
|
│ │ ├── accounts-payments.spec.ts
|
|
│ │ └── bottom-nav.spec.ts
|
|
│ ├── admin-bordered-buttons.spec.ts
|
|
│ ├── approvals-edit-highlight.spec.ts
|
|
│ ├── export-gate.spec.ts
|
|
│ ├── notification-bell.spec.ts
|
|
│ ├── partial-receipt.spec.ts
|
|
│ ├── payment-history.spec.ts
|
|
│ ├── po-submit-button.spec.ts
|
|
│ ├── profile.spec.ts
|
|
│ ├── rebrand.spec.ts
|
|
│ └── vendor-auto-verify.spec.ts
|
|
├── integration/ # Vitest integration tests (separate suite)
|
|
└── unit/ # Vitest unit tests (separate suite)
|
|
```
|
|
|
|
---
|
|
|
|
## Configuration (`playwright.config.ts`)
|
|
|
|
```ts
|
|
export default defineConfig({
|
|
testDir: "./tests/e2e",
|
|
fullyParallel: true,
|
|
forbidOnly: !!process.env.CI,
|
|
retries: process.env.CI ? 2 : 1, // 1 local retry reduces flakiness from auth concurrency
|
|
workers: process.env.CI ? 1 : 2, // 2 local workers — more causes NextAuth bcrypt flooding
|
|
reporter: "html",
|
|
use: {
|
|
baseURL: "http://localhost:3000",
|
|
trace: "on-first-retry",
|
|
},
|
|
webServer: {
|
|
command: "pnpm dev",
|
|
url: "http://localhost:3000",
|
|
reuseExistingServer: !process.env.CI, // reuse running dev server locally
|
|
},
|
|
});
|
|
```
|
|
|
|
### Why workers: 2
|
|
|
|
The app uses NextAuth v5 with bcrypt password hashing for every login. Under high
|
|
parallelism (the default of ~50% CPU cores) all workers attempt to authenticate
|
|
simultaneously, overwhelming the dev server and causing login redirects to time out.
|
|
Two workers provide enough parallelism to keep the suite fast without triggering
|
|
the concurrency limit.
|
|
|
|
---
|
|
|
|
## Shared Helpers (`tests/e2e/helpers/login.ts`)
|
|
|
|
### `USERS` — seed credentials
|
|
|
|
```ts
|
|
export const USERS = {
|
|
TECH: { email: "tech@pelagia.local", password: "tech1234" },
|
|
MANNING: { email: "manning@pelagia.local", password: "manning1234" },
|
|
ACCOUNTS: { email: "accounts@pelagia.local", password: "accounts1234" },
|
|
MANAGER: { email: "manager@pelagia.local", password: "manager1234" },
|
|
SUPERUSER: { email: "superuser@pelagia.local", password: "super1234" },
|
|
AUDITOR: { email: "auditor@pelagia.local", password: "audit1234" },
|
|
ADMIN: { email: "admin@pelagia.local", password: "admin1234" },
|
|
};
|
|
```
|
|
|
|
### `login(page, creds)`
|
|
|
|
Navigates to `/login`, fills credentials, and waits up to **20 s** for the
|
|
redirect away from `/login`. The 20 s timeout is intentional — the bcrypt hash
|
|
check plus DB round-trip can exceed the Playwright default 5 s under any load.
|
|
|
|
```ts
|
|
await login(page, USERS.MANAGER);
|
|
```
|
|
|
|
### `createDraftPo(page, title)`
|
|
|
|
Creates a minimal PO as DRAFT and returns the absolute PO URL. Uses
|
|
**`name`-attribute selectors** because the PO form labels have no `htmlFor`/`id`
|
|
binding — `getByLabel()` will not resolve.
|
|
|
|
```ts
|
|
const poUrl = await createDraftPo(page, "Test PO - boiler parts");
|
|
```
|
|
|
|
### `submitPo(page, title)`
|
|
|
|
Same as `createDraftPo` but clicks the **Submit for Approval** button instead of
|
|
Save as Draft. Returns the PO URL after redirect.
|
|
|
|
---
|
|
|
|
## Selector Conventions
|
|
|
|
### Critical: PO form has no accessible label bindings
|
|
|
|
The new-PO form (`/po/new`) and the edit form use `<label>` elements that are
|
|
**visual only** — they have no `for` attribute and the inputs have no `id`.
|
|
`page.getByLabel()` will not find them.
|
|
|
|
**Always use name-attribute selectors for PO form fields:**
|
|
|
|
```ts
|
|
// CORRECT
|
|
page.locator('input[name="title"]')
|
|
page.locator('select[name="vesselId"]')
|
|
page.locator('select[name="accountId"]')
|
|
page.locator('input[name="projectCode"]')
|
|
|
|
// WRONG — will time out
|
|
page.getByLabel(/title/i)
|
|
page.getByLabel(/vessel/i)
|
|
```
|
|
|
|
### Role-badge selectors (profile page)
|
|
|
|
The user's role appears in both the desktop sidebar/header and the profile page.
|
|
`getByText("Technical")` will fail with a strict-mode violation. Scope to `<dd>`:
|
|
|
|
```ts
|
|
// CORRECT — scoped to the profile <dd> role badge
|
|
await expect(page.locator("dd span").filter({ hasText: "Technical" })).toBeVisible();
|
|
|
|
// WRONG — strict-mode violation (role appears in header too)
|
|
await expect(page.getByText("Technical")).toBeVisible();
|
|
```
|
|
|
|
### Mobile viewport
|
|
|
|
Mobile tests must set the viewport explicitly before `login()`:
|
|
|
|
```ts
|
|
const MOBILE_VIEWPORT = { width: 375, height: 812 };
|
|
|
|
test("...", async ({ page }) => {
|
|
await page.setViewportSize(MOBILE_VIEWPORT);
|
|
await login(page, USERS.MANAGER);
|
|
// ...
|
|
});
|
|
```
|
|
|
|
### Checking CSS classes (not computed styles)
|
|
|
|
For visual-only assertions (bordered buttons, colored badges), check the element's
|
|
`class` attribute rather than computed CSS, since Tailwind classes are the source
|
|
of truth:
|
|
|
|
```ts
|
|
const cls = await page.locator("button", { hasText: "Edit" }).first().getAttribute("class");
|
|
expect(cls).toMatch(/border/);
|
|
```
|
|
|
|
---
|
|
|
|
## Writing a New Spec
|
|
|
|
### File naming
|
|
|
|
- Feature specs: `tests/e2e/<section>/<feature>.spec.ts`
|
|
- Pre-`@playwright/test` scripts: `tests/e2e/<section>/<feature>.js` (legacy format)
|
|
|
|
New specs must use `@playwright/test` format (`import { test, expect } from "@playwright/test"`).
|
|
|
|
### Header comment
|
|
|
|
Every new spec file must open with a JSDoc block:
|
|
|
|
```ts
|
|
/**
|
|
* User stories covered: Feature N — <Name>
|
|
* - Story 1
|
|
* - Story 2
|
|
*
|
|
* Created: YYYY-MM-DD
|
|
*/
|
|
```
|
|
|
|
### Step logging
|
|
|
|
Log every meaningful assertion with a `✓` prefix so CI output is scannable:
|
|
|
|
```ts
|
|
console.log("✓ Export buttons hidden on DRAFT PO");
|
|
console.log(`✓ Logged in as ${creds.email}`);
|
|
```
|
|
|
|
### Graceful skips for seed-dependent tests
|
|
|
|
Tests that require specific seed data (e.g., an item with multiple vendors, or a
|
|
PO in a particular status) should skip rather than fail hard if the precondition
|
|
is absent:
|
|
|
|
```ts
|
|
const poRow = page.locator("[data-status='MGR_APPROVED']").first();
|
|
if ((await poRow.count()) === 0) {
|
|
test.skip(true, "No MGR_APPROVED PO in seed data");
|
|
return;
|
|
}
|
|
```
|
|
|
|
### HTTP-level assertions (no browser needed)
|
|
|
|
Use `request.get()` for API-level checks (status codes, headers) rather than
|
|
driving the full browser:
|
|
|
|
```ts
|
|
test("export returns 403 for DRAFT PO", async ({ request }) => {
|
|
// Must first obtain a session cookie — use page-based login or apiRequestContext
|
|
const resp = await request.get(`/api/po/${draftPoId}/export?format=pdf`);
|
|
expect(resp.status()).toBe(403);
|
|
});
|
|
```
|
|
|
|
---
|
|
|
|
## Running the Suite
|
|
|
|
```bash
|
|
cd App/pelagia-portal
|
|
|
|
pnpm test:e2e # headless, 2 workers
|
|
pnpm test:e2e:ui # Playwright interactive UI
|
|
pnpm test:e2e -- --headed # watch the browser
|
|
|
|
# Single file
|
|
pnpm test:e2e -- tests/e2e/mobile/bottom-nav.spec.ts
|
|
|
|
# Name filter
|
|
pnpm test:e2e -- --grep "Feature 20"
|
|
|
|
# All tests with trace on every run (debugging)
|
|
pnpm test:e2e -- --trace on
|
|
```
|
|
|
|
HTML report at `playwright-report/index.html` after every run.
|
|
|
|
---
|
|
|
|
## Known Gotchas
|
|
|
|
| Situation | Symptom | Fix |
|
|
|---|---|---|
|
|
| Many workers, all trying to log in at once | Login times out; page stays on `/login` | Keep `workers ≤ 2` locally; use `storageState` for auth |
|
|
| `getByLabel(/title/i)` on PO form | Locator times out — no `htmlFor` binding | Use `locator('input[name="title"]')` |
|
|
| `getByText("Technical")` on profile page | Strict-mode violation — appears in header AND profile | Scope to `page.locator("dd span").filter(...)` |
|
|
| Multi-role flow in one test (submit → approve → pay) | Flaky under 2 workers; competing for same user | Use `beforeAll` + a dedicated seed PO or run single-threaded |
|
|
| Viewport-dependent `md:hidden` elements | Element found at desktop viewport but not mobile, or vice versa | Always set viewport before login for mobile tests |
|
|
| `router.push` soft navigation | `waitForLoadState('networkidle')` still sees old URL | Use `page.waitForURL(pattern)` concurrently with the click/select |
|
|
|
|
---
|
|
|
|
## Future Improvements
|
|
|
|
1. **Auth state sharing** — Save one `storageState` per role in a global setup file.
|
|
This eliminates ~100 login round-trips and should cut suite time from 25 min to
|
|
under 5 min.
|
|
|
|
2. **Fix pre-existing specs** — Update `submitter-journey.spec.ts` and
|
|
`po-export.spec.ts` to use the shared helper's name-based selectors (FIX-1 in
|
|
test report).
|
|
|
|
3. **`data-testid` attributes** — Add sparse `data-testid` attributes to
|
|
ambiguous elements (unit price input, line-item rows) so specs don't depend on
|
|
implementation details like placeholder text or CSS class names.
|
|
|
|
4. **CI integration** — Run `pnpm test:e2e` in GitHub Actions on every PR.
|
|
Use `workers: 1` and `retries: 2` (already wired for `process.env.CI`).
|
|
|
|
5. **Visual regression** — Add Percy or Playwright's built-in screenshot comparison
|
|
for the status badge colors and mobile card layout.
|