AI Testing Standards

SSOT for agent self-testing procedure. The global CLAUDE.md (and AGENTS.md / GEMINI.md) point here instead of embedding this content inline. This is “how an AI agent verifies its own work” — distinct from GlobalDevRules §5 (human + AI dev-testing philosophy for the codebase). When both apply, follow both.

NEVER suggest the user test something. If testing is necessary, DO IT YOURSELF.

Implementation Standards (testing obligations)

When implementing, you have full authority and responsibility:

• Test thoroughly and autonomously — implementation is NOT complete until tested and verified working
• Modify all related files — you have authority to edit any files directly related to the project
• Verify end-to-end — run actual tests, not just syntax checks. Simulate user workflows when possible
• Report real results — only report success after actual verification, not theoretical correctness

Core Testing Principles

1. Autonomous Testing is Mandatory

   • YOU must test the implementation before reporting completion
   • Tests must verify actual functionality, not just syntax
   • Tests must simulate real user workflows
   • Tests must provide concrete evidence (output, metrics, screenshots)

2. Testing is NOT Optional

   • “I haven’t tested this” = Task is INCOMPLETE
   • “You should test this” = UNACCEPTABLE
   • “It should work” = NOT SUFFICIENT
   • “All tests pass (see results)” = REQUIRED

3. Use the Right Testing Tools

   • Web applications: Use WebFetch, curl, or Task agents to fetch URLs and verify content
   • APIs: Make actual HTTP requests, verify status codes and response data
   • CLIs: Run commands, capture output, verify results
   • Files: Read files, verify content matches expectations
   • Browsers: Use Playwright CLI (preferred) or Playwright MCP for visual/interactive testing. Chromium is installed at ~/.cache/ms-playwright/chromium-1217/chrome-linux64/chrome; Node library at /tmp/node_modules/playwright-core (reinstall after reboot with cd /tmp && npm install playwright-core).
   • SVG / brand assets: Use Playwright CLI screenshots to verify SVG rendering — pixel-level offsets, color rendering (CSS-driven vs hardcoded), and structural CSS issues are invisible to code review but immediately visible in a screenshot. Confirmed: caught 6 bugs in sd-leaf-dot.svg that code inspection alone would have missed.

Testing Levels (All Required)

Level 1: Syntax/Configuration ✅

• Code parses correctly
• Configuration files load
• No syntax errors
• NOT SUFFICIENT ALONE

Level 2: Unit/Component ✅

• Individual functions work
• Components behave correctly in isolation
• NOT SUFFICIENT ALONE

Level 3: Integration ✅

• Components work together
• Services start and communicate
• NOT SUFFICIENT ALONE

Level 4: End-to-End ✅ REQUIRED

• Complete user workflow from start to finish
• Actual tools/browsers/APIs accessed
• Real data verified
• This is the MINIMUM standard

Web Application Testing (Specific Requirements)

When implementing web applications or servers:

1. Start the application (don’t just check config)
2. Fetch the URL (HTTP request, not just port check)
3. Verify HTTP 200 (actual success, not just connection)
4. Check content (HTML/JSON contains expected data)
5. Measure performance (load time, response size)
6. Test from intended access point (localhost, LAN, etc.)
7. Verify in browser (if visual/interactive features matter)

Example (Good):

# Start server
proc = start_server()
time.sleep(5)

# Test URL
response = requests.get("http://localhost:8000/")
assert response.status_code == 200
assert "Expected Title" in response.text
assert response.elapsed.total_seconds() < 3.0

print(f"✓ Server responds: {response.status_code}")
print(f"✓ Content verified: {len(response.text)} bytes")
print(f"✓ Load time: {response.elapsed.total_seconds():.2f}s")

Example (Bad):

# Just check if port is open
sock.connect(("localhost", 8000))
print("Server is running")  # NOT SUFFICIENT!

8. Test BOTH the production build AND the dev server (cold-start). They take different code paths — the dev server’s dependency scanner / HMR pipeline can fail on files the production build compiles fine. Clear any dev cache (e.g. node_modules/.vite) before the dev run, or a previous successful scan will be silently reused and mask the failure. (Discovered 2026-07-08: MBR astro dev failed at cold scan on a pre-existing Astro 5.17.1 regression while build + preview + full Playwright suites all passed — the human hit the error on first pnpm dev. The cache also produced a false “fixed” result during bisection.)

9. Shut down every dev server you started — and verify it’s actually dead. Some dev servers detach as background daemons that outlive the test command and the session (Astro ≥7 runs astro dev this way). A leaked daemon holds its port and blocks the human’s own pnpm dev with “Another dev server is already running”. Stop it with the tool’s own stop command (astro dev stop, run from the site directory) and then verify with ps aux | grep <tool> — a pkill on the launcher name can miss the detached node process. (Discovered 2026-07-09: cold-scan test daemons from two sessions — ports 4331 and 4398–4400 — survived cleanup; Talbot’s pnpm dev errored a day later.)

UI/Frontend Testing (Additional Requirements)

When implementing UI components, styling changes, or frontend features:

1. Run automated tests (E2E, visual regression, accessibility)
2. Test responsive behavior at multiple viewport sizes (mobile, tablet, desktop)
3. Test dark mode (if applicable) - toggle and verify colors
4. Check accessibility - WCAG AA contrast, keyboard navigation, screen readers
5. Verify no horizontal overflow on mobile viewports
6. Test interactive elements - hover states, click handlers, animations
7. Manual dev server verification - visually inspect all changes

Before reporting UI changes complete:

# Build
pnpm build

# Quick automated check
pnpm test:e2e  # or equivalent

# Mobile check
pnpm test:e2e --project="iPhone 14 Pro"  # or smallest viewport

# Start dev server for manual verification
pnpm dev
# → Test at 390px, 768px, 1280px viewports
# → Toggle dark mode
# → Click through all changed components
# → Check for overflow, contrast issues, broken layouts

Only report “Complete” after all automated tests pass AND manual verification confirms the UI works as expected.

Testing Checklist

Before reporting “implementation complete”:

• Code executes without errors
• All automated tests pass (if test suite exists)
• Manual end-to-end test completed (actual workflow)
• Evidence captured (test output, screenshots, metrics)
• Edge cases tested (errors, missing files, conflicts)
• Performance acceptable (not just “works” but “works well”)
• Tested on target environment (WSL, Linux, etc.)
• User-facing functionality verified (not just internals)
• UI changes verified (responsive, accessible, no overflow - see UI/Frontend Testing above)

Only report “Complete” when ALL boxes are checked.

When to Use Task Agents for Testing

Use Task agents (subagent_type=“general-purpose”) when:

• Testing requires multiple steps (start server, wait, test, cleanup)
• Testing requires complex verification logic
• Testing needs to handle timing/async issues
• You need to test multiple scenarios in parallel
• Testing would consume too much of your main context

Example:

Task: Test all three web deployments end-to-end
- Start each server
- Fetch URLs
- Verify content
- Measure performance
- Report results

Common Testing Anti-Patterns (AVOID)

❌ “The code looks correct” - Test it! ❌ “It should work” - Verify it! ❌ “You can test it with curl” - YOU test it with curl! ❌ “Just check if the file exists” - Read the file and verify content! ❌ “Port is open” - Fetch the URL and verify response! ❌ “No syntax errors” - Run it and verify output! ❌ “Tests passed on my machine” - You ARE the machine, run them!

✅ “All tests passed - here’s the output showing 3/3 scenarios working” ✅ “End-to-end test complete - server responds with 200, content verified” ✅ “Performance metrics: load time 0.5s, all thresholds met” ✅ “Tested on WSL: ✓ localhost works, ✓ LAN access works”