Rate Scanner Issues — Fix & Extensions

What changed

Pipeline deadlock (root cause fix)

daily-run.sh was gating export on logs/health-check-status. Health-check Check 4 tests rates.json freshness → circular deadlock: stale rates.json → health FAIL → no export → stale. Fixed by removing the health-check gate; health-check is now observability-only. Only the anomaly-status sentinel (written by the run itself) gates export. rates.json manually restored by running rate-scanner export.

Schema fields added

• province_restrictions — added to Product model, SQLite DDL + idempotent ALTER TABLE migration, upsert_product, and both export paths in cli.py. Convention: exclusion list (empty = Canada-wide). Populated for 5 products in rates.yaml. Rendered in ops dashboard Fine Print column.
• notes — already in model + YAML; was not flowing to export. Added to cli.py export and build.py fine print display.
• min_balance_cad — already correctly implemented end-to-end; no changes needed.

Daily status email

New rate_scanner/notify.py module + rate-scanner notify CLI command. Sends plain-text summary after each pipeline run: best HISA vs inertia baseline, pipeline health/anomaly status, dashboard link. Added as Step 7 in daily-run.sh (non-fatal). EMAIL_PASSWORD SSoT: my_backup/.env (no credential duplication). python-dotenv added as explicit dependency.

$10k basis

Changed HISA delta and CC debt comparison from $100k/$5k → $10k in notify.py, cli.py, and build.py.

READMEs

Written for /home/ta/projects/mbr/rate-scanner/ and /home/ta/projects/mbr/ops-dashboard/.

Key files changed

• scripts/daily-run.sh — health-check gate removed; Step 7 (notify) added
• rate_scanner/models.py — province_restrictions on Product
• rate_scanner/db.py — DDL, migration, upsert
• rate_scanner/cli.py — notes + province_restrictions in export; $10k delta; notify command
• rate_scanner/notify.py — new module
• ops-dashboard/build.py — fine print display; $10k basis for HISA + CC
• data/rates.yaml — province_restrictions on 5 products
• tests/ — 39/39 passing (new test_product_province_restrictions_in_db + extended fine print tests)
• rate-scanner/README.md, ops-dashboard/README.md — new
• pyproject.toml / uv.lock — python-dotenv added

Warnings / gotchas

• logs/health-check-status will show FAILED until the next 09:00 ET cron run — self-heals once rates.json is fresh
• province_restrictions uses exclusion convention (non-empty = provinces where product is NOT available). The 5 populated products are all needs_verification: true — they’ll appear in the dashboard once verified
• External utility import order matters: load .env before sys.path.insert + import notify_manager to beat its module-level load_dotenv()