Skip to content
FSS Knowledge Base

Skill Security Gate — build + rollout

Section titled “Skill Security Gate — build + rollout”

Ask: external skills pose real risk (prompt injection, exfiltration, dangerous code) — need a pre-use security check, a global rule gating it, and a security-tested frontmatter convention. Test on webapp-testing. Borrow tools over building.

What shipped

Section titled “What shipped”
  • skill-security-review — new dependency-free skill (Read+Grep only, no third-party scanner) at ~/ai-config/skills/skill-security-review/SKILL.md. Checklist: prompt injection/instruction-hijack, code-execution risk, supply chain, file structure. Two-tier: Official (Anthropic anthropics/skills, claude-plugins-official marketplace, self-authored) → lighter read-through; External/untrusted → full checklist. Deployed to WSL/Windows Claude Code, AntiGravity, monorepo Cursor.
  • Global rule — AGENTS.md Skills Quality Standard, approved by Talbot with one amendment: no tier is auto-pass, even “Official” gets a check, never a skip. security-tested: YYYY-MM-DD frontmatter stamp is the pass record; absence = unreviewed, regardless of install age or usage history.
  • /audit-skills Step 5 — monthly audit now flags any skill missing security-tested, and any skill edited since its stamped date (mtime vs. stamp).
  • Tested live on webapp-testing (anthropics/skills) — classified Official, ran lighter check, flagged its own “don’t read before running” instruction per the checklist’s own rule, read the script anyway (with_server.py) — clean, local-only subprocess lifecycle, no exfiltration. PASS.
  • Retroactive sweep — all 14 self-authored skills + 4 official plugins (superpowers, frontend-design, skill-creator, playwright) reviewed (grep sweep + spot-read), all clean, all stamped/recorded PASS.
  • caveman@caveman plugin — the one currently-installed genuinely external skill (marketplace github.com/JuliusBrussee/caveman, not claude-plugins-official). Full checklist not run — it’s a ~30-subdirectory monorepo, out of scope for a single-round check. Did spot-check the parts that execute automatically every session (src/hooks/*) — clean, no exfil/eval/obfuscation. Talbot accepted this as sufficient for now (known/named public repo, already in daily use) rather than commissioning a full audit.

Tools evaluated (borrow vs. build)

Section titled “Tools evaluated (borrow vs. build)”

Real scanners exist (Cisco Skill Scanner, a claude-code-security-scanner on mcpmarket, community skill-security-auditor repos with Python static analyzers) — all rejected as dependencies. Every one is itself an unvetted third-party artifact; installing one to check other skills just relocates the trust problem. Borrowed the checklist categories only, kept the actual review tool dependency-free.

Commits

Section titled “Commits”

ai-config: a905600 (skill built) → e0760b3 (fixed deploy.sh USER_SKILLS gap — see LESSONS.md) → d4ce7ae (AGENTS.md rule + 14 skills stamped) → 788f38e (/audit-skills Step 5) → 70d1a96 (CHANGELOG/LESSONS closeout).

Lesson promoted

Section titled “Lesson promoted”

ai-config/LESSONS.md: new skill dir + commit ≠ deployed — deploy.sh’s USER_SKILLS array is a hardcoded allowlist, must add the name explicitly; the script echoes “Deploy complete” regardless. Verify against the actual target, not the echo.

Open item

Section titled “Open item”

Full caveman plugin checklist — not commissioned as a follow-up task; revisit only if risk profile changes (e.g. plugin gets deeper hooks, or an incident elsewhere raises the bar).