SkillSpector

Security scanner for AI agent skills. Detect vulnerabilities, malicious patterns, security risks, prompt injection, data exfiltration, and supply-chain risks in Claude Code, Codex, and MCP skills before you install them.

Features

• Multi-format input: Scan Git repos, URLs, zip files, directories, or single files
• 71 vulnerability patterns across 17 categories: prompt injection, data exfiltration, privilege escalation, supply chain, excessive agency, output handling, system prompt leakage, memory poisoning, tool misuse, rogue agent, anti-refusal, trigger abuse, dangerous code (AST), taint tracking, YARA signatures, MCP least privilege, and MCP tool poisoning
• Two-stage analysis: Fast static analysis + optional LLM semantic evaluation
• Live vulnerability lookups: SC4 queries OSV.dev for real-time CVE data with automatic offline fallback
• Multiple output formats: Terminal, JSON, Markdown, and SARIF reports
• Risk scoring: 0-100 score with severity labels and clear recommendations
• Baseline / false-positive suppression: Accept known findings via a glob-rule or fingerprint baseline so re-scans surface only new issues (docs)

Quick Start

Installation

Open-source software notice: This project will download and install additional third-party open source software projects. Review the license terms of these open source projects before use.

Create and activate a virtual environment first (all make targets assume the venv is active). Use uv or pip; the Makefile uses uv if available, otherwise pip.

Quick install with uv (CLI-only):

If you plan to run skillspector mcp, install the MCP extra at install time:

From source:

Docker (no Python required)

Run SkillSpector without installing Python by building it locally from the included Dockerfile. The image is based on the Docker Official Python 3.12-slim-bookworm image.

Build the image:

Scan a local directory by mounting your current directory into /scan, the container's working directory:

Scan with LLM analysis by passing credentials with a local .env file:

Or pass credentials directly from your shell environment:

Write a report to the host filesystem by writing to the mounted directory:

Optional alias for repeated static scans:

Basic Usage

Size limits

SkillSpector enforces two independent caps on remote and archive inputs to bound the impact of oversized downloads and zip bombs:

• Per-ingest cap: INGEST_MAX_BYTES (100 MiB) — applied to streamed URL downloads, total uncompressed size of zip archives, and post-clone disk usage of Git repos.
• Zip member cap: INGEST_MAX_ZIP_MEMBERS (10,000) — caps the number of entries in a single zip.

Note that the per-file 1 MB analysis cap (MAX_FILE_BYTES) is a separate, downstream limit: it bounds what individual analyzers will read out of an already-ingested directory. The ingest caps above bound how much content can land on disk in the first place. A breach of either ingest cap fails closed with an IngestLimitExceededError.

Output Formats

Batch Scanning

Scan entire directories of skills in parallel from contrib/batch_scan/:

Supports multilingual detection (zh/ja/ko) and terminal/JSON/Markdown output.

For LLM scans with higher concurrency, configure multiple API keys following .env.example — the pool improves throughput and resilience, provided the keys don't share an account-level rate limit.

See the contrib guide for details.

Note on LLM support: The default configuration targets DeepSeek as the cheapest public option. DeepSeek-Chat is expected to sunset, and the contributor does not have hardware to test against local models. The batch scanner was originally tested with OpenAI-compatible endpoints — DeepSeek's lack of structured-output support required manual JSON-parsing patches. If you can contribute a more universal backend (Ollama, vLLM, or a different provider), PRs are very welcome.

Suppressing False Positives (baseline)

Suppress known/accepted findings so the risk score reflects only un-triaged issues and re-scans surface only new findings. See the suppression guide for the full reference.

A baseline can also use drift-tolerant glob rules (by rule id, file path, or message) — see .skillspector-baseline.example.yaml. Exact fingerprint baselines are evidence-bound: changing the scanned source or SkillSpector version keeps the finding active until it is reviewed again. When a selected baseline or baseline output is stored inside the skill directory, SkillSpector excludes that exact file from content analysis so its suppression text cannot create findings or enter regenerated fingerprints; sibling files remain in normal scan scope.

LLM Analysis

For the best results, configure an OpenAI-compatible LLM endpoint for semantic analysis. Pick a provider with SKILLSPECTOR_PROVIDER; hosted providers ship bundled default models, while CLI providers fall back to the local runtime's default model unless SKILLSPECTOR_MODEL is set. SkillSpector also works against local OpenAI-compatible servers (Ollama, vLLM, llama.cpp) and managed inference gateways.

MCP Server

Run SkillSpector as a Model Context Protocol server so any MCP-capable agent (Claude Code, Codex CLI, Gemini CLI) or remote runtime can call scanning as a tool and gate skill/MCP installs on the result — turning SkillSpector into a runtime guardrail instead of an out-of-band audit step.

skillspector mcp requires skillspector[mcp].

The stdio transport is the current FastMCP path for local CLI agents, and the initialize hang reported in issue #199 still applies there.

The server exposes a single tool:

• scan_skill(target, use_llm=true, output_format="json") — scans a Git URL, file URL, .zip, .md file, or directory and returns a structured verdict: risk_score (0-100), severity, recommendation, safe_to_install, and findings. It also reports llm_used / scan_mode so a low score from a static-only scan is never mistaken for a clean full scan.

Register it with Claude Code via:

Security — HTTP transport trust model

The HTTP transport ships without authentication. Any caller that can reach the port can invoke scan_skill. Over stdio or 127.0.0.1 this is the same trust boundary as the CLI. If you bind to a routable interface:

• Sit the server behind an authenticating reverse proxy (e.g. nginx + mTLS) before exposing it externally.
• Local paths and file:// URLs are automatically rejected over HTTP to prevent unauthenticated callers from reading arbitrary host files. Only remote Git and .zip URLs are accepted.

Vulnerability Patterns

SkillSpector detects 71 vulnerability patterns across 17 categories:

Prompt Injection (6 patterns)

Anti-Refusal (3 patterns)

Data Exfiltration (4 patterns)

Privilege Escalation (3 patterns)

Supply Chain (9+ patterns)

Excessive Agency (5 patterns)

Output Handling (3 patterns)

System Prompt Leakage (3 patterns)

Memory Poisoning (3 patterns)

Tool Misuse (3 patterns)

Rogue Agent (2 patterns)

Trigger Abuse (3 patterns)

Behavioral AST (9 patterns)

Taint Tracking (5 patterns)

YARA Signatures (4 patterns)

MCP Least Privilege (4 patterns)

MCP Tool Poisoning (4 patterns)

All detected patterns are listed in the tables above.

Risk Scoring

Score Calculation

• CRITICAL issues: +50 points
• HIGH issues: +25 points
• MEDIUM issues: +10 points
• LOW issues: +5 points
• Executable scripts: 1.3x multiplier

Severity Levels

Example Output

Terminal Output

Configuration

Environment Variables

CLI providers (claude_cli, codex_cli): No API key is needed. Authentication is managed entirely by the agent CLI's own login session (claude auth login / codex login). SkillSpector never reads or forwards API keys when these providers are active. The subprocess is run in a hardened sandbox: tools disabled, no MCP, read-only sandbox mode (codex), and untrusted skill content is delivered only via stdin.

CLI Options

Integrating SkillSpector

SkillSpector is built to be driven by other tools (CI pipelines, install gates, editor integrations). Its exit code and JSON output are a stable contract.

Exit codes

skillspector scan exits with:

The exit code collapses SAFE and CAUTION into 0. To act differently on them (e.g. warn on CAUTION but block on DO_NOT_INSTALL), read the recommendation field from the JSON output rather than relying on the exit code.

Machine-readable output

--format json produces a JSON report; with no --output/-o it is written to stdout:

The top-level shape is (this example shows a full LLM-backed scan; with --no-llm, metadata.llm_requested is false):

• risk_assessment.severity ∈ LOW | MEDIUM | HIGH | CRITICAL.
• risk_assessment.recommendation ∈ SAFE | CAUTION | DO_NOT_INSTALL, mapped from severity: LOW → SAFE, MEDIUM → CAUTION, HIGH/CRITICAL → DO_NOT_INSTALL.
• metadata.llm_error appears only when LLM analysis was requested but unavailable.
• metadata.inference_usage contains one sanitized record per LLM response when the provider exposes token counters. It is an empty list when usage is unavailable; SkillSpector never estimates missing tokens. Prompt totals are inclusive of cache reads and writes so downstream pricing can separate those partitions safely. model_source distinguishes an independently identified provider model from the exact requested model used when response identity is absent or ambiguous. SkillSpector does not currently send Anthropic prompt-cache controls, so its scan requests cannot select the separate 5-minute or 1-hour cache-write tiers; TTL-specific response fields are normalized defensively into the aggregate cache-write counter.
• See Inference usage telemetry for the complete provenance, cache-accounting, privacy, fail-closed ingestion, and downstream pricing contract.
• The full per-issue shape is defined by Finding.to_dict() in models.py; rely on the fields above and treat any additional fields as best-effort.

For CI/IDE tooling, --format sarif emits SARIF 2.1.0.

Recommended gate mapping

When using SkillSpector as an install gate, map the recommendation to an action:

SkillSpector computes the score band and recommendation; how strict the gate is (e.g. whether CAUTION blocks in CI) is a policy decision for the integrating tool.

Development

Setup

All make targets assume a virtual environment is already created and activated. The Makefile uses uv if available, else pip.

How It Works

SkillSpector uses a two-stage detection pipeline:

Stage 1: Static Analysis

• Fast regex-based pattern matching across 11 static analyzers
• AST-based behavioral analysis detecting dangerous calls (exec, eval, subprocess, etc.)
• Live vulnerability lookups via OSV.dev for known CVEs in dependencies
• Scans all analyzer-eligible files in the skill
• High recall (catches most issues)
• Moderate precision (some false positives)

A valid, root-level OpenSSF Model Signing signature (skill.oms.sig) is retained in the component inventory as type oms_signature, but excluded from static and LLM content analysis. OMS bundles necessarily contain long base64-encoded payload, signature, and certificate fields; generic obfuscated-code checks can otherwise misclassify those fields as hidden executable content. The recognizer checks the minimal OMS DSSE/in-toto structure; it does not verify the signature, certificate chain, transparency-log entry, or signer identity. Invalid or unrecognized signature files are scanned normally.

Stage 2: LLM Semantic Analysis (Optional)

• Evaluates context and intent
• Filters false positives
• Provides human-readable explanations
• Improves precision to ~87%

The LLM prompt includes anti-jailbreak protections to prevent malicious skills from manipulating the analysis.

Live Vulnerability Lookups (SC4)

SC4 uses the OSV.dev API to check dependencies against the full Open Source Vulnerabilities database — covering tens of thousands of advisories across PyPI and npm.

• No API key required — OSV.dev is free and unauthenticated.
• Batch queries — all dependencies are checked in a single HTTP call.
• Automatic fallback — if OSV.dev is unreachable (air-gapped/offline), a small built-in fallback list is used.
• Caching — results are cached in-memory for 1 hour to avoid redundant API calls during a session.

The tool requires outbound HTTPS access to api.osv.dev for live vulnerability data. When that is not available, findings are limited to the static fallback list.

Trust model and data egress

SkillSpector is defense-in-depth, not a sandbox. Know what it does and does not do before relying on it:

• It never executes the scanned skill. All analysis is static (regex, Python AST, YARA) plus optional LLM evaluation of file contents — the skill's code is never run.
• LLM analysis sends analyzer-eligible file contents to the configured provider. When LLM analysis is enabled (the default), file contents are sent to the active SKILLSPECTOR_PROVIDER endpoint. Recognized OMS signature files are excluded. Use --no-llm to keep contents local (static analysis only).
• SC4 sends dependency names to OSV.dev. The supply-chain check queries OSV.dev with the package names and versions the skill declares, to look up known CVEs. This is fundamental to the check and runs even with --no-llm. It sends dependency coordinates (not file contents), requires no API key, and falls back to a bundled list when OSV.dev is unreachable.
• It does not sandbox the host. SkillSpector flags risky patterns before you install a skill; it does not contain or isolate a skill you choose to install anyway.

Limitations

• Non-English content: May miss patterns in other languages
• Image-based attacks: Cannot analyze text in images
• Encrypted/binary code: Cannot analyze compiled or encrypted content
• Runtime behavior: Static analysis only, no dynamic execution
• Offline SC4: Without network access to api.osv.dev, SC4 uses a small static fallback list

Research Background

Based on research from "Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale" (Liu et al., 2026):

• Dataset: 42,447 skills from major marketplaces
• Vulnerable: 26.1% contain at least one vulnerability
• High-severity: 5.2% show likely malicious intent
• Key finding: Skills with executable scripts are 2.12x more likely to be vulnerable

Python API Integration

License

Apache License 2.0 - see LICENSE for details.

Contributing

Contributions are welcome! Please read our contributing guidelines and submit pull requests.

Support

• Issues: GitHub Issues