Awesome AI Security

A curated list of awesome AI security related frameworks, standards, learning resources and open source tools.

If you want to contribute, create a PR or contact me @ottosulin.

Table of Contents

• Learning Resources
  • Reading & Guides
  • Courses, Labs & CTFs
  • Podcasts
• Governance & Risk Management
  • Frameworks
  • Standards & Verification
  • Taxonomies, Terminology & Risk Databases
  • Checklists & Practical Guidance
• Attack Techniques & Red Teaming
  • Adversarial ML & Classical Models
  • LLM & GenAI Red Teaming
  • Agentic AI & MCP Attack Tools
  • AI-Assisted Offensive Security
  • Steganography & Covert Channels
• Benchmarks & Evaluations
• Defense & Security Controls
  • Input/Output Guardrails
  • Agent Runtime Security & Sandboxing
  • MCP Security
  • Model & Artifact Scanning
  • AI-Assisted Defensive Security
  • Privacy & Confidential Computing
  • Data & Supply Chain Security
• Agentic AI Security Skills
• Security-Focused AI Models

Learning Resources

Reading & Guides

• OWASP ML TOP 10
• OWASP LLM TOP 10
• OWASP AI Exchange
• NIST AIRC - NIST Trustworthy & Responsible AI Resource Center
• The MLSecOps Top 10 by Institute for Ethical AI & Machine Learning
• OWASP Multi-Agentic System Threat Modeling
• OWASP: CheatSheet – A Practical Guide for Securely Using Third-Party MCP Servers 1.0
• OWASP GenAI Threat & Defense Compass - Reference guide mapping GenAI threats to corresponding defensive controls.
• NCSC Guidelines for Secure AI System Development - Practical guidelines co-authored by NCSC (UK) and CISA for secure AI development, covering design, development, deployment, and operation.
• CoSAI – Preparing Defenders for a Changing Cybersecurity Landscape - Coalition for Secure AI workstream preparing defenders for AI-era threats.
• Prompt Injection Defenses - Comprehensive collection of every practical and proposed defense against prompt injection.

Courses, Labs & CTFs

• Damn Vulnerable MCP Server - A deliberately vulnerable implementation of the Model Context Protocol (MCP) for educational purposes.
• otto-support - A vulnerable MCP server implementation using mcp-go with tiered authentication (4 roles), 19 tools, and built-in sandboxed container with Claude Code for practicing privilege escalation and tool misuse.
• OWASP WrongSecrets LLM exercise
• vulnerable-mcp-servers-lab - A collection of servers which are deliberately vulnerable to learn Pentesting MCP Servers.
• FinBot Agentic AI Capture The Flag (CTF) Application - FinBot is an Agentic Security Capture The Flag (CTF) interactive platform that simulates real-world vulnerabilities in agentic AI systems using a simulated Financial Services-focused application.
• AI-Red-Teaming-Playground-Labs - AI Red Teaming playground labs to run AI Red Teaming trainings including infrastructure.
• Damn Vulnerable LLM Agent - Intentionally vulnerable LLM agent for learning about prompt injection, tool misuse, and agent security
• • LLMVault - An open-source CTF-style LLM security lab for learning the OWASP LLM Top 10 through hands-on vulnerable exercises covering prompt injection, RAG attacks, system prompt leakage, tool misuse, and agent security.
• AIGoat - Open-source AI security playground for LLM red teaming: 17 attack labs covering the full OWASP LLM Top 10, 9 CTF challenges, and 3 progressive defense levels, running locally with Ollama.
• Pwned Labs indirect prompt injection lab - Free hands-on lab: instructions planted in retrieved content take an AI assistant past its trust boundary into Azure storage and deployment credentials. A direct injection lab covers the first step.

Podcasts

• MLSecOps podcast
• AI Security Podcast
• AI Security Ops - Weekly podcasts from Black Hills Information Security exploring how AI transforms cybersecurity—covering emerging threats, tools, and trends with practical, actionable knowledge.
• GenAI Security podcast

Governance & Risk Management