Awesome AI Security

A curated list of awesome AI security related frameworks, standards, learning resources and open source tools.

If you want to contribute, create a PR or contact me @ottosulin.

Table of Contents

• Learning Resources
  • Reading & Guides
  • Courses, Labs & CTFs
  • Podcasts
• Governance & Risk Management
  • Frameworks
  • Standards & Verification
  • Taxonomies, Terminology & Risk Databases
  • Checklists & Practical Guidance
• Attack Techniques & Red Teaming
  • Adversarial ML & Classical Models
  • LLM & GenAI Red Teaming
  • Agentic AI & MCP Attack Tools
  • AI-Assisted Offensive Security
  • Steganography & Covert Channels
• Benchmarks & Evaluations
• Defense & Security Controls
  • Input/Output Guardrails
  • Agent Runtime Security & Sandboxing
  • MCP Security
  • Model & Artifact Scanning
  • AI-Assisted Defensive Security
  • Privacy & Confidential Computing
  • Data & Supply Chain Security
• Agentic AI Security Skills
• Security-Focused AI Models

Learning Resources

Reading & Guides

• OWASP ML TOP 10
• OWASP LLM TOP 10
• OWASP AI Security and Privacy Guide
• NIST AIRC - NIST Trustworthy & Responsible AI Resource Center
• The MLSecOps Top 10 by Institute for Ethical AI & Machine Learning
• OWASP Multi-Agentic System Threat Modeling
• OWASP: CheatSheet – A Practical Guide for Securely Using Third-Party MCP Servers 1.0
• OWASP GenAI Threat & Defense Compass - Reference guide mapping GenAI threats to corresponding defensive controls.
• NCSC Guidelines for Secure AI System Development - Practical guidelines co-authored by NCSC (UK) and CISA for secure AI development, covering design, development, deployment, and operation.
• CoSAI – Preparing Defenders for a Changing Cybersecurity Landscape - Coalition for Secure AI workstream preparing defenders for AI-era threats.

Courses, Labs & CTFs

• Damn Vulnerable MCP Server - A deliberately vulnerable implementation of the Model Context Protocol (MCP) for educational purposes.
• otto-support - A vulnerable MCP server implementation using mcp-go with tiered authentication (4 roles), 19 tools, and built-in sandboxed container with Claude Code for practicing privilege escalation and tool misuse.
• OWASP WrongSecrets LLM exercise
• vulnerable-mcp-servers-lab - A collection of servers which are deliberately vulnerable to learn Pentesting MCP Servers.
• FinBot Agentic AI Capture The Flag (CTF) Application - FinBot is an Agentic Security Capture The Flag (CTF) interactive platform that simulates real-world vulnerabilities in agentic AI systems using a simulated Financial Services-focused application.
• AI-Red-Teaming-Playground-Labs - AI Red Teaming playground labs to run AI Red Teaming trainings including infrastructure.
• Damn Vulnerable LLM Agent - Intentionally vulnerable LLM agent for learning about prompt injection, tool misuse, and agent security
• • LLMVault - An open-source CTF-style LLM security lab for learning the OWASP LLM Top 10 through hands-on vulnerable exercises covering prompt injection, RAG attacks, system prompt leakage, tool misuse, and agent security.

Podcasts

• MLSecOps podcast
• AI Security Podcast
• AI Security Ops - Weekly podcasts from Black Hills Information Security exploring how AI transforms cybersecurity—covering emerging threats, tools, and trends with practical, actionable knowledge.
• GenAI Security podcast

Governance & Risk Management

Frameworks

• NIST AI Risk Management Framework
• ISO/IEC 42001 Artificial Intelligence Management System
• ISO/IEC 23894:2023 Information technology — Artificial intelligence — Guidance on risk management
• Google Secure AI Framework (SAIF)
• ENISA Multilayer Framework for Good Cybersecurity Practices for AI
• OWASP Artificial Intelligence Maturity Assessment
• CSA AI Model Risk Framework
• CSA Maestro AI Threat Modeling Framework
• CSA AI Controls Matrix - Comprehensive controls matrix for AI systems covering governance, risk, and compliance.
• OWASP Agentic AI Top 10 - Top 10 for Agentic AI serving as the core for OWASP and CSA red teaming.
• OWASP GenAI Crosswalk - Interactive crosswalk mapping OWASP GenAI risks (LLM Top 10, Agentic Top 10, DSGAI 2026) to 25 industry frameworks with coverage scoring and gap analysis.
• NIST SP 800-218A Secure Software Development Practices for Generative AI and Dual-Use Foundation Models - Extension of SSDF (SP 800-218) with specific practices for GenAI and dual-use foundation model development.
• NIST AI 600-1 Generative AI Profile - Companion profile to NIST AI RMF 1.0 with specific risk actions for generative AI systems including unique risks like hallucination, data poisoning, and privacy.