
promptfoo
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and…

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and…

Open-source framework for red-teaming generative AI systems: automate attack prompts, score model responses, and audit behavior to identify security…

Adversary Emulation Framework

A privacy-first app that strips AI watermarks from content you own.

Modular LLM vulnerability scanner that probes for hallucination, data leakage, prompt injection, jailbreaks, and toxicity using static, dynamic, and…

GitLab Duo DevSecOps loop where a second AI re-attacks every AI-written fix, saving each successful bypass as a permanent antibody replayed in CI and…

A collection of awesome resources related AI security

Security scanner for AI/ML model files. Detects malicious code, backdoors, and vulnerabilities before deployment

Fully automatic censorship removal for language models

🐢 Open-Source Evaluation & Testing library for LLM Agents

The AI Security Verification Standard (AISVS) focuses on providing developers, architects, and security professionals with a structured checklist to…

Interactive dashboards and libraries for responsible AI model debugging, covering error analysis, fairness, interpretability, counterfactuals, causal…

Hardware Breakpoint (DR0-DR7) based patch-less user-mode hooking & telemetry instrumentation engine (AMSI, WLDP & ETW PoC).

The exploit server for out-of-band findings. Point a target at a domain you own. Every HTTP request and every email it sends back lands in a…

A curated list of AI Security materials and resources for Pentesters, Bug Hunters, and Security Researchers.

A repo for jailbreaking various LLMs, mainly Claude

Curated collection of LLM jailbreak prompts and bypass techniques, documenting adversarial inputs that circumvent AI model safety guardrails.

Clusters and elements to attach to MISP events or attributes (like threat actors)