PyRIT
microsoft/pyritOpen-source framework for red-teaming generative AI systems: automate attack prompts, score model responses, and audit behavior to identify security…
adversarial-attackai-securitymachine-learning+2
4.6k
microsoft/pyritOpen-source framework for red-teaming generative AI systems: automate attack prompts, score model responses, and audit behavior to identify security…
promptfoo/promptfooTest your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and…
trusted-ai/adversarial-robustness-toolboxPython library for adversarial machine learning security, enabling red and blue teams to run evasion, poisoning, extraction, and inference attacks…
microsoft/responsible-ai-toolboxInteractive dashboards and libraries for responsible AI model debugging, covering error analysis, fairness, interpretability, counterfactuals, causal…
lightless-labs/third-thoughtsResearch toolkit for analyzing AI agent behavioral patterns through multi-disciplinary corpus analysis. Parses session logs, runs 23 analytical…
promptfoo/modelauditSecurity scanner for AI/ML model files. Detects malicious code, backdoors, and vulnerabilities before deployment
giskard-ai/giskard-oss🐢 Open-Source Evaluation & Testing library for LLM Agents
jpoindexter/interrogation-gameVoice-based detective interrogation game. Mistral Large 3 + Voxtral STT + ElevenLabs TTS. Built for the Mistral Worldwide Hackathon 2026.
p-e-w/hereticFully automatic censorship removal for language models
jiep/offensive-ai-compilationA curated list of useful resources that cover Offensive AI.
nvidia/garakModular LLM vulnerability scanner that probes for hallucination, data leakage, prompt injection, jailbreaks, and toxicity using static, dynamic, and…
guillaumemeyer/watermarks-removerA privacy-first app that strips AI watermarks from content you own.
meridianlabs-ai/inspect_petriAn alignment auditing agent capable of quickly exploring alignment hypothesis
kevinchunye/prompt_injectionBenchmark harness measuring where prompt injection defenses fire in tool-using LLM agent pipelines, tracking canary tokens across exposed, persisted,…
sliu11-byte/mea-benchA lifecycle benchmark for black-box LLM extraction attacks, defenses, and adaptive attacks.
kaill-jlq/mmskillriskBenchmark and evaluation harness testing whether LLM agents resist malicious instructions hidden in multimodal skill images, with 108 cases across…
albert-y1n/pismithPISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses
albert-y1n/piforgeOpen framework for RL-based prompt injection red teaming, with a shared trainer, curriculum learning, and benchmarks like AgentDojo, InjecAgent, and…