CEREBRO-RED v2 (Research Edition)

Autonomous Local LLM Red Teaming Suite

A research-grade framework for automated vulnerability discovery in local LLMs using Agentic Fuzzing and Adaptive Adversarial Mutation (AAM).

Research Goals

• Implement PAIR Algorithm (Prompt Automatic Iterative Refinement) from arxiv.org/abs/2310.08419
• LLM-as-a-Judge semantic evaluation with Chain-of-Thought reasoning
• Telemetry-First architecture for whitepaper-grade analysis
• Multi-provider LLM support (Ollama, Azure OpenAI, OpenAI)

Architecture

Tech Stack

• Backend: FastAPI (async/await), Pydantic (strict types), Uvicorn
• LLM Gateway: litellm (universal adapter for Ollama/Azure/OpenAI)
• Database: SQLite (experiments) + JSONL (audit logs)
• Frontend: React + Vite + TailwindCSS + ShadcnUI + Recharts
• Container: Docker + Docker Compose

System architecture showing main components and data flow

Core Modules

1. Orchestrator (backend/core/engine.py): Async batch processing with exponential backoff
2. Mutator (backend/core/mutator.py): PAIR algorithm with mutation strategies
3. Judge (backend/core/judge.py): LLM-as-a-Judge with CoT evaluation
4. Telemetry (backend/core/telemetry.py): Thread-safe JSONL audit logger

Frontend Dashboard

The React-based frontend provides a comprehensive interface for managing experiments, monitoring progress, and analyzing results.

Main dashboard interface showing experiment overview and statistics

Experiment management view with real-time status updates and experiment list

Complete user interface overview showing all available features

Experiment Results & Analysis

Results view displaying experiment outcomes, vulnerability findings, and detailed analysis

Settings and configuration panel for customizing experiment parameters

Real-Time Monitoring

Real-time monitoring dashboard with live experiment progress and status indicators

Telemetry view showing detailed audit logs, system events, and performance metrics

Detailed logs view with filtering and search capabilities

Performance metrics and statistics dashboard

System status overview showing health checks and component status

API Documentation

Interactive API documentation interface with endpoint explorer

For detailed architecture documentation, see docs/ARCHITECTURE.md.

Quick Start

Prerequisites

• Docker 24.0+
• Docker Compose 2.20+
• Ollama running on host (or Azure/OpenAI API keys)

Docker Setup

If Docker is not running, start the Docker daemon:

# Start Docker daemon
sudo systemctl start docker

# Enable Docker to start on boot
sudo systemctl enable docker

# Add your user to the docker group (to run Docker without sudo)
sudo usermod -aG docker $USER

# Apply group changes (logout/login or use newgrp)
newgrp docker
# OR logout and login again for changes to take effect

Verify Docker is running:

docker --version
docker compose version

Installation

1. Clone repository:

   git clone https://github.com/Leviticus-Triage/cerebro-red-v2.git
   cd cerebro-red-v2
   

2. Configure environment:

   cp .env.example .env
   # Edit .env with your LLM provider credentials
   

3. WICHTIG: Prüfe Port 8000

   # Falls Port 8000 belegt ist:
   lsof -i :8000  # Finde Prozess
   # Oder ändere Port in .env: CEREBRO_PORT=8001
   

4. Start Backend (WICHTIG - muss laufen!):

   # Option 1: Automatisch (empfohlen)
   ./START_BACKEND.sh
   
   # Option 2: Docker
   docker compose up -d cerebro-backend
   
   # Option 3: Lokal
   cd backend
   uvicorn main:app --reload --port 9000
   

5. Prüfe Backend-Status:

   curl http://localhost:9000/health
   # Sollte {"status": "healthy", ...} zurückgeben
   

6. Quick Tests ausführen:

   ./QUICK_TEST_EXAMPLES.sh
   

7. Access dashboard:

   • Backend API: http://localhost:9000
   • Frontend UI: http://localhost:3000 (optional: docker compose up -d cerebro-frontend)
   • API Docs: http://localhost:9000/docs

   

   Frontend user interface showing experiment management and monitoring

---

Quickstart: Local vs Cloud Deployment

Local Deployment (Ollama)

Best for: Privacy-focused testing, no API costs, offline operation.

# 1. Install and start Ollama
curl -fsSL https://ollama.ai/install.sh | sh
ollama pull llama3.2:3b
ollama serve

# 2. Configure .env for local
cat > .env << 'EOF'
TARGET_MODEL=ollama/llama3.2:3b
ATTACKER_MODEL=ollama/llama3.2:3b
JUDGE_MODEL=ollama/llama3.2:3b
OLLAMA_BASE_URL=http://host.docker.internal:11434

# Relaxed circuit breaker for local (slower responses)
CIRCUIT_BREAKER_FAILURE_THRESHOLD=15
CIRCUIT_BREAKER_TIMEOUT=120
CIRCUIT_BREAKER_JITTER_ENABLED=true
EOF

# 3. Start services
docker compose up -d

# 4. Verify
curl http://localhost:9000/health | jq

Cloud Deployment (OpenAI)

Best for: Faster responses, higher quality mutations, production testing.

# 1. Configure .env for cloud
cat > .env << 'EOF'
TARGET_MODEL=openai/gpt-4o-mini
ATTACKER_MODEL=openai/gpt-4o-mini
JUDGE_MODEL=openai/gpt-4o-mini
OPENAI_API_KEY=sk-your-key-here

# Standard circuit breaker for cloud
CIRCUIT_BREAKER_FAILURE_THRESHOLD=10
CIRCUIT_BREAKER_TIMEOUT=60
CIRCUIT_BREAKER_JITTER_ENABLED=true
EOF

# 2. Start services
docker compose up -d

# 3. Verify
curl http://localhost:9000/health | jq

Hybrid Deployment (Multi-Provider)

Best for: Cost optimization (cheap target, quality attacker/judge).

# Configure .env for hybrid
cat > .env << 'EOF'
# Target on local Ollama (cheap, many requests)
TARGET_MODEL=ollama/llama3.2:3b
OLLAMA_BASE_URL=http://host.docker.internal:11434