一种黑盒优化方法,用于保护大型语言模型(LLM)系统提示免受对抗性提取攻击。
本仓库实现了提示敏感性最小化(PSM),这是一种防御机制,利用 LLM 作为优化器,在系统提示周围自动生成保护性“护盾”。PSM 通过迭代优化过程,在保持任务效用的同时最小化可观测的泄漏。
git clone <repository-url>
cd psm
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
config/config.yaml 文件,填入你的 LLM API 配置:llms:
gpt-4o-mini:
model: "gpt-4o-mini"
model_provider: "openai"
api_key: "your-api-key"
temperature: 0.7
# Add other model configurations as needed
使用 PSM 生成受保护的系统提示词:
python run.py
这将:
data/victim_prompts/ 加载系统提示词data/defense_prompts/评估防御机制对对抗性攻击的抵御能力:
python experiments/evaluate_defenses.py
这将:
results/ 中生成全面的结果from run import PSMRunner, RunPSMConfig
# Configure PSM
config = RunPSMConfig(
dataset_name="unnatural-test",
attack_samples=50,
validation_samples=10,
target_dataset_samples=30,
llm_target="gpt-5-mini",
llm_optimizer="gpt-4o-mini",
llm_validation="gpt-4o",
llm_judge="gpt-4o-mini",
n_optimization_iterations=10,
n_initial_shields=5,
n_shields_per_step=5,
max_candidates_per_step=10,
target_utility_threshold=0.9,
target_leakage_threshold=0.65,
leakage_aggregation="logsumexp",
logsumexp_temperature=10.0
)
# Run PSM
runner = PSMRunner(config)
runner.run_experiments()
from experiments.evaluate_defenses import (
DefenseEvaluator, ExperimentConfig,
DefenseConfig, DefenseStrategy,
AttackType, Dataset, TargetModel
)
# Configure experiment
config = ExperimentConfig(
attack_types=[AttackType.RACCON, AttackType.LIANG],
defense_configs=[
DefenseConfig(DefenseStrategy.PSM, Dataset.UNNATURAL_TEST, TargetModel.GPT_4O_MINI)
],
max_victim_samples=30,
max_attack_samples=60,
use_am_metric=True,
am_threshold=0.9,
use_jm_metric=True,
jm_threshold=0.7
)
# Run evaluation
evaluator = DefenseEvaluator(config)
await evaluator.run_experiments()
evaluator.save_results()
evaluator.print_results_summary()
src/defense/psm/psm.py)主要的优化引擎,其功能包括:
关键类:
PSM:主优化类PromptCandidate:表示带有评分的候选解决方案PSMConfig:配置参数src/defense/psm/psm_data_creation.py)处理以下内容:
src/metrics.py)评估指标:
experiments/evaluate_defenses.py)综合评估框架:
支持的攻击数据集:
raccon.json:RACCON 攻击提示liang.json:Liang et al. 攻击模式zhang.json:Zhang et al. 攻击模式ours.json:自定义攻击提示raccon_language.json:特定语言的 RACCON 攻击@dataclass
class RunPSMConfig:
# Dataset
dataset_name: str = "unnatural-test"
# Sample sizes
attack_samples: int = 50 # Attack inputs to use
validation_samples: int = 10 # Baseline validation queries
target_dataset_samples: int = 30 # System prompts to process
# Model selection
llm_target: str = "gpt-5-mini" # Target model to protect
llm_optimizer: str = "gpt-4o-mini" # Model for shield generation
llm_validation: str = "gpt-4o" # Model for validation
llm_judge: str = "gpt-4o-mini" # Model for judging
# Optimization parameters
n_optimization_iterations: int = 10 # Max optimization iterations
n_initial_shields: int = 5 # Initial shield population
n_shields_per_step: int = 5 # New shields per iteration
max_candidates_per_step: int = 10 # Max candidates to consider
# Thresholds
target_utility_threshold: float = 0.9 # Minimum utility score
target_leakage_threshold: float = 0.65 # Maximum leakage score
utility_penalty_weight: float = 100.0 # Weight for utility penalty
# Leakage aggregation
leakage_aggregation: str = "logsumexp" # Aggregation method
logsumexp_temperature: float = 10.0 # LogSumExp temperature
top_k: int = 3 # Top-k for top_k_mean
config = ExperimentConfig(
# Attack and defense selection
attack_types=[AttackType.RACCON, AttackType.LIANG],
defense_configs=[...],
# Sample limits
max_victim_samples=30,
max_attack_samples=60,
# AM metric
use_am_metric=True,
am_threshold=0.9,
# JM metric
use_jm_metric=True,
jm_threshold=0.7,
# Output options
output_dir="results",
save_detailed_results=True,
save_per_prompt_results=True
)
本节提供复现本仓库实验结果的详细说明。
config/config.yaml,填入你的 LLM API 配置:llms:
gpt-4o-mini:
model: "gpt-4o-mini"
model_provider: "openai"
api_key: "your-api-key"
temperature: 0
gpt-4o:
model: "gpt-4o"
model_provider: "openai"
api_key: "your-api-key"
temperature: 0
gpt-5-mini:
model: "gpt-5-mini"
model_provider: "openai"
api_key: "your-api-key"
temperature: 0
sentence-transformers/all-MiniLM-L6-v2:
model_provider: "huggingface"
data/victim_prompts/unnatural-test.jsonldata/victim_prompts/syntentic-system-prompt.jsonldata/attack_prompts/raccon.jsondata/attack_prompts/raccon_language.jsondata/attack_prompts/liang.jsondata/attack_prompts/zhang.json运行 PSM 优化过程:
python run.py
预期输出:
data/defense_prompts/psm_target_{model}_dataset_{dataset}.jsonlinstruction:带有防护盾的优化提示词original_instruction:原始系统提示词utility_score:效用保持得分leakage_score:泄漏最小化得分fitness_score:综合适应度得分默认配置:
n_optimization_iterations:10n_initial_shields:5n_shields_per_step:5attack_samples:50validation_samples:10leakage_aggregation:"logsumexp"针对攻击评估防御:
python experiments/evaluate_defenses.py
预期输出:
results/ 目录如需通过编程方式控制,请修改 run.py 中的配置:
config = RunPSMConfig(
dataset_name="unnatural-test",
attack_samples=50,
validation_samples=10,
target_dataset_samples=30,
llm_target="gpt-5-mini",
llm_optimizer="gpt-4o-mini",
llm_validation="gpt-4o",
llm_judge="gpt-4o-mini",
n_optimization_iterations=10,
leakage_aggregation="logsumexp",
logsumexp_temperature=10.0
)