研究论文的官方实现与工件：
"DVDR-LLM：大型语言模型何时协同工作能更好地检测和修复软件漏洞？"
DVDR-LLM 是一个集成框架，系统性地研究了聚合多个大型语言模型（LLM）进行软件漏洞检测与修复时的基本权衡。我们的综合评估揭示了在安全关键型应用中关于精确率-召回率平衡、模型多样性收益以及基于共识的方法的关键见解。
DVDR_LLM/
├── dvdr_llm/ # Main package (professional structure)
│ ├── __init__.py # Package exports
│ ├── cli.py # Command-line interface
│ ├── config.py # Configuration settings
│ ├── core/ # Core functionality
│ │ ├── api_client.py # LLM API communication
│ │ ├── detector.py # VulnerabilityDetector class
│ │ └── prompts.py # Prompt generation utilities
│ ├── analysis/ # Analysis modules
│ │ ├── consensus.py # Consensus analysis
│ │ └── metrics.py # Performance metrics
│ ├── evaluation/ # Model evaluation
│ │ └── evaluator.py # ModelEvaluator class
│ ├── visualization/ # Plotting and visualization
│ │ └── plotter.py # ResultsPlotter class
│ └── tools/ # Additional utilities
├── utils/ # Original utility modules
│ ├── api.py # LLM API interaction helpers
│ ├── database.py # SQLite helper functions
│ └── config.py # Configuration constants
├── data/ # Datasets and databases
│ ├── vulnerabilities.csv # Vulnerability data
│ └── vulnerable and patched codes.sqlite
├── output/ # Generated results and databases
│ ├── database_*.sqlite # Model-specific databases
│ ├── consensus_analysis/ # Consensus analysis results
│ ├── metrics/ # Performance metrics
│ └── database_exports/ # Exported data
├── examples/ # Usage examples
│ └── basic_usage.py # Basic usage demonstration
├── docs/ # Documentation
│ └── README_consensus_analysis.md
├── paper/ # Research paper
│ ├── main.pdf # Published paper
│ └── main.tex # LaTeX source
├── setup.py # Package installation
├── requirements.txt # Dependencies
├── CHANGELOG.md # Change log
├── LICENSE # MIT License
└── README.md # This file
克隆仓库：
git clone https://github.com/Erroristotle/DVDR_LLM.git
cd DVDR_LLM
安装依赖：
pip install -r requirements.txt
pip install -e . # Install package in development mode
验证安装：
python verify_package.py
# Run vulnerability detection with ensemble
dvdr-llm detect --models llama3-8b,codellama-7b --database data/vulnerabilities.sqlite
# Analyze consensus patterns (RQ2)
dvdr-llm analyze consensus --input results/metrics/model_predictions.csv --threshold 0.6
# Generate conflict pattern visualization (Figure in paper)
python dvdr_llm/visualization/conflict_pattern_plot.py results/metrics/reviewer_insights_disagreement_patterns.csv -o reviewer_insights
# Evaluate ensemble performance across abstraction levels (RQ3)
dvdr-llm evaluate --models-dir output/ --abstraction-analysis
from dvdr_llm import VulnerabilityDetector, ConsensusAnalyzer, ModelEvaluator
# Initialize vulnerability detector
detector = VulnerabilityDetector("llama3-8b-instruct")
# Connect to database
detector.connect_to_database("data/vulnerabilities.sqlite")
# Analyze code for vulnerabilities
code = """
void vulnerable_function(char *input) {
char buffer[100];
strcpy(buffer, input); // Buffer overflow
}
"""
# Get CVE predictions
cve_names = detector.analyze_code_for_cves(code, 2023)
print(f"Identified CVEs: {cve_names}")
# Detect vulnerability
is_vulnerable = detector.detect_vulnerability(code)
print(f"Is vulnerable: {is_vulnerable}")
# Process database entries
stats = detector.process_database_entries(limit=100)
print(f"Processed {stats['processed']} entries")
# Clean up
detector.close()
# Multi-model consensus analysis
model_databases = {
"llama3-8b": "output/database_llama3-8b-instruct.sqlite",
"codellama-7b": "output/database_codellama-7b-instruct.sqlite",
"gemma2-9b": "output/database_gemma2-9b.sqlite"
}
analyzer = ConsensusAnalyzer()
consensus_results = analyzer.run_full_analysis(model_databases)
# Model evaluation
evaluator = ModelEvaluator()
evaluation_results = evaluator.evaluate_multiple_models(
model_databases,
"data/ground_truth.csv"
)
# Generate Table 2 (Model comparison with delta percentages)
python dvdr_llm/tools/model_contribution_analysis.py results/metrics/model_predictions.csv --compare-ensemble --threshold 0.7
# Analyze individual vs ensemble performance
dvdr-llm evaluate --comparison-analysis --threshold 0.7
# Generate Figure 3 (Threshold sensitivity analysis)
python dvdr_llm/tools/threshold_analysis.py results/metrics/model_predictions.csv --range 0.3-0.8
# Statistical validation of threshold selection
python dvdr_llm/analysis/statistical_validation.py --threshold-analysis
# Generate Table 3 (Abstraction level analysis)
python dvdr_llm/evaluation/evaluator.py --abstraction-analysis --threshold 0.6
# Level-specific performance evaluation
dvdr-llm evaluate --abstraction-levels 1,2,3 --ensemble
# Analyze patch quality metrics (ROUGE, CodeBLEU, Complexity)
python dvdr_llm/tools/patch_quality_analysis.py results/patches/ --weighted-scoring
# Generate Figure 4 (Patch similarity and complexity analysis)
dvdr-llm visualize --patch-analysis --metrics results/patch_metrics.csv
# Main conflict pattern analysis figure (Figure 2)
python dvdr_llm/visualization/conflict_pattern_plot.py results/metrics/reviewer_insights_disagreement_patterns.csv -o reviewer_insights
# Threshold sensitivity analysis (Figure 3)
python dvdr_llm/visualization/enhanced_figure3_generator.py results/metrics/model_predictions.csv
# Statistical validation plots (Appendix)
python dvdr_llm/analysis/statistical_significance_analysis.py --generate-plots
编辑 dvdr_llm/config.py 以配置LLM端点：
LLM_MODELS = {
"llama3_8b": "ollama run llama3:8b-instruct",
"codellama_7b": "ollama run codellama:7b-instruct",
"mistral_7b": "ollama run mistral:7b-instruct",
# Add your model configurations
}
# Consensus thresholds for different scenarios
CONSENSUS_THRESHOLDS = {
"conservative": 0.8, # High precision, low false positives
"balanced": 0.6, # Balanced precision-recall
"sensitive": 0.4 # High recall, catch more vulnerabilities
}
该框架使用标准化的提示模板以确保可复现性：
Δ%表示与集成基线的性能差异
# Implement custom consensus strategy
from dvdr_llm.analysis.consensus import ConsensusAnalyzer
class WeightedConsensus(ConsensusAnalyzer):
def __init__(self, model_weights):
self.weights = model_weights
def weighted_majority_vote(self, predictions):
weighted_sum = sum(pred * weight for pred, weight in zip(predictions, self.weights))
return weighted_sum >= 0.5
# Use custom visualization
from dvdr_llm.visualization import ConflictPatternPlotter
plotter = ConflictPatternPlotter()
plotter.create_professional_figure(
disagreement_data,
output_prefix="custom_analysis",
style="publication"
)
# Add new LLM model
from dvdr_llm.core.api_client import LLMClient
class CustomLLMClient(LLMClient):
def __init__(self, api_endpoint, model_name):
super().__init__(api_endpoint, model_name)
def generate_response(self, prompt, **kwargs):
# Implement custom API interaction
pass
# Register new model
detector.register_model("custom-llm", CustomLLMClient("api_url", "model_name"))
完整的研究发现记录在 paper/main.tex 中。主要贡献包括：
我们欢迎贡献！请参阅我们的贡献指南：
git checkout -b feature/new-analysis）git commit -am 'Add new analysis method'）git push origin feature/new-analysis）请参阅LICENCE文件。
如果您使用DVDR-LLM，请引用： @article{zibaeirad2025diverse, title={Diverse LLMs vs. Vulnerabilities: Who Detects and Fixes Them Better?}, author={Zibaeirad, Arastoo and Vieira, Marco}, journal={arXiv preprint arXiv:2512.12536}, year={2025} }
| 模型 | SVD1 (Δ%) | SVD2 (Δ%) | SVD3 (Δ%) | SVD4 (Δ%) |
|---|
| Llama3-8b | +59.8% | -87.5% | +127.1% | -73.5% |
| Llama3-70b | -50.8% | +54.4% | -19.6% | +13.5% |
| CodeLlama-7b | +16.8% | -34.6% | +86.9% | -45.4% |
| 集成（60%） | Baseline | Baseline | Baseline | Baseline |