Multi-engine framework for unpacking and analyzing VM-protected binaries using dynamic taint tracking, symbolic execution, pattern classification, and ML-driven prioritization to automate reverse engineering of commercial and custom protectors.
Project will be public by mid-October Refactoring In Progress
Advanced Virtual Machine Detection and Analysis Framework
VMDragonSlayer is a comprehensive framework for analyzing binaries protected by Virtual Machine (VM) based protectors such as VMProtect 2.x/3.x, Themida, and custom malware VMs. The framework combines multiple analysis engines including Dynamic Taint Tracking (DTT), Symbolic Execution (SE), Pattern Classification, and Machine Learning to automate the reverse engineering process.
Goal: Transform complex protected binary analysis from weeks/months of manual work into structured, automated analysis with explainable results.
| Domain | Engine / Module | Highlights |
|---|---|---|
| VM Discovery | analysis.vm_discovery | Dispatcher & handler table identification, nested VM heuristics |
| Pattern Analysis | analysis.pattern_analysis | Rule-based + similarity + ML (hybrid auto-selection) |
| Taint Tracking | analysis.taint_tracking | Intel Pin–driven byte-level taint, handler discovery, flow confidence |
| Symbolic Execution | analysis.symbolic_execution.executor | PathPrioritizer ML-weighted exploration, constraint & state tracking |
| Hybrid Orchestration | (Python core) | Sequential / parallel / adaptive workflows (Ghidra report indicates implemented) |
| Synthetic Data | data/training/synthetic_sample_generator.py | Obfuscation mutation, multi-architecture sample generation |
| Pattern DB | data/patterns/ | JSON + enhanced DB + SQLite-backed runtime patterns |
| Ghidra Plugin | plugins/ghidra/ | In-progress UI integration (several templates missing) |
| Schemas / Validation | data/schemas/ | JSON schema–validated analysis output & pattern formats |
VMDragonSlayer uses a modular architecture where multiple analysis engines work together:
graph TD
A[VM Discovery Engine] --> B[Pattern/ML Classifier]
B --> C[Symbolic Execution Engine]
B --> D[Dynamic Taint Tracker]
D --> C
subgraph DataSources ["Data Sources"]
E[Pattern Database]
F[ML Models - PoC]
end
subgraph Coordination
G[Orchestrator - Workflow Management & Coordination]
end
E --> B
F --> B
A --> G
B --> G
C --> G
D --> G
E --> G
F --> G
G --> H[REST API Server]
G --> I[Plugins - RE Tools]
dragonslayer.analysis.vm_discovery)dragonslayer.analysis.taint_tracking)dragonslayer.analysis.pattern_analysis)dragonslayer.analysis.symbolic_execution)dragonslayer.ml)VMDragonSlayer/
├── dragonslayer/ # Main Python package
│ ├── analysis/ # Analysis engines
│ │ ├── vm_discovery/ # VM detection and classification
│ │ ├── pattern_analysis/ # Pattern matching and ML classification
│ │ ├── symbolic_execution/ # Symbolic execution engine
│ │ ├── taint_tracking/ # Dynamic taint analysis
│ │ └── anti_evasion/ # Anti-analysis countermeasures
│ ├── api/ # REST API server and client
│ ├── core/ # Core framework components
│ ├── ml/ # Machine learning pipeline
│ ├── analytics/ # Analysis reporting and metrics
│ ├── gpu/ # GPU acceleration support
│ ├── utils/ # Utility functions
├── data/ # Configuration and data files
│ ├── patterns/ # Pattern database
│ ├── models/ # ML models and metadata
│ │ ├── pretrained/ # Pre-trained models (PoC)
│ │ └── metadata/ # Model metadata and schemas
│ ├── samples/ # Sample files and registries
│ ├── schemas/ # JSON schemas for validation
│ └── training/ # Training configurations
├── plugins/ # Reverse engineering tool plugins
│ ├── ghidra/ # Ghidra plugin (Java/Gradle)
│ ├── idapro/ # IDA Pro plugin (Python)
│ └── binaryninja/ # Binary Ninja plugin (Python)
├── tests/ # Tests suite
├── documentation/ # Documentation
└── LICENSE # GPL v3 License
VMDragonSlayer integrates with major reverse engineering tools:
Note: The included ML models are basic proof-of-concept implementations designed for research and educational purposes.