RetDec - A Retargetable Machine-Code Decompiler Based On LLVM
RetDec is a retargetable machine-code decompiler based on LLVM.
The decompiler is not limited to any particular target architecture, operating system, or executable file format:
- Supported file formats: ELF, PE, Mach-O, COFF, AR (archive), Intel HEX, and raw machine code
- Supported architectures:
- 32-bit: Intel x86, ARM, MIPS, PIC32, and PowerPC
- 64-bit: x86-64, ARM64 (AArch64)
- Static analysis of executable files with detailed information.
- Compiler and packer detection.
- Loading and instruction decoding.
- Signature-based removal of statically linked library code.
- Extraction and utilization of debugging information (DWARF, PDB).
- Reconstruction of instruction idioms.
- Detection and reconstruction of C++ class hierarchies (RTTI, vtables).
- Demangling of symbols from C++ binaries (GCC, MSVC, Borland).
- Reconstruction of functions, types, and high-level constructs.
- Integrated disassembler.
- Output in two high-level languages: C and a Python-like language.
- Generation of call graphs, control-flow graphs, and various statistics.
- Wiki (in progress)
- Botconf 2017 talk: slides, video
- REcon Montreal 2018 talk: slides
- Publications
Installation and Use
Currently, we support Windows (7 or later), Linux, macOS, and (experimentally) FreeBSD. An installed version of RetDec requires approximately 4 GB of free disk space.
Windows
- Either download and unpack a pre-built package, or build and install the decompiler by yourself (the process is described below).
- Install Microsoft Visual C++ Redistributable for Visual Studio 2015.
- Install the following programs:
- Now, you are all set to run the decompiler. To decompile a binary file named
test.exe
, run the following command (ensure thatpython
runs Python 3; as an alternative, you can trypy -3
)
For more information, runpython $RETDEC_INSTALL_DIR/bin/retdec-decompiler.py test.exe
retdec-decompiler.py
with--help
.
Linux
- Either download and unpack a pre-built package, or build and install the decompiler by yourself (the process is described below).
- After you have built the decompiler, you will need to install the following packages via your distribution's package manager:
- Now, you are all set to run the decompiler. To decompile a binary file named
test.exe
, run
For more information, run$RETDEC_INSTALL_DIR/bin/retdec-decompiler.py test.exe
retdec-decompiler.py
with--help
.
macOS
- Either download and unpack a pre-built package, or build and install the decompiler by yourself (the process is described below).
- After you have built the decompiler, you will need to install the following packages:
- Now, you are all set to run the decompiler. To decompile a binary file named
test.exe
, run
For more information, run$RETDEC_INSTALL_DIR/bin/retdec-decompiler.py test.exe
retdec-decompiler.py
with--help
.
FreeBSD (Experimental)
- There are currently no pre-built "ports" packages for FreeBSD. You will have to build and install the decompiler by yourself. The process is described below.
- After you have built the decompiler, you may need to install the following packages and execute the following command:
sudo pkg install python37 sudo ln -s /usr/local/bin/python3.7 /usr/local/bin/python3
- Now, you are all set to run the decompiler. To decompile a binary file named
test.exe
, run
For more information, run$RETDEC_INSTALL_DIR/bin/retdec-decompiler.py test.exe
retdec-decompiler.py
with--help
.
Build and Installation
This section describes a local build and installation of RetDec. Instructions for Docker are given in the next section.
Requirements
Linux
- A C++ compiler and standard C++ library supporting C++14 (e.g. GCC >= 5)
- CMake (version >= 3.6)
- Git
- Perl
- Python (version >= 3.4)
- Bison (version >= 3.0)
- Flex (version >= 2.6)
- autotools (autoconf, automake, and libtool)
- pkg-config
- m4
- zlib
- Optional: Doxygen and Graphviz for generating API documentation
apt-get
:sudo apt-get install build-essential cmake git perl python3 bison flex libfl-dev autoconf automake libtool pkg-config m4 zlib1g-dev upx doxygen graphviz
dnf
:sudo dnf install gcc gcc-c++ cmake make git perl python3 bison flex autoconf automake libtool pkg-config m4 zlib-devel upx doxygen graphviz
pacman
:sudo pacman --needed -S base-devel cmake git perl python3 bison flex autoconf automake libtool pkg-config m4 zlib upx doxygen graphviz
Windows
- Microsoft Visual C++ (version >= Visual Studio 2015 Update 2)
- CMake (version >= 3.6)
- Git
- Flex + Bison (mirror) from the Win flex-bison project. Add the extracted directory to the system
Path
(HOWTO). - Active Perl. It needs to be the first Perl in
PATH
, or it has to be provided to CMake usingCMAKE_PROGRAM_PATH
variable, e.g.-DCMAKE_PROGRAM_PATH=/c/perl/bin
. Does NOT work with Strawberry Perl or MSYS2 Perl (you would have to install a pre-built version of OpenSSL, see below).- Alternatively, you can install OpenSSL directly from here. This means OpenSSL won't be built and you don't need to install any Perl. Do not install Light version of OpenSSL as they don't contain development files.
- Python (version >= 3.4)
- Optional: Doxygen and Graphviz for generating API documentation
macOS
Packages should be preferably installed via Homebrew.
- Full Xcode installation (including command-line tools, see #425 and #433)
- CMake (version >= 3.6)
- Git
- Perl
- Python (version >= 3.4)
- Bison (version >= 3.0)
- Flex (version >= 2.6)
- autotools (autoconf, automake, and libtool)
- Optional: Doxygen and Graphviz for generating API documentation
FreeBSD (Experimental)
Packages should be installed via FreeBSDs pre-compiled package repository using the
pkg
command or built from scratch using the ports
database method.- Full "pkg" tool instructions: handbook pkg method
pkg install cmake python37 bison git autotools
OR
- Full "ports" instructions: handbook ports method
portsnap fetch
portsnap extract
- For example,
cmake
would bewhereis cmake
cd /usr/ports/devel/cmake
make install clean
Process
Note: Although RetDec now supports a system-wide installation (#94), unless you use your distribution's package manager to install it, we recommend installing RetDec locally into a designated directory. The reason for this is that uninstallation will be easier as you will only need to remove a single directory. To perform a local installation, run
cmake
with the -DCMAKE_INSTALL_PREFIX=<path>
parameter, where <path>
is directory into which RetDec will be installed (e.g. $HOME/projects/retdec-install
on Linux and macOS, and C:\projects\retdec-install
on Windows).- Clone the repository:
git clone https://github.com/avast/retdec
- Linux:
cd retdec
mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=<path>
make -jN
(N
is the number of processes to use for parallel build, typically number of cores + 1 gives fastest compilation time)make install
- Windows:
- Open a command prompt (e.g.
cmd.exe
) cd retdec
mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=<path> -G<generator>
cmake --build . --config Release -- -m
cmake --build . --config Release --target install
- Alternatively, you can open
retdec.sln
generated bycmake
in Visual Studio IDE
- Open a command prompt (e.g.
- macOS:
cd retdec
mkdir build && cd build
-
# Apple ships old Flex & Bison, so Homebrew versions should be used. export CMAKE_INCLUDE_PATH="/usr/local/opt/flex/include" export CMAKE_LIBRARY_PATH="/usr/local/opt/flex/lib;/usr/local/opt/bison/lib" export PATH="/usr/local/opt/flex/bin:/usr/local/opt/bison/bin:$PATH"
cmake .. -DCMAKE_INSTALL_PREFIX=<path>
make -jN
(N
is the number of processes to use for parallel build, typically number of cores + 1 gives fastest compilation time)make install
- FreeBSD:
sudo pkg install git cmake
git clone https://github.com/avast/retdec
cd retdec
mkdir build && cd build
-
# FreeBSD (and other BSDs) do need cmake, python3, bison, git, autotools. Flex and perl are pre-installed in the OS but check versions. # Later versions may be available for each of the packages. # See what is installed: sudo pkg info cmake python37 bison autotools # Install/upgrade them: sudo pkg install cmake python37 bison autotools
cmake .. -DCMAKE_INSTALL_PREFIX=<path>
make -jN
(N
is the number of processes to use for parallel build, typically number of cores + 1 gives fastest compilation time)make install
cmake
:-DCMAKE_INSTALL_PREFIX=<path>
to set the installation path to<path>
. Quote the path if you are using backslashes on Windows (e.g.-DCMAKE_INSTALL_PREFIX="C:\retdec"
).- (Windows only)
-G<generator>
is-G"Visual Studio 14 2015"
for 32-bit build using Visual Studio 2015, or-G"Visual Studio 14 2015 Win64"
for 64-bit build using Visual Studio 2015. Later versions of Visual Studio may be used.
cmake
:-DRETDEC_DOC=ON
to build with API documentation (requires Doxygen and Graphviz, disabled by default).-DRETDEC_TESTS=ON
to build with tests (disabled by default).-DRETDEC_DEV_TOOLS=ON
to build with development tools (disabled by default).-DRETDEC_FORCE_OPENSSL_BUILD=ON
to force OpenSSL build even if it is installed in the system (disabled by default).-DRETDEC_COMPILE_YARA=OFF
to disable YARA rules compilation at installation step (enabled by default).-DCMAKE_BUILD_TYPE=Debug
to build with debugging information, which is useful during development. By default, the project is built in theRelease
mode. This has no effect on Windows, but the same thing can be achieved by runningcmake --build .
with the--config Debug
parameter.-DCMAKE_PROGRAM_PATH=<path>
to use Perl at<path>
(probably useful only on Windows).-D<dep>_LOCAL_DIR=<path>
where<dep>
is from{CAPSTONE, ELFIO, GOOGLETEST, JSONCPP, KEYSTONE, LIBDWARF, LLVM, PELIB, RAPIDJSON, TINYXML, YARACPP, YARAMOD}
(e.g.-DCAPSTONE_LOCAL_DIR=<path>
), to use the local repository clone at<path>
for RetDec dependency instead of downloading a fresh copy at build time. Multiple such options may be used at the same time.-DRETDEC_ENABLE_<component>=ON
to build only the specified component(s) (multiple such options can be used at once), and its (theirs) dependencies. By default, all the components are built. If at least one component is enabled via this mechanism, all the other components that were not explicitly enabled (and are not needed as dependencies of enabled components) are not built. See cmake/options.cmake for all the available component options.-DRETDEC_ENABLE_ALL=ON
can be used to (re-)enable all the components.- Alternatively,
-DRETDEC_ENABLE=<comma-separated component list>
can be used instead of-DRETDEC_ENABLE_<component>=ON
(e.g.-DRETDEC_ENABLE=fileformat,loader,ctypesparser
is equivalent to-DRETDEC_ENABLE_FILEFORMAT=ON -DRETDEC_ENABLE_LOADER=ON -DRETDEC_ENABLE_CTYPESPARSER=ON
).
Build in Docker
Docker support is maintained by community. If something does not work for you or if you have suggestions for improvements, open an issue or PR.
Build Image
Building in Docker does not require installation of the required libraries locally. This is a good option for trying out RetDec without setting up the whole build toolchain.
To build the RetDec Docker image, run
docker build -t retdec - < Dockerfile
This builds the image from the master branch of this repository.To build the image using the local copy of the repository, use the development Dockerfile,
Dockerfile.dev
:docker build -t retdec:dev . -f Dockerfile.dev
Run Container
If your
uid
is not 1000, make sure that the directory containing your input binary files is accessible for RetDec:chmod 0777 /path/to/local/directory
Now, you can run the decompiler inside a container:docker run --rm -v /path/to/local/directory:/destination retdec retdec-decompiler.py /destination/binary
Note: Do not modify the /destination
part is. You only need to change /path/to/local/directory
. Output files will then be generated to /path/to/local/directory
.Automated TeamCity Builds
Our TeamCity servers are continuously generating up-to-date RetDec packages from the latest commit in the
master
branch. These are mostly meant to be used by RetDec developers, contributors, and other people experimenting with the product (e.g. testing if an issue present in the official release still exists in the current master
).You can use these as you wish, but keep in mind that there are no guarantees they will work on your system (especially the Linux version), and that regressions are a possibility. To get a stable RetDec version, either download the latest official pre-built package or build the latest RetDec version tag.
Repository Overview
This repository contains the following libraries:
ar-extractor
- library for extracting object files from archives (based on LLVM).bin2llvmir
- library of LLVM passes for translating binaries into LLVM IR modules.capstone2llvmir
- binary instructions to LLVM IR translation library.config
- library for representing and managing RetDec configuration databases.cpdetect
- library for compiler and packer detection in binaries.crypto
- collection of cryptographic functions.ctypes
- C++ library for representing C function data types.debugformat
- library for uniform representation of DWARF and PDB debugging information.demangler
- demangling library capable to handle names generated by the GCC/Clang, Microsoft Visual C++, and Borland C++ compilers.dwarfparser
- library for high-level representation of DWARF debugging information.fileformat
- library for parsing and uniform representation of various object file formats. Currently supporting the following formats: COFF, ELF, Intel HEX, Mach-O, PE, raw data.llvm-support
- set of LLVM related utility functions.llvmir-emul
- LLVM IR emulation library used for unit testing.llvmir2hll
- library for translating LLVM IR modules to high-level source codes (C, Python-like language).loader
- library for uniform representation of binaries loaded to memory. Supports the same formats as fileformat.macho-extractor
- library for extracting regular Mach-O binaries from fat Mach-O binaries (based on LLVM).patterngen
- binary pattern extractor library.pdbparser
- Microsoft PDB files parser library.stacofin
- static code finder library.unpacker
- collection of unpacking functions.utils
- general C++ utility library.
ar-extractortool
- frontend for the ar-extractor library (installed asretdec-ar-extractor
).bin2llvmirtool
- frontend for thebin2llvmir
library (installed asretdec-bin2llvmir
).bin2pat
- tool for generating patterns from binaries (installed asretdec-bin2pat
).capstone2llvmirtool
- frontend for thecapstone2llvmir
library (installed asretdec-capstone2llvmir
).configtool
- frontend for theconfig
library (installed asretdec-config
).ctypesparser
- C++ library for parsing C function data types from JSON files intoctypes
representation (installed asretdec-ctypesparser
).demangler_grammar_gen
-- tool for generating new grammars for thedemangler
library (installed asretdec-demangler-grammar-gen
).demanglertool
-- frontend for thedemangler
library (installed asretdec-demangler
).fileinfo
- binary analysis tool. Supports the same formats asfileformat
(installed asretdec-fileinfo
).idr2pat
- tool for extracting patterns from IDR knowledge bases (installed asretdec-idr2pat
).llvmir2hlltool
- frontend for thellvmir2hll
library (installed asretdec-llvmir2hll
).macho-extractortool
- frontend for themacho-extractor
library (installed asretdec-macho-extractor
).pat2yara
- tool for processing patterns to YARA signatures (installed asretdec-pat2yara
).stacofintool
- frontend for thestacofin
library (installed asretdec-stacofin
).unpackertool
- plugin-based unpacker (installed asretdec-unpacker
).
retdec-decompiler.py
- the main decompilation script binding it all together. This is the tool to use for full binary-to-C decompilations.- Support scripts used by
retdec-decompiler.py
:retdec-color-c.py
- decorates output C sources with IDA color tags - syntax highlighting for IDA.retdec-config.py
- decompiler's configuration file.retdec-archive-decompiler.py
- decompiles objects in the given AR archive.retdec-fileinfo.py
- a Fileinfo tool wrapper.retdec-signature-from-library-creator.py
- extracts function signatures from the given library.retdec-unpacker.py
- tries to unpack the given executable file by using any of the supported unpackers.retdec-utils.py
- a collection of Python utilities.
retdec-tests-runner.py
- run all tests in the unit test directory.type_extractor
- generation of type information (for internal use only)
Project Documentation
See the project documentation for an up to date Doxygen-generated software reference corresponding to the latest commit in the
master
branch.Related Repositories
- retdec-idaplugin -- Embeds RetDec into IDA (Interactive Disassembler) and makes its use much easier.
- retdec-regression-tests-framework -- A framework for writing and running regression tests for RetDec and related tools. This is a must if you plan to contribute to the RetDec project.
- retdec-regression-tests -- A suite of regression tests for RetDec and related tools.
- vim-syntax-retdecdsm -- Vim syntax-highlighting file for the output from the RetDec's disassembler (
.dsm
files).
License
Copyright (c) 2017 Avast Software, licensed under the MIT license. See the
LICENSE
file for more details.RetDec uses third-party libraries or other resources listed, along with their licenses, in the
LICENSE-THIRD-PARTY
file.Contributing
See RetDec contribution guidelines.
Acknowledgements
This software was supported by the research funding TACR (Technology Agency of the Czech Republic), ALFA Programme No. TA01010667.
RetDec - A Retargetable Machine-Code Decompiler Based On LLVM
Reviewed by Zion3R
on
4:50 PM
Rating: