microsoft/cyberbattlesimAn experimentation and research platform to investigate the interaction of automated agents in an abstract simulated network environments.
April 8th, 2021: See the announcement on the Microsoft Security Blog.
CyberBattleSim is an experimentation research platform to investigate the interaction of automated agents operating in a simulated abstract enterprise network environment. The simulation provides a high-level abstraction of computer networks and cyber security concepts. Its Python-based Open AI Gym interface allows for the training of automated agents using reinforcement learning algorithms.
The simulation environment is parameterized by a fixed network topology and a set of vulnerabilities that agents can utilize to move laterally in the network. The goal of the attacker is to take ownership of a portion of the network by exploiting vulnerabilities that are planted in the computer nodes. While the attacker attempts to spread throughout the network, a defender agent watches the network activity and tries to detect any attack taking place and mitigate the impact on the system by evicting the attacker. We provide a basic stochastic defender that detects and mitigates ongoing attacks based on pre-defined probabilities of success. We implement mitigation by re-imaging the infected nodes, a process abstractly modeled as an operation spanning over multiple simulation steps.
To compare the performance of the agents we look at two metrics: the number of simulation steps taken to attain their goal and the cumulative rewards over simulation steps across training epochs.
We view this project as an experimentation platform to conduct research on the interaction of automated agents in abstract simulated network environments. By open-sourcing it, we hope to encourage the research community to investigate how cyber-agents interact and evolve in such network environments.
The simulation we provide is admittedly simplistic, but this has advantages. Its highly abstract nature prohibits direct application to real-world systems thus providing a safeguard against potential nefarious use of automated agents trained with it. At the same time, its simplicity allows us to focus on specific security aspects we aim to study and quickly experiment with recent machine learning and AI algorithms.
For instance, the current implementation focuses on the lateral movement cyber-attacks techniques, with the hope of understanding how network topology and configuration affects them. With this goal in mind, we felt that modeling actual network traffic was not necessary. This is just one example of a significant limitation in our system that future contributions might want to address.
On the algorithmic side, we provide some basic agents as starting points, but we would be curious to find out how state-of-the-art reinforcement learning algorithms compare to them. We found that the large action space intrinsic to any computer system is a particular challenge for Reinforcement Learning, in contrast to other applications such as video games or robot control. Training agents that can store and retrieve credentials is another challenge faced when applying RL techniques where agents typically do not feature internal memory. These are other areas of research where the simulation could be used for benchmarking purposes.
Other areas of interest include the responsible and ethical use of autonomous cyber-security systems: How to design an enterprise network that gives an intrinsic advantage to defender agents? How to conduct safe research aimed at defending enterprises against autonomous cyber-attacks while preventing nefarious use of such technology?
Read the Quick introduction to the project.
| Type | Branch | Status |
|---|---|---|
| CI | master |  |
| Docker image | master |  |
See Benchmark documentation. Jupyter notebooks with the latest runs are checked-in under a dedicated tag at /notebooks/benchmarks (latest_benchmark).
It is strongly recommended to work under a Linux environment, either directly or via WSL on Windows. Running Python on Windows directly should work but is not supported anymore.
Start by checking out the repository:
git clone https://github.com/microsoft/CyberBattleSim.git
If you get the following error when running the papermill on the notebooks
(or alternatively when running orca --help)
/home/wiblum/miniconda3/envs/cybersim/lib/orca_app/orca: error while loading shared libraries: libXss.so.1: cannot open shared object file: No such file or directory
or other share libraries like libgdk_pixbuf-2.0.so.0,
Then run the following command:
sudo apt install libnss3-dev libgtk-3-0 libxss1 libasound2-dev libgtk2.0-0 libgconf-2-4
The instructions were tested on a Linux Ubuntu distribution (both native and via WSL).
If conda is not installed already, you need to install it by running the install_conda.sh script.
bash install-conda.sh
Once this is done, open a new terminal and run the initialization script:
bash init.sh
This will create a conda environmen named cybersim with all the required OS and python dependencies.
To activate the environment run:
conda activate cybersim
The supported dev environment on Windows is via WSL. You first need to install an Ubuntu WSL distribution on your Windows machine, and then proceed with the Linux instructions (next section).
To authenticate with Git, you can either use SSH-based authentication or alternatively use the credential-helper trick to automatically generate a PAT token. The latter can be done by running the following command under WSL (more info here):
git config --global credential.helper "/mnt/c/Program\ Files/Git/mingw64/libexec/git-core/git-credential-manager.exe"
To run your environment within a docker container, we recommend running docker via Windows Subsystem on Linux (WSL) using the following instructions:
Installing Docker on Windows under WSL).
This method is not maintained anymore, please prefer instead running under
a WSL subsystem Linux environment.
But if you insist you want to start by installing Python 3.10 then in a Powershell prompt run the ./init.ps1 script.
The quickest method to get up and running is via the Docker container.
NOTE: For licensing reasons, we do not publicly redistribute any build artifact. In particular, the docker registry
spinshot.azurecr.ioreferred to in the commands below is kept private to the project maintainers only.As a workaround, you can recreate the docker image yourself using the provided
Dockerfile, publish the resulting image to your own docker registry and replace the registry name in the commands below.