Data from a BRAWL Automated Adversary Emulation Exercise
One of the challenging problems for cyber security researchers developing detection and response capabilities is finding a realistic environment in which to test their hypothesis and capabilities.
The cheapest method is to test capabilities on a small lab network. But this environment is lacking the scale of real enterprise network and the noise of real environments that makes detection much harder. In many ways, the best environment would be testing on multiple enterprise scale networks with a controlled but realistic attacker and real noise from users, system administrators, and third party software/devices. The challenge with testing in this environment is that it is expensive and in some scenarios high risk.
BRAWL seeks to create a compromise by creating a system to automatically create an enterprise network inside a cloud environment. OpenStack is the only currently supported environment, but it is being designed in such a way as to easily support other cloud environments in the future. BRAWL also builds an analysis network containing a data ingest and processing pipeline using LogStash and Kafka. As part of the analysis network, it creates an event storage and search system using Elasticsearch and Kibana. BRAWL spins up a enterprise network "Game Board" with Windows images. These images have Microsoft Sysmon and other sensors already installed and configured to forward logs into the data ingest framework.
BRAWL also has a concept of bots, which can be either Red, Blue, or Gray. Red bots are offensive, Blue bots are defensive, and Gray bots emulate legitimate user behavior in order to provide noise to make detection more difficult. When a user wants to test research hypotheses, they implement a BRAWL bot. The BRAWL bot registers itself with the BRAWL Controller, which then orchestrates games between BRAWL bots on the Game Board.
Note: Due to issues with file sizes and GitHub quotas, we are placing all the files into a zip file instead of leaving them as plain text in the git repo. All the data is in the file
This release consists of some data from a BRAWL prototype. We created a small enterprise network, described below. We then ran a single game using the MITRE CALDERA research project as a red bot.
CALDERA is a related MITRE research project that automates adversary emulation activity based on the information in Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK) model. It implements a set of ATT&CK tactics and techniques and uses a planning system (https://dl.acm.org/citation.cfm?id=2991111) to automate the actuation of those techniques and generate post-compromise adversary behavior within an enterprise network.
This data is released under the Creative Commons BY License
Our small enterprise network is a flat network that consists of a Domain Controller (dc.brawlco.com) and 16 workstations. Each PC has the name of the primary user in the pc name (e.g. user beane typically logs into beane-pc). That user has Local Administrator privileges on the computer.
All the PCs are running Windows 8.1. The Domain controller is running Windows Server 2012 R2.
On the Windows 8 PCs, we made changes to enable WDigest to keep plaintext passwords in LSASS's memory using the following registry command: reg ADD HKLM\SYSTEM\CurrentControlSet\Control\SecurityProviders\WDigest\ /v UseLogonCredential /t REG_DWORD /d 1 /F
For this exercise, CALDERA was the only BRAWL bot participating. While conceptually BRAWL can be used to test a variety of attacker behaviors and detection, many of MITRE's research efforts follow an "assume breach" philosophy. Therefore we give CALDERA a starting point as a Local Administrator on a box on the network at the beginning of the exercise.
Also, without a Grey bot performing logon across different hosts, the BRAWL Game Board is sterile from the perspective of credentials that can be stolen and used by Red bots. To enable lateral movement, the BRAWL Controller uses psexec to create logon events on hosts with the credentials of other users from the network.
CALDERA Performed the following ATT&CK Techniques during the exercise:
There are five types of data in this repository. Each is contained in its own file in the data/ folder.
| Data Type | Description |
|---|---|
| game_metadata | Data describing the BRAWL scenario |
| sysmon | Data gathered from Sysmon running on each of the workstations |
| win_event | Windows Event Logs |
| computer_properties | Data gathered from custom scripts that provides some information about the computers in the network |
| bsf | Red bot actions in BRAWL Shared Format (BSF) |
Red bots and blue bots are encouraged to log information about their activities or detections in BRAWL Shared Format (BSF). The goal of this is to make it easier to compare blue bot detection/actions with red bot actions.
The format is currently in development and could change in future data sets.
The fields for BSF are described below in the Data Sources Detail section.
Different event sources in BRAWL handle time differently. The time is either the time that an event hit our logging ingest framework, the time the event was generated on the host/endpoint, or the time recorded by a bot on the network. In general those times should be within a few milliseconds of each other. When possible the logging ingest framework uses the event time stored in the event instead of the time of the event hitting the ingest nodes. The table below details the method used for each data type.
| Data Source | Time Notes |
|---|---|
| computer_properties | from the time field |
| game_metadata | time it hits the ingest framework |
| sysmon | from the utc_time field |
| win_event | pulled from the windows event time |
| bsf | The @timestamp field is time it hits the ingest framework. However the time related BSF fields (e.g. happened_after,happened_before, etc) are the times the events started or ended based on the time on CALDERA's command and control server. |
