josevnz/suricatalogParse, filter, and visualize Suricata eve.json logs with CLI tools for alerts, flows, DNS, and payloads. Includes a tutorial for learning Suricata and testing rules.
# SuricataLog [](https://pepy.tech/project/suricatalog) When I started learning how to use [Suricata](https://suricata.io/) quickly found that I needed a tool to inspect the eve.json file; Most of the tutorials and documentation out there suggested installing a stack to do the following tasks: 1. Store the logs in a central location 2. Normalize and enrich the events, specially alerts 3. Use a frontend to dive into the data Which is very useful, but what if I just needed to do a quick inspection of the events? Sooner or later you will get [bored to death](https://suricata.readthedocs.io/en/suricata-6.0.0/output/eve/eve-json-examplesjq.html) doing this: ```shell cat eve.json | jq -r -c 'select(.event_type=="alert")|.payload'| base64 --decode ``` SuricataLog is a set of tools/ scripts to parse and display Suricata log files (like /var/log/suricata/eve.json) The [Eve JSON format](https://suricata.readthedocs.io/en/suricata-6.0.0/output/eve/eve-json-format.html) is not very complex, so I wrote few scripts with the features I tough would be more useful for my home network analysis. As a bonus, I wrote my learning experience as a [tutorial](https://github.com/josevnz/suricatalog/blob/dependabot/uv/uv-1176edd47e/TUTORIAL.md) that you can use to learn about Suricata and also how to test it. ## Installing from Pypi ### Installing from Pypi.org ```shell uv pip install SuricataLog ``` ### Installing from source ```shell git clone [email protected]:josevnz/SuricataLog.git cd SuricataLog uv build uv pip install dist/suricatalog-1.1.6-py3-none-any.whl ``` ### Developer installation So you want to contribute? Or found a bug and think you can submit a patch? Nice! Here is what you can dd to run on development mode: ```shell git clone [email protected]:josevnz/SuricataLog.git cd SuricataLog uv build uv sync --dev uv pip install --editable . ``` Running unit tests is very easy after that: ```shell (SuricataLog) [josevnz@dmaf5 SuricataLog]$ python -m unittest test/*.py ......... ---------------------------------------------------------------------- Ran 9 tests in 0.334s OK ``` If the unit tests fails, then this is most likely the first place to fix a problem. I do recommend also running the textualize console and watch for the console messages: ```shell # textual console textual console --exclude SYSTEM --exclude EVENT ``` Then on another terminal: ```shell textual run --dev --command eve_log --timestamp '2015-01-01 10:41:21.642899' test/eve.json ``` That will print even debug messages on the console (you can exclude these if you restart the console with '--exclude DEBUG') ### Creating a Docker image Please check [DOCKER.md](https://github.com/josevnz/suricatalog/blob/dependabot/uv/uv-1176edd47e/DOCKER.md) for more details. ## Running the scripts Once everything is installed in your virtual environment you should be able to call the scripts You can find out what applications got installed by using auto complete, after activating your virtual environment: ```shell eve_[tab][tab] ``` And if you install the Bash auto complete extension you will also get some suggestions for the flags. ### Simple EVE log parser Better see it by yourself (remember, use --help to learn what options are supported) #### Table format:  ````shell eve_log --timestamp '2015-01-01 10:41:21.642899' --formats TABLE test/eve.json ```` ### Canned reports with eve_json.py ```shell (suricatalog) [josevnz@dmaf5 SuricataLog]$ eve_json --help usage: eve_json [-h] [--nxdomain | --payload | --flow | --netflow NETFLOW | --useragent] eve [eve ...] This script is inspired by the examples provided on [15.1.3. Eve JSON ‘jq’ Examples](https://suricata.readthedocs.io/en/suricata-6.0.0/output/eve/eve-json- examplesjq.html) A few things: * The output uses colorized JSON positional arguments: eve Path to one or more /var/log/suricata/eve.json file to parse. optional arguments: -h, --help show this help message and exit --nxdomain Show DNS records with NXDOMAIN --payload Show alerts with a printable payload --flow Aggregated flow report per protocol and destination port --netflow NETFLOW Get the netflow for a given IP address --useragent Top user agent in HTTP traffic ``` Take a look at some examples below: #### NXDOMAIN  ```shell eve_json --nxdomain test/eve.json ``` #### PAYLOAD  ```shell eve_json --payload ~/Downloads/eve.json ``` #### FLOW  ```shell eve_json --flow test/eve_udp_flow.json ``` #### NETFLOW  ```shell eve_json --netflow 224.0.0.251 test/eve_udp_flow.json ``` #### USERAGENT  ```shell eve_json --useragent test/eve.json ``` ### Running eve_* applications on a browser  You can run Suricata Log applications on a browser by using the `eve_server` wrapper: ```shell # Show the flow report on a eve.json file eve_server --application eve_json -- --flow ~/eve.json # Show NX domain report eve_server --application eve_json -- --nxdomain ~/eve.json # Inspect the eve.json records eve_server --applications eve_log -- ~/eve.json ``` You need to pass the '--' to tell the server than this options belong to the underlying eve_* you want to call. ## Running from Docker It is also possible to run SuricataLog from a Docker container. Please see the [DOCKER.md](https://github.com/josevnz/suricatalog/blob/dependabot/uv/uv-1176edd47e/DOCKER.md) for more details ## Running with uv If you have uv, you can just do this: ```shell uvx --from SuricataLog eve_log /var/log/suricata/eve.json uvx --from SuricataLog eve_json --useragent /var/log/suricata/eve.json uvx --from SuricataLog eve_server --application eve_json -- --flow /var/log/suricata/eve.json ``` You get the idea. ## Bash Auto complete I tried my best to follow [8.6 Programmable Completion](https://www.gnu.org/software/bash/manual/html_node/Programmable-Completion.html#Programmable-Completion) I provided auto complete for the most common flags. You can figure out all the flags by passing '--help' to any of the scripts. ### Installation of auto complete commands You can install Bash autocomplete for all the SuricataLog scripts. Just run the 'eve_autocomplete' and follow directions: ```shell eve_autocomplete --help ``` ## Supported versions I work on this project on **my spare time** and I cannot support every version of Linux/ Python combination out there. This is my current test bed, and it may change without further notice | SuricataLog | Supported | OS | Python | Suricata | |-------------|-----------|----------------------------------|-----------|----------| | <= 0.8 | NO | NA | < 3.8 | 6.04 | | 0.9+ | NO | fedora 37 | => 3.11.4 | 6.04 | | 0.9+ | NO | Armbian 23.02.2 Jammy | => 3.10.6 | 6.04 | | 0.9+ | NO | Ubuntu 20.04.4 LTS (Focal Fossa) | => 3.8.10 | 6.04 | | 1.0.3+ | YES | fedora 40 | => 3.11.4 | 7.0.6 | *You are more than welcome to*: * Submit patches with new features and bug-fixes. * Open bug reports. Be as detailed as possible, otherwise I will have no choice but to close it. ## Tutorials * [Suricata, RaspberryPI4 and Python to secure your home network](https://github.com/josevnz/suricatalog/blob/dependabot/uv/uv-1176edd47e/TUTORIAL.md) * [Using SuricataLog to analyze your events](https://github.com/josevnz/suricatalog/blob/dependabot/uv/uv-1176edd47e/Using_SuricataLog_to_analyze_your_events.md)
