Standalone SIGMA-Based Detection Tool for EVTX, Auditd, Sysmon for Linux, XML, CSV, or JSONL/NDJSON Logs

Zircolite is a standalone tool written in Python 3 that allows you to use SIGMA rules on:

• MS Windows EVTX (EVTX, XML, and JSONL formats)
• Auditd logs
• Sysmon for Linux
• EVTXtract
• CSV and XML logs
• JSON Array logs

Key Features

• Automatic Log Type Detection: Automatically identifies log formats and timestamp fields using magic bytes, content analysis, and regex-based fallback -- no need to specify format flags in most cases.
• Multiple Input Formats: Supports various log formats including EVTX, JSON Lines, JSON Arrays, CSV, XML, and more. Compressed or archived logs (gzip, bzip2, ZIP, 7-Zip) are supported; use --archive-password for encrypted ZIP/7z.
• Native Sigma Support: Zircolite can directly use native Sigma rules (YAML) by converting them with pySigma.
• SIGMA Backend: It is based on a SIGMA backend (SQLite) and does not use internal SIGMA-to-something conversion.
• Advanced Log Manipulation: It can manipulate input logs by splitting fields and applying transformations, allowing for more flexible and powerful log analysis.
• Field Transforms: Apply custom Python transformations to fields during processing (e.g., Base64 decoding, hex-to-ASCII conversion).
• Flexible Export: Zircolite can export results to multiple formats using Jinja templates, including JSON, CSV, JSONL, Splunk, Elastic, OpenSearch, Timesketch, SARIF, ATT&CK Navigator, and more.
• Rich Terminal Output: Detection results displayed in severity-sorted tables with MITRE ATT&CK technique IDs, ATT&CK tactics heatmap, rule coverage metrics, and clickable output file links.

You can use Zircolite directly with Python.

Documentation is available here (dedicated site) or here (repository directory).

Requirements / Installation

The project has been tested with Python 3.10 and above. Install dependencies with: pip3 install -r requirements.txt.

Dependencies

• Required: orjson, xxhash, rich, rich-argparse, RestrictedPython, requests, urllib3, pySigma, evtx (pyevtx-rs), jinja2, lxml, chardet, psutil, pyyaml, py7zr
• py7zr is imported only when a input is opened; ZIP, gzip and bzip2 use the standard library.

⚠️ On some systems (Mac, ARM, etc.), the evtx Python library may require Rust and Cargo to be installed.

Quick Start

Check out (old) tutorials made by others (EN, ES, and FR) here.

EVTX Files

Help is available with:

python3 zircolite.py -h

If your EVTX files have the extension ".evtx":

# python3 zircolite.py --evtx <EVTX FOLDER or EVTX FILE> --ruleset <SIGMA RULESET> [--ruleset <OTHER RULESET>]
python3 zircolite.py --evtx sysmon.evtx --ruleset rules/rules_windows_merged.json

Using Native Sigma Rules (YAML)

You can use native Sigma rules (YAML) directly:

# Single YAML rule
python3 zircolite.py --evtx sample.evtx --ruleset path/to/rule.yml

# Directory of Sigma rules
python3 zircolite.py --evtx sample.evtx --ruleset ./sigma/rules/windows/process_creation

# With pySigma pipelines
python3 zircolite.py --evtx sample.evtx --ruleset rule.yml --pipeline sysmon --pipeline windows-logsources

Other Log Formats

Zircolite auto-detects the log format in most cases, so explicit format flags are optional:

# Auto-detection (recommended) - Zircolite identifies the format automatically
python3 zircolite.py --events auditd.log --ruleset rules/rules_linux.json
python3 zircolite.py --events sysmon.log --ruleset rules/rules_linux.json
python3 zircolite.py --events <JSON_FOLDER_OR_FILE> --ruleset rules/rules_windows_merged.json

# Explicit format flags (override auto-detection)
python3 zircolite.py --events auditd.log --ruleset rules/rules_linux.json --auditd
python3 zircolite.py --events sysmon.log --ruleset rules/rules_linux.json --sysmon4linux
python3 zircolite.py --events <JSON_FOLDER_OR_FILE> --ruleset rules/rules_windows_merged.json --jsononly
python3 zircolite.py --events <JSON_FOLDER_OR_FILE> --ruleset rules/rules_windows_merged.json --json-array
python3 zircolite.py --events <CSV_FOLDER_OR_FILE> --ruleset rules/rules_windows_merged.json --csv-input
python3 zircolite.py --events <XML_FOLDER_OR_FILE> --ruleset rules/rules_windows_merged.json --xml-input

• The --events argument can be a file or a folder. If it is a folder, all log files in the current folder and subfolders will be selected (use --no-recursion to disable).
• Use --file-pattern to specify a custom glob pattern for file selection.
• Use --no-auto-detect to disable automatic format detection.

[!TIP] If you want to try the tool, you can test with EVTX-ATTACK-SAMPLES (EVTX files).

Running with Docker

# Pull the Docker image
docker pull wagga40/zircolite:latest
# If your logs and rules are in a specific directory
docker run --rm --tty \
    -v $PWD:/case/input:ro \
    -v $PWD:/case/output \
    wagga40/zircolite:latest \
    -e /case/input \
    -o /case/output/detected_events.json \
    -r /case/input/a_sigma_rule.yml

• Replace $PWD with the directory (absolute path only) where your logs and rules/rulesets are stored.

Automatic Processing Optimization

Given several files, Zircolite measures them against available RAM and CPU, picks a database mode (one shared database, or one per file) and decides whether processing them in parallel is worth it — then adapts the worker count to memory pressure as it runs.

python3 zircolite.py --evtx ./logs/ --ruleset rules/rules_windows_merged.json

Override any of it with --no-auto-mode, --unified-db (one database for all files, which is what cross-file correlation rules need), --no-parallel or --parallel-workers N. See Automatic Processing Optimization for how the choice is made.

Using YAML Configuration Files

For complex or repeated analysis workflows, use a YAML configuration file:

# Generate a fully commented configuration file
python3 zircolite.py --generate-config my_config.yaml

# Run with it
python3 zircolite.py --yaml-config my_config.yaml

# CLI arguments override the file
python3 zircolite.py --yaml-config my_config.yaml --evtx ./other_logs/

The generated file documents every supported key; config/zircolite_example.yaml is a worked example. See YAML configuration for the merge rules and the options that have no YAML equivalent.

Updating Default Rulesets

python3 zircolite.py -U

Alternatively, if you use Task (go-task), run task update-rules from the project root to update rules from Zircolite-Rules-v2. See docs for other tasks (Docker build, clean, etc.).

[!IMPORTANT]
Please note that these rulesets are provided to use Zircolite out of the box, but you should generate your own rulesets as they can be noisy or slow. These auto-updated rulesets are available in the dedicated repository: Zircolite-Rules-v2.

Field Splitting and Transforms

Two configuration features shape events as they are ingested, both in config/config.yaml:

• Field splitting turns a packed key-value field into queryable ones. Sysmon's Hashes field (SHA1=abc123,MD5=def456,SHA256=789xyz) becomes separate SHA1, MD5 and SHA256 fields, so rules can match a hash directly.
• Field transforms run sandboxed Python over a field's value — decoding base64 command lines, extracting IOCs, flagging LOLBins — and can write the result to a new field rather than replacing the original. Zircolite ships 55 of them across 11 categories, off by default apart from the two auditd ones.

split:
  Hashes:
    separator: ","
    equal: "="

See Field Splitting and Field Transforms for the full configuration, the transforms Zircolite ships, and how to test your own.

Documentation

Complete documentation is available here.

Mini-GUI

The Mini-GUI can be used completely offline. It allows you to display and search results. You can automatically generate a Mini-GUI "package" with the --package option. Use --package-dir to specify the output directory. To learn how to use the Mini-GUI, check the documentation here.

Detected Events by MITRE ATT&CK® Techniques and Criticality Levels

Detected Events Timeline

Detected Events by MITRE ATT&CK® Techniques Displayed on the Matrix

Tutorials, References, and Related Projects

Tutorials

• English: Russ McRee has published a detailed tutorial on SIGMA and Zircolite on his blog.

• Spanish: César Marín has published a tutorial in Spanish here.

• French: IT-connect.fr has published an extensive tutorial on Zircolite in French.

• French: IT-connect.fr has also published a Hack the Box challenge write-up using Zircolite.

References

• Florian Roth cited Zircolite in his SIGMA Hall of Fame during his talk at the October 2021 EU ATT&CK Workshop.
• Zircolite has been cited and presented during JSAC 2023.
• Zircolite has been cited and used in multiple research papers:
  • CIDRE Team:
    • PWNJUTSU - Website
    • PWNJUTSU - Academic Paper
    • CERBERE: Cybersecurity Exercise for Red and Blue Team Entertainment, Reproducibility
  • Universidad de la República:
    • A Process Mining-Based Method for Attacker Profiling Using the MITRE ATT&CK Taxonomy

License

• All the code of the project is licensed under the GNU Lesser General Public License.
• evtx_dump is under the MIT license.
• The rules are released under the Detection Rule License (DRL) 1.0.