We're exploring a managed SaaS solution for dnsmonster! Help shape the future of passive DNS monitoring by sharing your feedback and requirements: Take our quick survey

Table of Contents

• Dnsmonster
• Main features
• Installation
  • Linux
    • Container
    • Build manually
    • Build statically
  • Windows
  • FreeBSD and MacOS
• Architecture
  • All-in-one Installation using Docker
    • All-in-one Demo
  • Enterprise Deployment
• Configuration
  • Command line options
  • Environment variables
  • Configuration file
  • What's the retention policy
• Sampling and Skipping
  • pre-process sampling
  • skip domains
  • allow domains
  • SAMPLE in clickhouse SELECT queries
• Supported Inputs
• Supported Outputs
• Roadmap
• Related projects

Dnsmonster

Passive DNS monitoring framework built on Golang. dnsmonster implements a packet sniffer for DNS traffic. It can accept traffic from a pcap file, a live interface or a dnstap socket, and can be used to index and store hundreds of thousands of DNS queries per second as it has shown to be capable of indexing 200k+ DNS queries per second on a commodity computer. It aims to be scalable, simple, and easy to use, and help security teams to understand the details about an enterprise's DNS traffic. dnsmonster doesn't look to follow DNS conversations, rather it aims to index DNS packets as soon as they come in. It also doesn't aim to breach the privacy of the end-users, with the ability to mask Layer 3 IPs (IPv4 and IPv6), enabling teams to perform trend analysis on aggregated data without being able to trace back the queries to an individual. Blogpost

The code before version 1.x is considered beta quality and is subject to breaking changes. Please visit the release notes for each tag to see the list of breaking scenarios between each release, and how to mitigate potential data loss.

graph TD
    subgraph Input
        B1["network input"]
        B2["pcap file"]
        B3["dnstap socket"]
    end
    
    subgraph "Process"
        C1["Sampling based of ratio"]
        C2["Packet Process"]
        C3["Dispatcher"]
        O11["Output1"]
        O12["Domain Skip (optional)"]
        O13["Domain Allow (optional)"]
        O21["Output2"]
        O22["Domain Skip (optional)"]
        O23["Domain Allow (optional)"]
        O31["Output3"]
        O32["Domain Skip (optional)"]
        O33["Domain Allow (optional)"]
    end
    
    B1 --> Process
    B2 --> Process
    B3 --> Process
    
    C1 --> C2
    C2 --> C3
    C3 --> O11
    C3 --> O21
    C3 --> O31
    
    O11 --> O12 --> O13
    O21 --> O22 --> O23
    O31 --> O32 --> O33
    
    subgraph Output
        Splunk
        Syslog
        H["ClickHouse"]
        Postgres
        Kafka
        I["JSON File"]
        Influx
        Elastic
        J["stdout"]
        Parquet
        Sentinel
    end
    
    O13 --> H
    O23 --> I
    O33 --> J

Main features

• Ability to use Linux's afpacket and zero-copy packet capture.
• Supports BPF
• Ability to mask IP address to enhance privacy
• Ability to have a pre-processing sampling ratio
• Ability to have a list of "skip" fqdns to avoid writing some domains/suffix/prefix to storage
• Ability to have a list of "allow" domains, used to log access to certain domains
• Hot-reload of skip and allow domain files/urls
• Modular output with configurable logic per output stream.
• Automatic data retention policy using ClickHouse's TTL attribute
• Built-in Grafana dashboard for ClickHouse output.
• Ability to be shipped as a single, statically linked binary
• Ability to be configured using environment variables, command line options or configuration file
• Ability to sample outputs using ClickHouse's SAMPLE capability
• Ability to send metrics using prometheus and statstd
• High compression ratio thanks to ClickHouse's built-in LZ4 storage
• Supports DNS Over TCP, Fragmented DNS (udp/tcp) and IPv6
• Supports dnstap over Unix socket or TCP
• built-in SIEM integration with Splunk and Microsoft Sentinel

Installation

Linux

Best way to get started with dnsmonster is to download the binary from the release section. The binary is statically built against musl, hence it should work out of the box for many distros. For afpacket support, you must use kernel 3.x+. Any modern Linux distribution (CentOS/RHEL 7+, Ubuntu 14.0.4.2+, Debian 7+) is shipped with a 3.x+ version so it should work out of the box. If your distro isn't working with the pre-compiled version, please submit an issue with the details, and build dnsmonster manually using this section Build Manually.

Container

Since dnsmonster uses raw packet capture funcationality, Docker/Podman daemon must grant the capability to the container

sudo docker run --rm -it --net=host --cap-add NET_RAW --cap-add NET_ADMIN --name dnsmonster ghcr.io/mosajjal/dnsmonster:latest --devName lo --stdoutOutputType=1

Build manually

• with libpcap: Make sure you have go, libpcap-devel and linux-headers packages installed. The name of the packages might differ based on your distribution. After this, simply clone the repository and run go build ./cmd/dnsmonster

git clone https://github.com/mosajjal/dnsmonster --depth 1 /tmp/dnsmonster 
cd /tmp/dnsmonster
go get
go build -o dnsmonster ./cmd/dnsmonster

• without libpcap: dnsmonster only uses one function from libpcap, and that's converting the tcpdump-style filters into BPF bytecode. If you can live with no BPF support, you can build dnsmonster without libpcap. Note that for any other platform, the packet capture falls back to libpcap so it becomes a hard dependency (*BSD, Windows, Darwin)

git clone https://github.com/mosajjal/dnsmonster --depth 1 /tmp/dnsmonster 
cd /tmp/dnsmonster
go get
go build -o dnsmonster -tags nolibpcap ./cmd/dnsmonster

The above build also works on ARMv7 (RPi4) and AArch64.

Build statically

If you have a copy of libpcap.a, you can build the statically link it to dnsmonster and build it fully statically. In the code below, please change /root/libpcap-1.9.1/libpcap.a to the location of your copy.

git clone https://github.com/mosajjal/dnsmonster --depth 1 /tmp/dnsmonster
cd /tmp/dnsmonster/
go get
go build --ldflags "-L /root/libpcap-1.9.1/libpcap.a -linkmode external -extldflags \"-I/usr/include/libnl3 -lnl-genl-3 -lnl-3 -static\"" -a -o dnsmonster ./cmd/dnsmonster

For more information on how the statically linked binary is created, take a look at this Dockerfile.