isnic/proberAutomated, reproducible network security testing framework using Ansible and BATS to verify DNS, host availability, open ports, and TLS configuration from multiple vantage points.
This project uses Ansible to generate BATS test files verifying security assuptions about network infrastructure. Tests are run from probe machines ("prober nodes") to which Prober is deployed.
This tool is meant for reproducible, automated testing of own networks. It supports multiple prober nodes running tests, each with a different view of the probed network — for example, a view from the external zone, and from an internal zone, and a view from inside the DMZ.
Using Prober to probe networks you are not authorized to might be illegal.
On Ansible master used to deploy tests to prober nodes:
ansible (duh!)python*-netaddr (on GNU/Linux) / py*-netaddr (on FreeBSD)On prober nodes themselves:
bashsshnmapdig/kdigtestsslOn a system you're reviewing the results (*.tap files), you might want to install tappy. Results are kept on each prober node, in local git repositories (by default, in /var/run/prober/results/, see below for configuration options), with commits tagged with the date of each scanning run finished on, for historical record and easy comparison.
Ansible is used to generate tests to prober nodes. A prober node can be any FreeBSD or GNU/Linux host, as long as it is possible to install Prober's dependencies. It does not need to be dedicated to testing, but it is recommended — a scan of a reasonably-sized network will take hours, use a lot of CPU, and generate quite a lot of traffic.
It makes sense to deploy Prober on a number of different machines to check how your network looks from different vantage points (for example: internal network, DMZ, the Internet).
In the test directory on prober nodes a run-tests.sh script is created in order to facilitate running the tests. Tests are run in a way to ensure <simultaneous_tests_count> tests are running simultaneously at all times — this variable is the main way of controlling how much resources a probing scan uses and how much bandwidth it needs.
Test results are then saved in a local git repository, with commits tagged with the date a given run finished on (which might be different than the start date for some long probing runs).
If dealing with an infrastructure larger than just a few hosts, it might make sense to use a tool like Mitogen to dramatically speed up test generation.
Prepare a Debian or FreeBSD server for use as a prober node (let's call it prober.example.com), make sure you have SSH access to it and can sudo on it.
Copy the example inventory as inventories/test/.
In that test inventory:
group_vars/all.yml and set zone_nameserver, zone_domains, address_blocks to your liking; let's assume you add example.com as the only domain one to be probedhosts file, configuring your prober.example.com prober node in [prober-nodes] section; let's say you set the tested_zone to test.Copy the example zone config as data/<tested_zone>.yml (so in our case: data/test.yml) and edit it; at minimum, the tested_zone_settings.name key must contain the tested_zone (so, test).
Export your DNS zone and save it as data/<dns_zone>.zone (in our case: example.com).
Run the playbook:
ansible-playbook prober.yml -i inventories/test/ -vvv
This will generate the tests on the prober node and set up a cronjob to run them every night at 01:00 AM
You can now ssh into the prober node and inspect the tests generated in /opt/prober/. Once you are satisfied, you can run them by executing: /opt/prober/run_tests.sh. Results will be saved in /var/run/prober/results.
Configuration variables (defined in probers.yml):
tests_directory (default: /opt/prober):
directory on the prober nodes where the test files (*.bats files) are generated to.
results_repo_directory (default: /var/run/prober/results)
directory of the repository for test results (*.tap files); a git repository is
initialized there and a tap/ subdirectory created for the actual results;
results are committed to the git repository, commits are tagged with a date
in yyyy-mm-dd format (the commit with results of a test that finished March 19th, 2020
is tagged as 2020-03-19, for instance).
simultaneous_tests_count (default: 20):
how many tests are run simultaneously.
prober_dev (default: undefined):
skip certain tasks that don't make sense on a dev machine
(like setting up cron or zabbix).
Zones are defined in ./data/<zone_name>.yml files with the apex key being the string "tested_zone_settings", which then contains keys:
name: the name of the zone, matching the filename (without extension); for example: "internal", "external", "dmz"default (optional): the default settings for all hosts in this zone^" character) matching multiple FQDNsThe default key, each regex key, and each domain key in turn can contain these keys:
resolve: should the host resolve to the relevant IP address (bool true/false, or the string "skip")ping: should the host respond to ping (bool true/false, or the string "skip")ports: list of TCP and UDP ports allowed to be open (sub-keys "tcp" and "udp", each containing an array of integers defining ports allowed to be open, or the string "skip"), and TLS-enabled ports to do a deeper inspection on (the tlssub-key, containing a dict with port numbers as keys, and type of TLS service or the word "skip" as value)ports: "skip" is an abbreviation of:
ports:
tcp: "skip"
udp: "skip"
tls: "skip"
testssl.sh supports for the -t/--starttls option, "https" for HTTPS test (including headers), or "tls" for generic TLS test.tcp" key.skip: whether the host should be skipped altogether (boolean)You can see the example one config here.
Global defaults are defined in probers.yml. For each probed host these are then combined with the relevant zone defaults, then with regex keys that match a given hostname, and finally with the specific host settings.
This means that specific hosts settings in a given zone take priority over settings from regex matched keys, which in turn take precedence over zone defaults, which in turn take priority over global defaults.
The ports key is treated a bit specially: if certain ports are listed somewhere in the configuration inheritance chain, it's impossible to de-list them lower in the chain, only to add additional port numbers, or to "skip" tests for all ports of a given protocol altogether.
Zone files are selected based on the tested_zone variable set for each prober node.