ThreatMapper - Identify Vulnerabilities In Running Containers, Images, Hosts And Repositories
The Deepfence Runtime Threat Mapper is a subset of the Deepfence cloud native workload protection platform, released as a community edition. This community edition empowers the users with following features:
-
Visualization: Visualize kubernetes clusters, virtual machines, containers and images, running processes, and network connections in near real time.
-
Runtime Vulnerability Management: Perform vulnerability scans on running containers & hosts as well as container images.
-
Container Registry Scanning: Check for vulnerabilities in images stored on AWS ECR, Azure Container Registry, Google Container Registry, Docker Hub, Docker Self-Hosted Private Registry, Quay, Harbor, Gitlab and JFrog registries.
-
CI/CD Scanning: Scan images as part of existing CI/CD Pipelines like CircleCI, Jenkins & GitLab.
-
Integrations with SIEM, Notification Channels & Ticketing: Ready to use integrations with Slack, PagerDuty, HTTP endpoint, Jira, Splunk, ELK, Sumo Logic and Amazon S3.
https://deepfence.io/community-demo-form/
Architecture
A pictorial depiction of the Deepfence Architecture is below
Features | Runtime Threat mapper (Community Edition) | Workload Protection Platform (Enterprise Edition) |
---|---|---|
Discover & Visualize Running Pods, Containers and Hosts | ✔️ (unlimited) | ✔️ (unlimited) |
Runtime Vulnerability Management for hosts/VMs | ✔️ (unlimited) | ✔️ (unlimited) |
Runtime Vulnerability Management for containers | ✔️ (unlimited) | ✔️ (unlimited) |
Container Registry Scanning | ✔️ | ✔️ |
CI/CD Integration | ✔️ | ✔️ |
Multiple Clusters | ✔️ | ✔️ |
Integrations with SIEMs, Slack and more | ✔️ | ✔️ |
Compliance Automation | ❌ | ✔️ |
Deep Packet Inspection of Encrypted & Plain Traffic | ❌ | ✔️ |
API Inspection | ❌ | ✔️ |
Runtime Integrity Monitoring | ❌ | ✔️ |
Network Connection & Resource Access Anomaly Detection | ❌ | ✔️ |
Workload Firewall for Containers, Pods and Hosts | ❌ | ✔️ |
Quarantine & Network Protection Policies | ❌ | ✔️ |
Alert Correlation | ❌ | ✔️ |
Serverless Protection | ❌ | ✔️ |
Windows Protection | ❌ | ✔️ |
Highly Available & Multi-node Deployment | ❌ | ✔️ |
Multi-tenancy & User Management | ❌ | ✔️ |
Enterprise Support | ❌ | ✔️ |
Getting Started
The Deepfence Management Console is first installed on a separate system. The Deepfence agents are then installed onto bare-metal servers, Virtual Machines, or Kubernetes clusters where the application workloads are deployed, so that the host systems, or the application workloads, can be scanned for vulnerabilities.
A pictorial depiction of the Deepfence security platform is as follows:
Deepfence Management Console
Pre-Requisites for Management Console
Feature | Requirements |
---|---|
CPU: No of cores | 4 |
RAM | 16 GB |
Disk space | At-least 120 GB |
Port range to be opened for receiving data from Deepfence agents | 8000 - 8010 |
Port to be opened for web browsers to be able to communicate with the Management console to view the UI | 443 |
Docker binaries | At-least version 18.03 |
Docker-compose binary | Version 1.20.1 |
Following table gives the number of nodes that can be supported with different console machine configurations assuming a single node deployment of console. Memory optimised instances are shown to perform better.
CPU | RAM | Nodes supported |
---|---|---|
4 cores | 16 GB RAM | 250 nodes |
8 cores | 16 GB RAM | 500 nodes |
8 cores | 32 GB RAM | 1000 nodes |
16 cores | 32 GB RAM | 1400-1500 nodes |
In order to support higher numbers of nodes (i.e. hosts as number of containers can be unlimited theoretically based on their life times) ThreatMapper needs to be deployed as a 3 node k8s cluster to scale up to 10000 nodes, instructions to follow.
Installation of Deepfence Management Console
Installing the Management Console is as easy as:
- Download the file docker-compose.yml to the desired system.
- Execute the following command
docker-compose -f docker-compose.yml up -d
- Open management console ip address / domain in the browser (https://x.x.x.x) and register a new account. Steps: Register a User
- Get Deepfence api key from UI: Goto
Settings
->User Management
, copy api key. In the following docker run command, replaceC8TtyEtNB0gBo1wGhpeAZICNSAaGWw71BSdS2kLELY0
with api Key. Steps: Deepfence API Keydocker run -dit --cpus=".2" --name=deepfence-agent --restart on-failure --pid=host --net=host --privileged=true -v /sys/kernel/debug:/sys/kernel/debug:rw -v /var/log/fenced -v /var/run/docker.sock:/var/run/docker.sock -v /:/fenced/mnt/host/:ro -e USER_DEFINED_TAGS="" -e DF_BACKEND_IP="127.0.0.1" -e DEEPFENCE_KEY="C8TtyEtNB0gBo1wGhpeAZICNSAaGWw71BSdS2kLELY0" deepfenceio/deepfence_agent_ce:latest
This is the minimal installation required to quickly get started on scanning various container images. The necessary images may now be downloaded onto this Management Console and scanned for vulnerabilities.
Terraform
- Terraform module to provision Deepfence ThreatMapper on GCP Compute Engine
Installation with custom TLS certificates
Custom TLS certificates are supported for the web application hosted on the console machine. On the console machine users have to place the certificate and private key on /etc/deepfence/certs folder. Deepfence looks for the file with .key and .crt extentions on the specified location on the host.
Deepfence Agent
In order to check a host for vulnerabilities, or if docker images or containers that have to be checked for vulnerabilities are saved on different hosts, then the Deepfence agent needs to be installed on those hosts.
Pre-Requisites for Deepfence Agent
Feature | Requirements |
---|---|
CPU: No of cores | 2 |
RAM | 1 GB |
Disk space | At-least 30 GB |
Connectivity | The host on which the Deepfence Agent is to be installed, is able to communicate with the Management Console on port range 8000-8010. |
Linux kernel version | >= 4.4 |
Docker binaries | At-least version 18.03 |
Deepfence Management Console | Installed on a host with IP Address x.x.x.x |
Installation of Deepfence Agent
Installation procedure for the Deepfence agent depends on the environment that is being used. Instructions for installing Deepfence agent on some of the common platforms are given in detail below:
Deepfence Agent on Standalone VM or Host
Installing the Deepfence Agent is now as easy as:
- Get Deepfence api key from UI: Goto
Settings
->User Management
, copy api key - In the following docker run command, replace
x.x.x.x
with the IP address of the Management Console and replaceC8TtyEtNB0gBo1wGhpeAZICNSAaGWw71BSdS2kLELY0
with api Keydocker run -dit --cpus=".2" --name=deepfence-agent --restart on-failure --pid=host --net=host --privileged=true -v /sys/kernel/debug:/sys/kernel/debug:rw -v /var/log/fenced -v /var/run/docker.sock:/var/run/docker.sock -v /:/fenced/mnt/host/:ro -e USER_DEFINED_TAGS="" -e DF_BACKEND_IP="x.x.x.x" -e DEEPFENCE_KEY="C8TtyEtNB0gBo1wGhpeAZICNSAaGWw71BSdS2kLELY0" deepfenceio/deepfence_agent_ce:latest
- Optionally the agent node can be tagged using
USER_DEFINED_TAGS=""
in the above command. Tags should be comma separated. Example: "dev,front-end"
Deepfence Agent on Amazon ECS
For detailed instructions to deploy agents on Amazon ECS, please refer to our Amazon ECS wiki page.
Deepfence Agent Helm chart for Kubernetes
- Start deepfence agent (replace
x.x.x.x
with the IP address of the Management Console andC8TtyEtNB0gBo1wGhpeAZICNSAaGWw71BSdS2kLELY0
with api key)
# helm v2
helm install --repo https://deepfence.github.io/ThreatMapper/files/helm-chart deepfence-agent \
--name=deepfence-agent \
--set managementConsoleIp=x.x.x.x \
--set deepfenceKey=C8TtyEtNB0gBo1wGhpeAZICNSAaGWw71BSdS2kLELY0
# helm v3
helm install deepfence-agent --repo https://deepfence.github.io/ThreatMapper/files/helm-chart deepfence-agent \
--set managementConsoleIp=x.x.x.x \
--set deepfenceKey=C8TtyEtNB0gBo1wGhpeAZICNSAaGWw71BSdS2kLELY0
- Delete deepfence agent
# helm v2
helm delete --purge deepfence-agent
# helm v3
helm delete deepfence-agent
Deepfence Agent on Google GKE
For detailed instructions to deploy agents on Google GKE, please refer to our Google GKE wiki page.
Deepfence Agent on Azure AKS
For detailed instructions to deploy agents on Azure Kubernetes Service, please refer to our Azure AKS wiki page.
Deepfence Agent on self-managed / on-premise Kubernetes
For detailed instructions to deploy agents on a Kubernetes cluster, please refer to our Self-managed/On-premise Kubernetes wiki page.