Dshell

一个可扩展的网络取证分析框架。支持通过快速开发插件来解析网络数据包捕获。

主要特点：

使用专用插件进行深度数据包分析
强大的流重组能力
支持 IPv4 和 IPv6
多种用户可选的输出格式，并可创建自定义输出处理程序
可链式组合的插件
并行处理选项，可将数据处理分配到独立的 Python 进程
支持开发外部插件包，以便共享和安装新开发的外部插件，而不会与核心 Dshell 插件目录重叠

指南

Dshell 用户指南
- 包含安装指南以及基于示例的基础和高级分析
- 帮助新手和有经验的终端用户理解和使用解码器外壳（Dshell）框架
Dshell 开发指南
- 提供基于基础示例的插件开发指南，以及核心函数和类定义，并对数据流程进行概述
- 帮助终端用户开发新的自定义 Dshell 插件，以及修改现有插件

要求

Linux（基于 Ubuntu 20.04 LTS 开发）
Python 3（基于 Python 3.8.10 开发）
pypacker
pcapy-ng
pyOpenSSL
geoip2
- MaxMind GeoIP2 数据集
  - 用于将 IP 地址映射为国家/地区代码
  - 参见安装部分进行配置

可选

oui.txt
- 某些处理 MAC 地址的插件会用到
- 放置于 <dshell>/data/ 目录下
elasticsearch
- 用于 elasticout 输出模块
- 仅当你计划使用 elasticsearch 存储输出时需要
pyJA3
- 用于 tls 插件

安装

使用 pip 安装 Dshell

python3 -m pip install Dshell/ 或 python3 -m pip install <Dshell-tarball>

配置 geoip2：将 MaxMind GeoLite2 数据集文件（GeoLite2-ASN.mmdb、GeoLite2-City.mmdb、GeoLite2-Country.mmdb）放入 [...]/site-packages/dshell/data/GeoIP/ 目录
运行 dshell。你应该会进入 Dshell> 提示符。

基本用法

decode -l
- 列出所有可用插件及其基本信息
decode -h
- 显示大多数插件通用的命令行标志，例如面向色盲友好的颜色输出模式
decode -p <plugin>
- 显示插件的详细信息，包括可用的命令行标志
decode -p <plugin> <pcap>
- 在 pcap 或 pcapng 文件上运行所选插件
decode -p <plugin1>+<plugin2> <pcap>
- 将两个（或更多）插件链式组合，并在 pcap 文件上运行
decode -p <plugin> -i <interface>
- 在接口上实时运行所选插件（可能需要超级用户权限）

使用示例

在示例流量中显示 DNS 查询

Dshell> decode -p dns ~/pcap/dns.cap | sort
[DNS] 2005-03-30 03:47:46    192.168.170.8:32795 --   192.168.170.20:53    ** ID: 4146, TXT? google.com., TXT: b'\x0fv=spf1 ptr ?all' **
[DNS] 2005-03-30 03:47:50    192.168.170.8:32795 --   192.168.170.20:53    ** ID: 63343, MX? google.com., MX: b'\x00(\x05smtp4\xc0\x0c', MX: b'\x00\n\x05smtp5\xc0\x0c', MX: b'\x00\n\x05smtp6\xc0\x0c', MX: b'\x00\n\x05smtp1\xc0\x0c', MX: b'\x00\n\x05smtp2\xc0\x0c', MX: b'\x00(\x05smtp3\xc0\x0c' **
[DNS] 2005-03-30 03:47:59    192.168.170.8:32795 --   192.168.170.20:53    ** ID: 18849, LOC? google.com. **
[DNS] 2005-03-30 03:48:07    192.168.170.8:32795 --   192.168.170.20:53    ** ID: 39867, PTR? 104.9.192.66.in-addr.arpa., PTR: 66-192-9-104.gen.twtelecom.net. **
[DNS] 2005-03-30 03:49:18    192.168.170.8:32795 --   192.168.170.20:53    ** ID: 30144, A? www.netbsd.org., A: 204.152.190.12 (ttl 82159s) **
[DNS] 2005-03-30 03:49:35    192.168.170.8:32795 --   192.168.170.20:53    ** ID: 61652, AAAA? www.netbsd.org., AAAA: 2001:4f8:4:7:2e0:81ff:fe52:9a6b (ttl 86400s) **
[DNS] 2005-03-30 03:50:35    192.168.170.8:32795 --   192.168.170.20:53    ** ID: 32569, AAAA? www.netbsd.org., AAAA: 2001:4f8:4:7:2e0:81ff:fe52:9a6b (ttl 86340s) **
[DNS] 2005-03-30 03:50:44    192.168.170.8:32795 --   192.168.170.20:53    ** ID: 36275, AAAA? www.google.com., CNAME: 'www.l.google.com.' **
[DNS] 2005-03-30 03:50:54    192.168.170.8:32795 --   192.168.170.20:53    ** ID: 56482, AAAA? www.l.google.com. **
[DNS] 2005-03-30 03:51:35    192.168.170.8:32795 --   192.168.170.20:53    ** ID: 48159, AAAA? www.example.com. **
[DNS] 2005-03-30 03:51:46    192.168.170.8:32795 --   192.168.170.20:53    ** ID: 9837, AAAA? www.example.notginh., NXDOMAIN **
[DNS] 2005-03-30 03:52:17    192.168.170.8:32795 --   192.168.170.20:53    ** ID: 65251, AAAA: 2001:4f8:0:2::d (ttl 600s), A: 204.152.184.88 (ttl 600s) **
[DNS] 2005-03-30 03:52:17    192.168.170.8:32796 --   192.168.170.20:53    ** ID: 23123, PTR? 1.0.0.127.in-addr.arpa., PTR: localhost. **
[DNS] 2005-03-30 03:52:17    192.168.170.8:32797 --   192.168.170.20:53    ** ID: 8330, NS: b'\x06ns-ext\x04nrt1\xc0\x0c', NS: b'\x06ns-ext\x04sth1\xc0\x0c', NS: b'\x06ns-ext\xc0\x0c', NS: b'\x06ns-ext\x04lga1\xc0\x0c' **
[DNS] 2005-03-30 03:52:17   192.168.170.56:1707  --      217.13.4.24:53    ** ID: 12910, SRV? _ldap._tcp.Default-First-Site-Name._sites.dc._msdcs.utelsystems.local., NXDOMAIN **
[DNS] 2005-03-30 03:52:17   192.168.170.56:1708  --      217.13.4.24:53    ** ID: 61793, SRV? _ldap._tcp.dc._msdcs.utelsystems.local., NXDOMAIN **
[DNS] 2005-03-30 03:52:17   192.168.170.56:1709  --      217.13.4.24:53    ** ID: 33633, SRV? _ldap._tcp.05b5292b-34b8-4fb7-85a3-8beef5fd2069.domains._msdcs.utelsystems.local., NXDOMAIN **
[DNS] 2005-03-30 03:52:17   192.168.170.56:1710  --      217.13.4.24:53    ** ID: 53344, A? GRIMM.utelsystems.local., NXDOMAIN **
[DNS] 2005-03-30 03:52:25   192.168.170.56:1711  --      217.13.4.24:53    ** ID: 30307, A? GRIMM.utelsystems.local., NXDOMAIN **

在示例流量中跟踪并重组一个流

Dshell> decode -p followstream ~/pcap/v6-http.cap 
Connection 1 (TCP)
Start: 2007-08-05 15:16:44.189851
End:   2007-08-05 15:16:44.219460
2001:6f8:102d:0:2d0:9ff:fee3:e8de: 59201 -> 2001:6f8:900:7c0::2:    80 (300 bytes)
2001:6f8:900:7c0::2:    80 -> 2001:6f8:102d:0:2d0:9ff:fee3:e8de: 59201 (2379 bytes)

GET / HTTP/1.0
Host: cl-1985.ham-01.de.sixxs.net
Accept: text/html, text/plain, text/css, text/sgml, */*;q=0.01
Accept-Encoding: gzip, bzip2
Accept-Language: en
User-Agent: Lynx/2.8.6rel.2 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.8b



HTTP/1.1 200 OK
Date: Sun, 05 Aug 2007 19:16:44 GMT
Server: Apache
Content-Length: 2121
Connection: close
Content-Type: text/html

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<html>
 <head>
  <title>Index of /</title>
 </head>
 <body>
<h1>Index of /</h1>
<pre><img src="https://raw.githubusercontent.com/usarmyresearchlab/dshell/master/icons/blank.gif" alt="Icon "> <a href="?C=N;O=D">Name</a>                    <a href="?C=M;O=A">Last modified</a>      <a href="?C=S;O=A">Size</a>  <a href="?C=D;O=A">Description</a><hr><img src="https://raw.githubusercontent.com/usarmyresearchlab/dshell/master/icons/folder.gif" alt="[DIR]"> <a href="https://github.com/usarmyresearchlab/dshell/blob/master/202-vorbereitung">202-vorbereitung/</a>       06-Jul-2007 14:31    -   
<img src="https://raw.githubusercontent.com/usarmyresearchlab/dshell/master/icons/layout.gif" alt="[   ]"> <a href="https://github.com/usarmyresearchlab/dshell/blob/master/Efficient_Video_on_demand_over_Multicast.pdf">Efficient_Video_on_d..&gt;</a> 19-Dec-2006 03:17  291K  
<img src="https://raw.githubusercontent.com/usarmyresearchlab/dshell/master/icons/unknown.gif" alt="[   ]"> <a href="https://github.com/usarmyresearchlab/dshell/blob/master/Welcome%20Stranger%21%21%21">Welcome Stranger!!!</a>     28-Dec-2006 03:46    0   
<img src="https://raw.githubusercontent.com/usarmyresearchlab/dshell/master/icons/text.gif" alt="[TXT]"> <a href="https://github.com/usarmyresearchlab/dshell/blob/master/barschel.htm">barschel.htm</a>            31-Jul-2007 02:21   44K  
<img src="https://raw.githubusercontent.com/usarmyresearchlab/dshell/master/icons/folder.gif" alt="[DIR]"> <a href="https://github.com/usarmyresearchlab/dshell/blob/master/bnd">bnd/</a>                    30-Dec-2006 08:59    -   
<img src="https://raw.githubusercontent.com/usarmyresearchlab/dshell/master/icons/folder.gif" alt="[DIR]"> <a href="https://github.com/usarmyresearchlab/dshell/blob/master/cia">cia/</a>                    28-Jun-2007 00:04    -   
<img src="https://raw.githubusercontent.com/usarmyresearchlab/dshell/master/icons/layout.gif" alt="[   ]"> <a href="https://github.com/usarmyresearchlab/dshell/blob/master/cisco_ccna_640-801_command_reference_guide.pdf">cisco_ccna_640-801_c..&gt;</a> 28-Dec-2006 03:48  236K  
<img src="https://raw.githubusercontent.com/usarmyresearchlab/dshell/master/icons/folder.gif" alt="[DIR]"> <a href="https://github.com/usarmyresearchlab/dshell/blob/master/doc">doc/</a>                    19-Sep-2006 01:43    -   
<img src="https://raw.githubusercontent.com/usarmyresearchlab/dshell/master/icons/folder.gif" alt="[DIR]"> <a href="https://github.com/usarmyresearchlab/dshell/blob/master/freenetproto">freenetproto/</a>           06-Dec-2006 09:00    -   
<img src="https://raw.githubusercontent.com/usarmyresearchlab/dshell/master/icons/folder.gif" alt="[DIR]"> <a href="https://github.com/usarmyresearchlab/dshell/blob/master/korrupt">korrupt/</a>                03-Jul-2007 11:57    -   
<img src="https://raw.githubusercontent.com/usarmyresearchlab/dshell/master/icons/folder.gif" alt="[DIR]"> <a href="https://github.com/usarmyresearchlab/dshell/blob/master/mp3_technosets">mp3_technosets/</a>         04-Jul-2007 08:56    -   
<img src="https://raw.githubusercontent.com/usarmyresearchlab/dshell/master/icons/text.gif" alt="[TXT]"> <a href="https://github.com/usarmyresearchlab/dshell/blob/master/neues_von_rainald_goetz.htm">neues_von_rainald_go..&gt;</a> 21-Mar-2007 23:27   31K  
<img src="https://raw.githubusercontent.com/usarmyresearchlab/dshell/master/icons/text.gif" alt="[TXT]"> <a href="https://github.com/usarmyresearchlab/dshell/blob/master/neues_von_rainald_goetz0.htm">neues_von_rainald_go..&gt;</a> 21-Mar-2007 23:29   36K  
<img src="https://raw.githubusercontent.com/usarmyresearchlab/dshell/master/icons/layout.gif" alt="[   ]"> <a href="https://github.com/usarmyresearchlab/dshell/blob/master/pruef.pdf">pruef.pdf</a>               28-Dec-2006 07:48   88K  
<hr></pre>
</body></html>

链式组合插件，在示例流量中查看特定国家/地区的流数据（注意：TCP 握手不包含在数据包计数中）

Dshell> decode -p country+netflow --country_code=JP ~/pcap/SkypeIRC.cap
2006-08-25 15:32:20.766761       192.168.1.2 ->  202.232.205.123  (-- -> JP)   UDP   60583   33438     1      0       64        0  0.0000s
2006-08-25 15:32:20.634046       192.168.1.2 ->  202.232.205.123  (-- -> JP)   UDP   60583   33435     1      0       64        0  0.0000s
2006-08-25 15:32:20.747503       192.168.1.2 ->  202.232.205.123  (-- -> JP)   UDP   60583   33437     1      0       64        0  0.0000s
2006-08-25 15:32:20.651501       192.168.1.2 ->  202.232.205.123  (-- -> JP)   UDP   60583   33436     1      0       64        0  0.0000s

从多个文件中收集 DNS 流量并将其存储到新的 pcap 文件中。

Dshell> decode -p dns+pcapwriter --pcapwriter_outfile=test.pcap ~/pcap/*.cap > /dev/null
Dshell> tcpdump -nnr test.pcap | head
reading from file test.pcap, link-type EN10MB (Ethernet)
15:36:08.670569 IP 192.168.1.2.2131 > 192.168.1.1.53: 40209+ A? ui.skype.com. (30)
15:36:08.670687 IP 192.168.1.2.2131 > 192.168.1.1.53: 40210+ AAAA? ui.skype.com. (30)
15:36:08.674022 IP 192.168.1.1.53 > 192.168.1.2.2131: 40209- 1/0/0 A 212.72.49.131 (46)
15:36:09.011208 IP 192.168.1.1.53 > 192.168.1.2.2131: 40210 0/1/0 (94)
15:36:10.171350 IP 192.168.1.2.2131 > 192.168.1.1.53: 40210+ AAAA? ui.skype.com. (30)
15:36:10.961350 IP 192.168.1.1.53 > 192.168.1.2.2131: 40210* 0/1/0 (85)
15:36:10.961608 IP 192.168.1.2.2131 > 192.168.1.1.53: 40211+ AAAA? ui.skype.com. (30)
15:36:11.294333 IP 192.168.1.1.53 > 192.168.1.2.2131: 40211 0/1/0 (94)
15:32:21.664798 IP 192.168.1.2.2130 > 192.168.1.1.53: 39862+ A? ui.skype.com. (30)
15:32:21.664913 IP 192.168.1.2.2130 > 192.168.1.1.53: 39863+ AAAA? ui.skype.com. (30)

使用示例流量收集 TFTP 数据并将告警转换为 JSON 格式

Dshell> decode -p tftp -O jsonout ~/pcap/tftp_*.pcap
{"ts": 1367411051.972852, "sip": "192.168.0.253", "sport": 50618, "dip": "192.168.0.10", "dport": 3445, "readwrite": "read", "filename": "rfc1350.txt", "plugin": "tftp", "pcapfile": "/home/pcap/tftp_rrq.pcap", "data": "read  rfc1350.txt (24599 bytes) "}
{"ts": 1367053679.45274, "sip": "192.168.0.1", "sport": 57509, "dip": "192.168.0.13", "dport": 2087, "readwrite": "write", "filename": "rfc1350.txt", "plugin": "tftp", "pcapfile": "/home/pcap/tftp_wrq.pcap", "data": "write rfc1350.txt (24599 bytes) "}

在单独的 Python 脚本中运行插件，使用示例流量

# Import required Dshell libraries
import dshell.decode as decode
import dshell.plugins.tftp.tftp as tftp

# Instantiate plugin
plugin = tftp.DshellPlugin()
# Define plugin-specific arguments, if needed
dargs = {plugin: {"rip": True, "outdir": "/tmp/"}}
# Add plugin(s) to plugin chain
decode.plugin_chain = [plugin]
# Run decode main function with all other arguments
decode.main(
    debug=True,
    files=["/home/user/pcap/tftp_rrq.pcap", "/home/user/pcap/tftp_wrq.pcap"],
    plugin_args=dargs
)

Dshell

最受欢迎

Dshell

指南

要求

可选

安装

基本用法

使用示例