66 61 74 74 2e
给所有事物打指纹！
有关指纹识别方法、示例用例和研究结果的更多信息将很快添加到仓库中。敬请期待！
一个用于从数据包捕获文件（pcap）或实时网络流量中提取网络元数据和指纹（如 JA3 和 HASSH）的脚本。主要用途是监控蜜罐，但你也可以将其用于其他场景，例如网络取证分析。fatt 可在 Linux、macOS 和 Windows 上运行。
请注意，fatt 使用 pyshark（tshark 的 Python 封装），因此性能不佳！但这并不是大问题，显然这不是一个用于生产环境的工具。对于更严肃的用例，你可以使用其他网络分析工具，如 Bro/Zeek、Suricata 或 Netcap。Joy 是另一个可用于捕获和分析网络流数据的优秀工具。
除此之外，我正在开发一个基于 Go 的 fatt 版本，速度更快，并且你可以将其库用于基于 gopacket 的工具（如 packetbeat）。我已经发布了其 gQUIC 库的初始版本（QUICk）。
你首先需要安装 tshark。确保你拥有 v2.9.0 或更高版本。从 v2.9.0 开始，Tshark/Wireshak 将 'ssl' 重命名为 'tls'，并且 fatt 是基于新版本 tshark 编写的。
如果你使用的是旧版本 tshark（< v2.9.0），可以使用来自 "old-tshark" 分支 的 fatt 脚本。
cd fatt/
pip3 install pipenv
pipenv install
或者，如果你不想使用虚拟环境，直接安装 pyshark：
pip3 install pyshark==0.4.2.2
要激活虚拟环境，运行 pipenv shell：
$ pipenv shell
Launching subshell in virtual environment…
bash-3.2$ . /Users/adel/.local/share/virtualenvs/fatt-ucJHMzzt/bin/activate
(fatt-ucJHMzzt) bash-3.2$ python3 fatt.py -h
或者，使用 pipenv run 在虚拟环境中运行命令：
$ pipenv run python3 fatt.py -h
输出：
usage: fatt.py [-h] [-r READ_FILE] [-d READ_DIRECTORY] [-i INTERFACE]
[-fp [{tls,ssh,rdp,http,gquic} [{tls,ssh,rdp,http,gquic} ...]]]
[-da DECODE_AS] [-f BPF_FILTER] [-j] [-o OUTPUT_FILE]
[-w WRITE_PCAP] [-p]
A python script for extracting network fingerprints
optional arguments:
-h, --help show this help message and exit
-r READ_FILE, --read_file READ_FILE
pcap file to process
-d READ_DIRECTORY, --read_directory READ_DIRECTORY
directory of pcap files to process
-i INTERFACE, --interface INTERFACE
listen on interface
-fp [{tls,ssh,rdp,http,gquic} [{tls,ssh,rdp,http,gquic} ...]], --fingerprint [{tls,ssh,rdp,http,gquic} [{tls,ssh,rdp,http,gquic} ...]]
protocols to fingerprint. Default: all
-da DECODE_AS, --decode_as DECODE_AS
a dictionary of {decode_criterion_string:
decode_as_protocol} that is used to tell tshark to
decode protocols in situations it wouldn't usually.
-f BPF_FILTER, --bpf_filter BPF_FILTER
BPF capture filter to use (for live capture only).'
-j, --json_logging log the output in json format
-o OUTPUT_FILE, --output_file OUTPUT_FILE
specify the output log file. Default: fatt.log
-w WRITE_PCAP, --write_pcap WRITE_PCAP
save the live captured packets to this file
-p, --print_output print the output
$ python3 fatt.py -i en0 --print_output --json_logging
192.168.1.10:59565 -> 192.168.1.3:80 [HTTP] hash=598c34a2838e82f9ec3175305f233b89 userAgent="Spotify/109600181 OSX/0 (MacBookPro14,3)"
192.168.1.10:59566 -> 13.237.44.5:22 [SSH] hassh=ec7378c1a92f5a8dde7e8b7a1ddf33d1 client=SSH-2.0-OpenSSH_7.9
13.237.44.5:22 -> 192.168.1.10:59566 [SSH] hasshS=3f0099d323fed5119bbfcca064478207 server=SSH-2.0-babeld-80573d3e
192.168.1.10:59584 -> 93.184.216.34:443 [TLS] ja3=e6573e91e6eb777c0933c5b8f97f10cd serverName=example.com
93.184.216.34:443 -> 192.168.1.10:59584 [TLS] ja3s=ae53107a2e47ea20c72ac44821a728bf
192.168.1.10:59588 -> 192.168.1.3:80 [HTTP] hash=598c34a2838e82f9ec3175305f233b89 userAgent="Spotify/109600181 OSX/0 (MacBookPro14,3)"
192.168.1.10:59601 -> 216.58.196.142:80 [HTTP] hash=d6662c018cd4169689ddf7c6c0f8ca1b userAgent="curl/7.54.0"
216.58.196.142:80 -> 192.168.1.10:59601 [HTTP] hash=c5241aca9a7c86f06f476592f5dda9a1 server=gws
192.168.1.10:54387 -> 216.58.203.99:443 [QUIC] UAID="Chrome/74.0.3729.169 Intel Mac OS X 10_14_5" SNI=clientservices.googleapis.com AEAD=AESG KEXS=C255
JSON 输出：
$ cat fatt.log
{"timestamp": "2019-05-28T03:41:25.415086", "sourceIp": "192.168.1.10", "destinationIp": "192.168.1.3", "sourcePort": "59565", "destinationPort": "80", "protocol": "http", "http": {"requestURI": "/DIAL/apps/com.spotify.Spotify.TVv2", "requestFullURI": "http://192.168.1.3/DIAL/apps/com.spotify.Spotify.TVv2", "requestVersion": "HTTP/1.1", "requestMethod": "GET", "userAgent": "Spotify/109600181 OSX/0 (MacBookPro14,3)", "clientHeaderOrder": "connection,accept_encoding,host,user_agent", "clientHeaderHash": "598c34a2838e82f9ec3175305f233b89"}}
{"timestamp": "2019-05-28T03:41:26.099574", "sourceIp": "13.237.44.5", "destinationIp": "192.168.1.10", "sourcePort": "22", "destinationPort": "59566", "protocol": "ssh", "ssh": {"server": "SSH-2.0-babeld-80573d3e", "hasshServer": "3f0099d323fed5119bbfcca064478207", "hasshServerAlgorithms": "curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256;[email protected],[email protected],[email protected],aes256-ctr,aes192-ctr,aes128-ctr,aes256-cbc,aes192-cbc,aes128-cbc;[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1;none,zlib,[email protected]", "hasshVersion": "1.0", "skex": "curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256", "seastc": "[email protected],[email protected],[email protected],aes256-ctr,aes192-ctr,aes128-ctr,aes256-cbc,aes192-cbc,aes128-cbc", "smastc": "[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1", "scastc": "none,zlib,[email protected]", "slcts": "[Empty]", "slstc": "[Empty]", "seacts": "[email protected],[email protected],[email protected],aes256-ctr,aes192-ctr,aes128-ctr,aes256-cbc,aes192-cbc,aes128-cbc", "smacts": "[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1", "scacts": "none,zlib,[email protected]", "sshka": "ssh-dss,rsa-sha2-512,rsa-sha2-256,ssh-rsa"}}
{"timestamp": "2019-05-28T03:41:26.106737", "sourceIp": "192.168.1.10", "destinationIp": "13.237.44.5", "sourcePort": "59566", "destinationPort": "22", "protocol": "ssh", "ssh": {"client": "SSH-2.0-OpenSSH_7.9", "hassh": "ec7378c1a92f5a8dde7e8b7a1ddf33d1", "hasshAlgorithms": "curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c;[email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected];[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1;none,[email protected],zlib", "hasshVersion": "1.0", "ckex": "curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c", "ceacts": "[email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]", "cmacts": "[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1", "ccacts": "none,[email protected],zlib", "clcts": "[Empty]", "clstc": "[Empty]", "ceastc": "[email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]", "cmastc": "[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1", "ccastc": "none,[email protected],zlib", "cshka": "[email protected],[email protected],[email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa,[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519"}}
{"timestamp": "2019-05-28T03:41:36.762811", "sourceIp": "192.168.1.10", "destinationIp": "93.184.216.34", "sourcePort": "59584", "destinationPort": "443", "protocol": "tls", "tls": {"serverName": "example.com", "ja3": "e6573e91e6eb777c0933c5b8f97f10cd", "ja3Algorithms": "771,49200-49196-49192-49188-49172-49162-159-107-57-52393-52392-52394-65413-196-136-129-157-61-53-192-132-49199-49195-49191-49187-49171-49161-158-103-51-190-69-156-60-47-186-65-49170-49160-22-10-255,0-11-10-13-16,29-23-24,0", "ja3Version": "771", "ja3Ciphers": "49200-49196-49192-49188-49172-49162-159-107-57-52393-52392-52394-65413-196-136-129-157-61-53-192-132-49199-49195-49191-49187-49171-49161-158-103-51-190-69-156-60-47-186-65-49170-49160-22-10-255", "ja3Extensions": "0-11-10-13-16", "ja3Ec": "29-23-24", "ja3EcFmt": "0"}}
{"timestamp": "2019-05-28T03:41:36.920935", "sourceIp": "93.184.216.34", "destinationIp": "192.168.1.10", "sourcePort": "443", "destinationPort": "59584", "protocol": "tls", "tls": {"ja3s": "ae53107a2e47ea20c72ac44821a728bf", "ja3sAlgorithms": "771,49199,65281-0-11-16", "ja3sVersion": "771", "ja3sCiphers": "49199", "ja3sExtensions": "65281-0-11-16"}}
{"timestamp": "2019-05-28T03:41:37.487609", "sourceIp": "192.168.1.10", "destinationIp": "192.168.1.3", "sourcePort": "59588", "destinationPort": "80", "protocol": "http", "http": {"requestURI": "/DIAL/apps/com.spotify.Spotify.TVv2", "requestFullURI": "http://192.168.1.3/DIAL/apps/com.spotify.Spotify.TVv2", "requestVersion": "HTTP/1.1", "requestMethod": "GET", "userAgent": "Spotify/109600181 OSX/0 (MacBookPro14,3)", "clientHeaderOrder": "connection,accept_encoding,host,user_agent", "clientHeaderHash": "598c34a2838e82f9ec3175305f233b89"}}
{"timestamp": "2019-05-28T03:41:48.700730", "sourceIp": "192.168.1.10", "destinationIp": "216.58.196.142", "sourcePort": "59601", "destinationPort": "80", "protocol": "http", "http": {"requestURI": "/", "requestFullURI": "http://google.com/", "requestVersion": "HTTP/1.1", "requestMethod": "GET", "userAgent": "curl/7.54.0", "clientHeaderOrder": "host,user_agent,accept", "clientHeaderHash": "d6662c018cd4169689ddf7c6c0f8ca1b"}}
{"timestamp": "2019-05-28T03:41:48.805393", "sourceIp": "216.58.196.142", "destinationIp": "192.168.1.10", "sourcePort": "80", "destinationPort": "59601", "protocol": "http", "http": {"server": "gws", "serverHeaderOrder": "location,content_type,date,cache_control,server,content_length", "serverHeaderHash": "c5241aca9a7c86f06f476592f5dda9a1"}}
{"timestamp": "2019-05-28T03:41:58.038530", "sourceIp": "192.168.1.10", "destinationIp": "216.58.203.99", "sourcePort": "54387", "destinationPort": "443", "protocol": "gquic", "gquic": {"tagNumber": "25", "sni": "clientservices.googleapis.com", "uaid": "Chrome/74.0.3729.169 Intel Mac OS X 10_14_5", "ver": "Q043", "aead": "AESG", "smhl": "1", "mids": "100", "kexs": "C255", "xlct": "cd9baccc808a6d3b", "copt": "NSTP", "ccrt": "cd9baccc808a6d3b67f8adc58015e3ff", "stk": "d6a64aeb563a19fe091bc34e8c038b0a3a884c5db7caae071180c5b739bca3dd7c42e861386718982fbe6db9d1cb136f799e8d10fd5a", "pdmd": "X509", "ccs": "01e8816092921ae8", "scid": "376976b980c73b669fea57104fb725c6"}}
让我们看看针对近期 CVE-2019-0708 RDP 漏洞（BlueKeep）的 Metasploit 辅助扫描器的捕获流量。
$ python3 fatt.py -r RDP/cve-2019-0708_metasploit_aux.pcap -p -j; cat fatt.log | python -m json.tool
192.168.1.10:39079 -> 192.168.1.20:3389 [RDP] rdfp=3ba3d115055e593e3550575a36e68153 cookie="mstshash=user0" req_protocols=0x00000000
{
"destinationIp": "192.168.1.20",
"destinationPort": "3389",
"protocol": "rdp",
"rdp": {
"channelDefArray": {
"0": {
"name": "cliprdr",
"options": "c0a00000"
},
"1": {
"name": "MS_T120",
"options": "80800000"
},
"2": {
"name": "rdpsnd",
"options": "c0000000"
},
"3": {
"name": "snddbg",
"options": "c0000000"
},
"4": {
"name": "rdpdr",
"options": "80800000"
}
},
"clientBuild": "2600",
"clientDigProductId": "00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
"clientName": "x1810",
"clientProductId": "1",
"clusterFlags": "09000000",
"colorDepth": "0x0000ca01",
"connectionType": "0",
"cookie": "mstshash=user0",
"desktopHeight": "600",
"desktopWidth": "800",
"earlyCapabilityFlags": "1",
"encryptionMethods": "03000000",
"extEncMethods": "00000000",
"highColorDepth": "0x00000018",
"keyboardFuncKey": "12",
"keyboardLayout": "1033",
"keyboardSubtype": "0",
"keyboardType": "4",
"pad1Octet": "00",
"postbeta2ColorDepth": "0x0000ca01",
"rdfp": "3ba3d115055e593e3550575a36e68153",
"rdfpAlgorithms": "4,8,09000000,03000000,00000000,cliprdr:c0a00000-MS_T120:80800000-rdpsnd:c0000000-snddbg:c0000000-rdpdr:80800000",
"rdfpVersion": "0.3",
"requestedProtocols": "0x00000000",
"sasSequence": "43523",
"serialNumber": "0",
"supportedColorDepths": "0x00000007",
"verMajor": "4",
"verMinor": "8"
},
"sourceIp": "192.168.1.10",
"sourcePort": "39079",
"timestamp": "2019-05-23T03:51:25.438445"
}
让我们用另一个 CVE-2019-0708 PoC 测试：
$ python3 fatt.py -r RDP/cve-2019-0708_poc.pcap -p -j; cat fatt.log | python -m json.tool
192.168.1.10:54303 -> 192.168.1.20:3389 [RDP] req_protocols=0x00000001
{
"destinationIp": "192.168.1.20",
"destinationPort": "3389",
"protocol": "rdp",
"rdp": {
"requestedProtocols": "0x00000001"
},
"sourceIp": "192.168.1.10",
"sourcePort": "54303",
"timestamp": "2019-05-23T18:41:42.572758"
}
这次我们没有看到 RDP ClientInfo 消息，因为该 PoC 使用了 TLS（而非标准 RDP 安全协议）。因此我们只能看到 Negotiation Request 消息，但如果你将数据包解码为 TLS，你就可以看到 TLS clientHello 和 JA3 指纹。以下是将特定端口解码为另一协议的方法：
$ python3 fatt.py -r RDP//cve-2019-0708_poc.pcap -p -j --decode_as '{"tcp.port==3389": "tls"}'
192.168.1.10:50026 -> 192.168.1.20:3389 [TLS] ja3=67e3d18fd9dddbbc8eca65f7dedac674 serverName=192.168.1.20
192.168.1.20:3389 -> 192.168.1.10:50026 [TLS] ja3s=649d6810e8392f63dc311eecb6b7098b
$ cat fatt.log
{"timestamp": "2019-05-23T17:21:56.056200", "sourceIp": "192.168.1.10", "destinationIp": "192.168.1.20", "sourcePort": "50026", "destinationPort": "3389", "protocol": "tls", "tls": {"serverName": "192.168.1.20", "ja3": "67e3d18fd9dddbbc8eca65f7dedac674", "ja3Algorithms": "771,49196-49195-49200-49199-159-158-49188-49187-49192-49191-49162-49161-49172-49171-57-51-157-156-61-60-53-47-10-106-64-56-50-19-5-4,0-5-10-11-13-35-23-65281,29-23-24,0", "ja3Version": "771", "ja3Ciphers": "49196-49195-49200-49199-159-158-49188-49187-49192-49191-49162-49161-49172-49171-57-51-157-156-61-60-53-47-10-106-64-56-50-19-5-4", "ja3Extensions": "0-5-10-11-13-35-23-65281", "ja3Ec": "29-23-24", "ja3EcFmt": "0"}}
{"timestamp": "2019-05-23T17:21:56.059333", "sourceIp": "192.168.1.20", "destinationIp": "192.168.1.10", "sourcePort": "3389", "destinationPort": "50026", "protocol": "tls", "tls": {"ja3s": "649d6810e8392f63dc311eecb6b7098b", "ja3sAlgorithms": "771,49192,23-65281", "ja3sVersion": "771", "ja3sCiphers": "49192", "ja3sExtensions": "23-65281"}}