In the realm of cybersecurity, accurately identifying and characterizing web servers is crucial for threat detection, vulnerability assessment, and network mapping. We introduce HTTP-Basma, a novel active fingerprinting algorithm that unveils unique server profiles through a multi-layered approach.
HTTP-Basma 部署于 https://httpbasma.netomize.ca/
在网络安全领域，准确识别和描述 Web 服务器特征对于威胁检测、漏洞评估和网络映射至关重要。我们提出 HTTP-Basma，一种新颖的主动指纹识别算法，通过多层次方法揭示独特的服务器配置文件，从而应对这一挑战。
主要特性： 精心构造的请求，揭示性的响应：HTTP-Basma 发送 8 个精心设计的 HTTP 探测，引发反映服务器配置的独特响应。 双哈希实现多功能性：该算法生成两个哈希值：
聚类与追踪：这些哈希支持服务器聚类、识别独特和相似的服务器，并以更高置信度追踪恶意行为者。
模块化设计便于扩展：算法的架构鼓励添加新的哈希变体，促进协作和适应性。
在本文中，我们首先回顾 HTTP 指纹识别领域的现有重要工作，然后探讨算法的功能、设计、架构和成果。此外，我们将展示 扫描 Majestic 前 100 万网站 的引人注目的发现，包括识别和聚类多种恶意软件家族的 C&C HTTP 服务器。
HTTP-Basma 算法的核心思想是发送 8 个具有不同要求的特制 HTTP 请求，以从服务器获取不同响应。获取服务器响应后，对 HTTP 状态行进行精细分解，提取所有元素并在优化后进行编码。此外，还会检查并编码服务器响应中的选定标头。
它发送的请求类型如下：
每次请求后，服务器响应会被分析以提取特定标头和值。提取的数据经过进一步处理，包括分解和编码，生成可逆的指纹。
算法完整的技术细节详见附带的 论文。
这种模块化设计将每个请求指纹视为构建块，允许优雅地重构，并支持添加或移除任何请求的指纹。
指纹示例：
- verbosus fp: 011420958a0014514bd5221420958a221420958a221420958a2200001420958a22000000001f
- pacto fp: 02464ae8b7d86f82c9918e2c2b9d6b91
- note: false-positive rate (72/986,910)
- verbosus fp: 01142494d60914514bd522142494d6221420958a701420958a220000140e04922032c37f1609
- pacto fp: 020769322f3d94ac2f258ddf5ce08502
- note-1: false-positive rate 0
- note-2: tevedadav.site/43.209.165.126:443 (TLS)
- sample-(sha-256): 9aa1dec8dd12f8adc7fc1274e1958f3613450109ee8b4ec6442a0fcf06df0972
- verbosus fp: 01140a85e40014512f3612140a85e422140a85e422140a85e4220000140a85e4220000000001
- pacto fp: 0207292309a7a7e798e417d69df5f2a5
- note: false-positive rate (73/986,910)
- verbosus fp: 01140a85e4001320958a22142494d62214254c5e2214254c5e22080014254c5e220000000000
- pacto fp: 0202be780e1eaae0eaa6184e20c909b6
- note: false-positive rate (4/986,910)
- verbosus fp: 01140a85e4011320958a22142494d67214254c5e2214254c5e22080014254c5e220000000000
- pacto fp: 02cc5be6d05192e17de041538508bc22
- note: false-positive rate (38/986,910)
- verbosus fp: 01140a85e40914514bd522140a85e4721420958a701420958a220800140a85e4720000001609
- pacto fp: 0221b4e46bbd0e5c037f5a852ca3fdc0
- note: false-positive rate (6/986,910)
HTTP-Basma 是一个用 C++ 开发的工具，用于展示该算法的实用性和可行性。它利用 Chilkat 库处理所有 HTTP 套接字交互，并使用该库中的其他支持类。此外，该工具还包含一个反混淆器功能，可以分解并逆向 verbosus 模糊哈希，输出全面的 JSON 对象，以及一个比较器功能，输出两个 verbosus 指纹之间的差异。
注意：工具的部分输出可能使用略有不同的探测编号，但底层顺序保持一致：P1->P1, P2->P2, P3->P3, P4->P4, P->P5, P6->P6F, P7->P6L, P8->P7a。
Usage:
HTTP-Basma [OPTION...]
-d, --domain arg domains/IPs (you may query multiple domains, comma separated)
-p, --port arg port number
-s, --ssl does the HTTP connection have to be carried over SSL/TLS?
-q, --qpath check domain with url path included (not recommended)
-w, --redirect enable/disable HTTP redirects. If disabled/false, only the next redirect is followed,
otherwise, all redirects are followed (default: true)
-t, --ctimeout arg socket connection timeout value in seconds (default: 1)
-g, --rtimeout arg socket read (from the server) timeout value in seconds (default: 1)
-e, --sleep arg the duration (in milliseconds) to pause between each request (default: 100)
-x, --proxy arg proxy config: <"socks4|socks5|http">,<domain>,<port>,<bool:direct_tls>,<login>,<pass>
all values are comma-separated. <direct_tls> is ignored with a non-HTTP proxy
-f, --file arg file with list of domains/IPs (requires "-c/--csv" or "-j/--json")
-P, --parallel Scan list of domains passed via the "-f/--file" option in parallel
-c, --csv save to csv file; if the option 'n' is not specified, the CSV filename will be auto
generated
-n, --csvfile arg name of the CSV file
-j, --json save to json file; if the option 'l' is not specified, the JSON filename will be auto
generated
-l, --jsonfile arg name of the JSON file
-r, --saveh save request response headers
-o, --pjson display fingerprint dissection to the console as a JSON object
-i, --demangle_json arg demangle a fingerprint into a detailed json format (you can have more than one, comma
separated)
-u, --demangle_txt arg output a concise text format of the fingerprint, comma-separated for multiple results
-C, --compare arg compare two verbosus fingerprints (comma-separated)
-a, --pacto arg obtain the Pacto fingerprint using Verbosus
-h, --help print usage
当请求给定域名/IP 时，响应可以保存到 CSV 或 JSON 文件中，包含有关服务器响应标头和每个探测唯一指纹的大量信息。
例如，获取服务器 https://google.com 的指纹，并将结果保存到 JSON 和 CSV 文件，同时保存每个探测的 HTTP 响应标头：
HTTPBasma.exe -d https://google.com --json --csv --saveh
在 Output 文件夹中，你会找到 CSV 文件 google_hb_results_2026-05-19_08-35-38_am.csv 和 JSON 文件 google_hb_results_2026-05-19_08-35-38_am.json。
工具的反混淆器功能 -i/--demangle_json 接受一个 verbosus 指纹，重建每个探测的属性，输出全面的 JSON 对象。值得注意的是，当尝试逆向 FNV-1a 哈希时，反混淆器会利用两个本地数据库：options.csv（用于允许的 HTTP 方法）和 status_line_db.csv（用于状态行原因短语）。如果缺少其中任何一个数据库文件，相应的哈希逆向功能将自动禁用。这些数据库是根据对 Majestic 前 100 万网站的扫描结果编译的。
对域名 example.com 的 verbosus 指纹进行反混淆：
HTTPBasma.exe --demangle_json 01140a85e40014514bd522142494d67214254c5e721420958a22020214254c5e720000001609
{
"type": "verbosus",
"fp": "01140a85e40014514bd522142494d67214254c5e721420958a22020214254c5e720000001609",
"p1": {
"type": "get_normal",
"fp": "140a85e400",
"status_line": {
"http_version": {
"fp": "14",
"val_cmt": "HTTP/1.1"
},
"status_code": {
"fp": "0a",
"val_cmt": "200"
},
"http_reason": {
"fp": "85e4",
"val_cmt": "OK"
},
"sl_reversed_db": {
"http_version": "HTTP/1.1",
"status_code": [
200,
404,
403,
500,
204,
999,
888,
603
],
"http_reason": "OK"
}
},
"sts_hdr": {
"fp": "00",
"cmt": "this header is not used"
}
},
"p2": {
"type": "get_invalid_ver_nb",
"fp": "14514bd522",
"status_line": {
"http_version": {
"fp": "14",
"val_cmt": "HTTP/1.1"
},
"status_code": {
"fp": "51",
"val_cmt": "505"
},
"http_reason": {
"fp": "4bd5",
"val_cmt": "HTTP Version Not Supported"
},
"sl_reversed_db": {
"http_version": "HTTP/1.1",
"status_code": [
505
],
"http_reason": "HTTP Version Not Supported"
}
},
"cont_len_hdr": {
"fp": "22",
"name": "Content-Length",
"value": ">1",
"cmt": "content-length/transfer-encoding:chunked header is present with either of the size values: [0,1,>1]"
},
"cnx": {
"ka": false,
"c": true
}
},
"p3": {
"type": "get_rnd_resource",
"fp": "142494d672",
"status_line": {
"http_version": {
"fp": "14",
"val_cmt": "HTTP/1.1"
},
"status_code": {
"fp": "24",
"val_cmt": "404"
},
"http_reason": {
"fp": "94d6",
"val_cmt": "Not Found"
},
"sl_reversed_db": {
"http_version": "HTTP/1.1",
"status_code": [
404,
403,
501,
410,
204,
400,
200,
418
],
"http_reason": "Not Found"
}
},
"cont_len_hdr": {
"fp": "72",
"name": "Transfer-Encoding",
"value": ">1",
"cmt": "content-length/transfer-encoding:chunked header is present with either of the size values: [0,1,>1]"
},
"cnx": {
"ka": true,
"c": false
}
},
"p4": {
"type": "get_rnd_verb",
"fp": "14254c5e72",
"status_line": {
"http_version": {
"fp": "14",
"val_cmt": "HTTP/1.1"
},
"status_code": {
"fp": "25",
"val_cmt": "405"
},
"http_reason": {
"fp": "4c5e",
"val_cmt": "Method Not Allowed"
},
"sl_reversed_db": {
"http_version": "HTTP/1.1",
"status_code": [
405,
403,
204,
418,
404
],
"http_reason": "Method Not Allowed"
}
},
"cont_len_hdr": {
"fp": "72",
"name": "Transfer-Encoding",
"value": ">1",
"cmt": "content-length/transfer-encoding:chunked header is present with either of the size values: [0,1,>1]"
},
"cnx": {
"ka": true,
"c": false
}
},
"p5": {
"type": "get_lowercase_verb",
"fp": "1420958a22",
"status_line": {
"http_version": {
"fp": "14",
"val_cmt": "HTTP/1.1"
},
"status_code": {
"fp": "20",
"val_cmt": "400"
},
"http_reason": {
"fp": "958a",
"val_cmt": "Bad Request"
},
"sl_reversed_db": {
"http_version": "HTTP/1.1",
"status_code": [
400,
422,
405,
401
],
"http_reason": "Bad Request"
}
},
"cont_len_hdr": {
"fp": "22",
"name": "Content-Length",
"value": ">1",
"cmt": "content-length/transfer-encoding:chunked header is present with either of the size values: [0,1,>1]"
},
"cnx": {
"ka": false,
"c": true
}
},
"p6f": {
"type": "get_accept_encoding_full",
"fp": "02",
"cont_enc_hdr": {
"value": "br",
"empty_value": false,
"total_plus": 0
}
},
"p6l": {
"type": "get_accept_encoding_less",
"fp": "02",
"cont_enc_hdr": {
"value": "br",
"empty_value": false,
"total_plus": 0
}
},
"p7a": {
"type": "options_allow_hdr",
"fp": "14254c5e72000000",
"status_line": {
"http_version": {
"fp": "14",
"val_cmt": "HTTP/1.1"
},
"status_code": {
"fp": "25",
"val_cmt": "405"
},
"http_reason": {
"fp": "4c5e",
"val_cmt": "Method Not Allowed"
},
"sl_reversed_db": {
"http_version": "HTTP/1.1",
"status_code": [
405,
403,
204,
418,
404
],
"http_reason": "Method Not Allowed"
}
},
"cont_len_hdr": {
"fp": "72",
"name": "Transfer-Encoding",
"value": ">1",
"cmt": "content-length/transfer-encoding:chunked header is present with either of the size values: [0,1,>1]"
},
"allow_hdr": {
"fp": "000000",
"cmt": "this header is not used"
},
"cnx": {
"ka": true,
"c": false
}
}
}
请注意，"status_code" 数组包含多个 HTTP 状态码。这是因为不同的服务器可能对不同的状态码使用相同的原因短语，从而导致相同的 FNV-1a 哈希。
比较器选项 -C/--compare 比较两个 verbosus 指纹，并输出每个探测主要组件之间的差异。
例如，比较以下 Google 和 YouTube 的两个指纹：
HTTPBasma.exe --compare 01140a85e4001320958a22142494d62214254c5e2214254c5e22080014254c5e220000000000,01140a85e4011320958a22142494d67214254c5e2214254c5e22080014254c5e220000000000
输出结果如下：
< FPrnt-1 Vs. FPrnt-2 >
[ P1 ]
{Strict-Transport-Security}
sts header: 00 != 01
[ P2 ]
[ P3 ]
{Content-Length}
cl_name: 2 != 7
[ P4 ]
[ P5 ]
[ P6F ]
[ P6L ]
[ P7a ]
输出显示差异具体在于 P1 探测的哈希组件，其中一个指纹中存在 STS 标头，而另一个没有。此外，对于 P3 探测，两个指纹之间的 "Content-Length" 编码也不同。
Netomize 提供本仓库公共代码的编译后的 Windows 和 Linux x64 版本。此外，Majestic 前 100 万 HTTP-Basma 指纹 CSV 文件数据集 随首次发布一同提供。
欢迎提交 pull request 和 issue。非常感激您的评论和建议。