HTTP(S) サーバーをスキャンし、HTTP レスポンスのボディとヘッダーから文字列や 正規表現パターンを検索（grep）する、高速で非同期の Python ツールです。
単一ホスト、URL、CIDR 範囲、IP 範囲、またはファイルを受け付けます。ターゲットごとに 複数のポート（単一、カンマ区切りリスト、または範囲）をスキャンし、ポートごとに TLS か平文かを自動検出します。TLS 証明書から直接、名前ベースの (v)ホストを取得して スキャンできます。マッチ結果をターミナルにライブでストリーム表示し、結果をテキスト、 CSV、または JSONL ログファイルに書き出すこともできます。
大規模スキャン向けに設計されています。非同期コアが数千の同時接続を駆動し、TCP プレフライトで応答しないポートを低コストでスキップし、ホストごとおよび全体のタイムアウト により、遅い・応答しないターゲットでハングアップすることがなく、中断した実行も再開 できます。
termios と asyncio の Unix シグナル処理を
使用）上の Python 3.11+pip install -r requirements.txt
（または pip install httpx）uvloop（より高速なイベントループ）、aiodns
（-r 用のノンブロッキング DNS）、h2（-2 用の HTTP/2）、httpx[socks] / socksio
（SOCKS プロキシ）httpgrep は単一の自己完結型スクリプトです - ./httpgrep.py を実行するだけです。
$ httpgrep -H
__ __ __
/ /_ / /_/ /_____ ____ _________ ____
/ __ \/ __/ __/ __ \/ __ `/ ___/ _ \/ __ \
/ / / / /_/ /_/ /_/ / /_/ / / / __/ /_/ /
/_/ /_/\__/\__/ .___/\__, /_/ \___/ .___/
/_/ /____/ /_/
--== [ by nullsecurity.net ] ==--
usage
httpgrep -h <arg> -s <arg> [opts] | <misc>
target options
-h <hosts|file> - single host/url or host-/cidr-range or file containing
hosts or file containing URLs, e.g.: foobar.net,
192.168.0.1-192.168.0.254, 192.168.0.0/24, /tmp/hosts.txt
a comma-separated list of hosts also works, e.g.:
1.2.3.4,foo.net,10.0.0.0/24
NOTE: hosts can also contain ':<ports>' on cmdline or in
file, where <ports> is a single port, comma-list or
range, e.g.: foo.net:8080, foo.net:80,443, 10.0.0.1:1-1024
-p <ports|file> - port(s) to connect to: single port, comma-separated list,
range, or a file with one spec per line, e.g.: 80,
80,443,8080, 8000-8100, /tmp/ports.txt
(default: 80, or 443 when -t is given)
-t - force TLS/SSL on all ports. by default the scheme is
auto-detected per port (plain http, switching to TLS if
the port speaks it)
-u <URI|file> - URI or comma-separated URIs or file with URIs (one per
line) to search given strings in, e.g.: /foobar/,
/foo.html, /admin,/login, /tmp/paths.txt (default: /)
-r - show the reverse-dns (PTR) name of scanned IPv4s as a
label; the ip stays the scan target (no scope drift).
non-blocking with the aiodns package
http options
-X <method> - HTTP request method to use, any case (default: get).
use '?' to list available methods.
-a <user:pass> - http auth credentials (format: 'user:pass')
-U <UA> - set custom User-Agent (default: latest ms edge, windows)
-A - use random user-agent per request
-R <headers> - set custom headers (format: 'foo=bar;lol=lulz;...')
-C <cookies> - set cookies (format: 'foo=bar;lol=lulz;...')
-F - don't follow HTTP redirects
-L <num> - max redirects to follow (default: 10; ignored with -F)
-E - verify TLS/SSL certificates (default: no verification)
-P <proxy> - use proxy (format: '[http|https|socks4|socks5]://host:port')
(socks needs the 'httpx[socks]' / socksio package)
-f <codes> - only report responses with given HTTP status codes,
e.g.: '200', '200,301,302'
-e <codes> - exclude responses with given HTTP status codes,
e.g.: '404', '403,404,500'
-2 - try HTTP/2 (ALPN-negotiated on TLS, falls back to 1.1;
plain http stays 1.1). needs the 'h2' package
search options
-s <str|file> - a single string/regex or multiple strings/regex in a file
to find in HTTP response bodies and headers (see -w),
e.g.: 'tomcat 8', '/tmp/igot0daysforthese.txt'
-S <str|file> - invert (grep -v): drop ALL matches of a response if this
string/regex (or file) appears anywhere in its body or
headers, e.g. to filter out dynamic error / 404 pages
-w <where> - where to search: headers, body, or headers,body
(default: headers,body)
-b <bytes> - num bytes of context to show from a body match
(default: 64)
-m <size> - max body to read + search; suffix b/kb/mb, no suffix = kb,
e.g.: 512, 1mb, 262144b (default: 256kb)
-i - use case-insensitive search
-I - use case-insensitive invert (for -S)
scan options
-x <num> - max concurrent connections (async; default: 300). raise
ulimit -n accordingly for very high values
-c <seconds> - per-host read timeout in seconds, also caps body read
time. the tcp preflight is capped at 2s regardless, so
filtered/dead hosts free their slot fast (default: 3.0)
-G <seconds> - global timeout: hard-stop the whole scan after N seconds
(safety net against any hang; default: none)
-y <num> - retry a failed probe up to <num> times (default: 0).
helps with flaky hosts at scale; keep it small
-1 - once a host has a match, skip its not-yet-started probes
(best-effort; in-flight requests still finish, so under
high -x you may still see a few matches per host)
-z <size> - scan targets in random order within a memory-bounded
window of <size> ram (suffix b/kb/mb/gb), e.g.: -z 1gb.
keeps huge ranges/files from exhausting memory
-Z <num> - cap the -z window at <num> targets (default 2000000,
~267mb at ~140 bytes each). more = wider mixing on huge
ranges, at the cost of ram and start-up buffering
-W - save/resume: on ctrl+c write progress to httpgrep.session;
rerun with -W to resume from it (else start fresh)
-T <0|1> - also probe the cert (v)hosts (CN + SAN) as extra requests
on top of the direct scan. 0 = via Host header on the
same ip (in-scope); 1 = ALSO by dns name/SNI (may leave
scope). needs TLS (https url, -t, or a *443 port).
output options
-l <file> - log found matches to <file>.<fmt> per chosen -O format
(e.g. -l out -O csv,jsonl => out.csv, out.jsonl)
-O <formats> - log file format(s), comma-list of: txt, csv, jsonl
(default: txt; use '?' to list). terminal output always
stays human-readable.
-v - verbose: print each url as it gets scanned
-7 - escape non-ASCII in terminal output to \xNN, so a hostile
response body can't corrupt your terminal (logs stay raw)
misc options
-H - print help
-V - print version information
examples
# grep for 'apache' in headers and body of a single host
$ httpgrep -h foobar.net -s apache
# scan a CIDR range on port 8080, search for 'tomcat' in body only
$ httpgrep -h 192.168.0.0/24 -p 8080 -s tomcat -w body
# scan a host across multiple ports and a port range for 'jenkins'
$ httpgrep -h 192.168.0.10 -p 80,443,8080,8000-8100 -s jenkins -i
# scan host list, search string file, log matches (-> /tmp/out.txt)
$ httpgrep -h /tmp/hosts.txt -s /tmp/strings.txt -x 200 -l /tmp/out