noptrix/httpgrepAsynchroner HTTP(S)-Scanner, der Antworttexte und -Header über Hosts, Ports, CIDR/Bereiche und TLS-Zertifikat-vhosts hinweg nach Strings oder Regex durchsucht.
Ein schnelles, asynchrones Python-Tool, das HTTP(S)-Server scannt und in HTTP-Antworttexten und -Headern nach Strings oder Regex-Mustern sucht.
Es akzeptiert einzelne Hosts, URLs, CIDR-Bereiche, IP-Bereiche oder Dateien; scannt mehrere Ports pro Ziel (einzeln, Komma-Listen oder Bereiche, mit automatischer Erkennung von TLS vs. Klartext pro Port); kann namensbasierte (v)Hosts direkt aus TLS-Zertifikaten ziehen und scannen; streamt Treffer live ins Terminal; und kann Ergebnisse in Text-, CSV- oder JSONL-Logdateien schreiben.
Es ist für große Scans gebaut: Ein asynchroner Kern steuert tausende gleichzeitige Verbindungen, ein TCP-Preflight überspringt tote Ports kostengünstig, Per-Host- und globale Timeouts verhindern, dass es an langsamen/toten Zielen hängen bleibt, und ein unterbrochener Lauf kann fortgesetzt werden.
termios und
die Unix-Signalbehandlung von asyncio)pip install -r requirements.txt
(oder pip install httpx)uvloop (schnellere Ereignisschleife), aiodns
(nicht blockierendes DNS für -r), h2 (HTTP/2 für -2), httpx[socks] / socksio
(SOCKS-Proxys)httpgrep ist ein einzelnes, in sich geschlossenes Skript - einfach ./httpgrep.py ausführen.
$ httpgrep -H
__ __ __
/ /_ / /_/ /_____ ____ _________ ____
/ __ \/ __/ __/ __ \/ __ `/ ___/ _ \/ __ \
/ / / / /_/ /_/ /_/ / /_/ / / / __/ /_/ /
/_/ /_/\__/\__/ .___/\__, /_/ \___/ .___/
/_/ /____/ /_/
--== [ by nullsecurity.net ] ==--
usage
httpgrep -h <arg> -s <arg> [opts] | <misc>
target options
-h <hosts|file> - single host/url or host-/cidr-range or file containing
hosts or file containing URLs, e.g.: foobar.net,
192.168.0.1-192.168.0.254, 192.168.0.0/24, /tmp/hosts.txt
a comma-separated list of hosts also works, e.g.:
1.2.3.4,foo.net,10.0.0.0/24
NOTE: hosts can also contain ':<ports>' on cmdline or in
file, where <ports> is a single port, comma-list or
range, e.g.: foo.net:8080, foo.net:80,443, 10.0.0.1:1-1024
-p <ports|file> - port(s) to connect to: single port, comma-separated list,
range, or a file with one spec per line, e.g.: 80,
80,443,8080, 8000-8100, /tmp/ports.txt
(default: 80, or 443 when -t is given)
-t - force TLS/SSL on all ports. by default the scheme is
auto-detected per port (plain http, switching to TLS if
the port speaks it)
-u <URI|file> - URI or comma-separated URIs or file with URIs (one per
line) to search given strings in, e.g.: /foobar/,
/foo.html, /admin,/login, /tmp/paths.txt (default: /)
-r - show the reverse-dns (PTR) name of scanned IPv4s as a
label; the ip stays the scan target (no scope drift).
non-blocking with the aiodns package
http options
-X <method> - HTTP request method to use, any case (default: get).
use '?' to list available methods.
-a <user:pass> - http auth credentials (format: 'user:pass')
-U <UA> - set custom User-Agent (default: latest ms edge, windows)
-A - use random user-agent per request
-R <headers> - set custom headers (format: 'foo=bar;lol=lulz;...')
-C <cookies> - set cookies (format: 'foo=bar;lol=lulz;...')
-F - don't follow HTTP redirects
-L <num> - max redirects to follow (default: 10; ignored with -F)
-E - verify TLS/SSL certificates (default: no verification)
-P <proxy> - use proxy (format: '[http|https|socks4|socks5]://host:port')
(socks needs the 'httpx[socks]' / socksio package)
-f <codes> - only report responses with given HTTP status codes,
e.g.: '200', '200,301,302'
-e <codes> - exclude responses with given HTTP status codes,
e.g.: '404', '403,404,500'
-2 - try HTTP/2 (ALPN-negotiated on TLS, falls back to 1.1;
plain http stays 1.1). needs the 'h2' package
search options
-s <str|file> - a single string/regex or multiple strings/regex in a file
to find in HTTP response bodies and headers (see -w),
e.g.: 'tomcat 8', '/tmp/igot0daysforthese.txt'
-S <str|file> - invert (grep -v): drop ALL matches of a response if this
string/regex (or file) appears anywhere in its body or
headers, e.g. to filter out dynamic error / 404 pages
-w <where> - where to search: headers, body, or headers,body
(default: headers,body)
-b <bytes> - num bytes of context to show from a body match
(default: 64)
-m <size> - max body to read + search; suffix b/kb/mb, no suffix = kb,
e.g.: 512, 1mb, 262144b (default: 256kb)
-i - use case-insensitive search
-I - use case-insensitive invert (for -S)
scan options
-x <num> - max concurrent connections (async; default: 300). raise
ulimit -n accordingly for very high values
-c <seconds> - per-host read timeout in seconds, also caps body read
time. the tcp preflight is capped at 2s regardless, so
filtered/dead hosts free their slot fast (default: 3.0)
-G <seconds> - global timeout: hard-stop the whole scan after N seconds
(safety net against any hang; default: none)
-y <num> - retry a failed probe up to <num> times (default: 0).
helps with flaky hosts at scale; keep it small
-1 - once a host has a match, skip its not-yet-started probes
(best-effort; in-flight requests still finish, so under
high -x you may still see a few matches per host)
-z <size> - scan targets in random order within a memory-bounded
window of <size> ram (suffix b/kb/mb/gb), e.g.: -z 1gb.
keeps huge ranges/files from exhausting memory
-Z <num> - cap the -z window at <num> targets (default 2000000,
~267mb at ~140 bytes each). more = wider mixing on huge
ranges, at the cost of ram and start-up buffering
-W - save/resume: on ctrl+c write progress to httpgrep.session;
rerun with -W to resume from it (else start fresh)
-T <0|1> - also probe the cert (v)hosts (CN + SAN) as extra requests
on top of the direct scan. 0 = via Host header on the
same ip (in-scope); 1 = ALSO by dns name/SNI (may leave
scope). needs TLS (https url, -t, or a *443 port).
output options
-l <file> - log found matches to <file>.<fmt> per chosen -O format
(e.g. -l out -O csv,jsonl => out.csv, out.jsonl)
-O <formats> - log file format(s), comma-list of: txt, csv, jsonl
(default: txt; use '?' to list). terminal output always
stays human-readable.
-v - verbose: print each url as it gets scanned
-7 - escape non-ASCII in terminal output to \xNN, so a hostile
response body can't corrupt your terminal (logs stay raw)
misc options
-H - print help
-V - print version information
examples