Web crawling, scraping, spidering, and automated reconnaissance tools for security testing.
hisxo/jspectorA simple Burp Suite extension to crawl JavaScript (JS) files in passive mode and display the results directly on the issues
xorbit01/webpalmMulti-threaded web crawler for structured site mapping and regex-based data extraction. Exports results in JSON, XML, or TXT for reconnaissance and…
vincentkaufmann/noapi-google-search-mcpMCP server for Google search and page fetching using headless Chromium
daffainfo/oneliner-bugbountyA collection oneliner scripts for bug bounty
behindthefirewalls/parseroParsero | Robots.txt audit tool
hahwul/deadfinderFast CLI tool to scan websites, sitemaps, and URL lists for broken links, with concurrent workers, coverage reporting, and multiple output formats.
batuhaniskr/twitter-intelligenceTwitter Intelligence OSINT project performs tracking and analysis of the Twitter
sirbugs/endextExtracts all possible endpoints, URLs, and paths from JavaScript files using customizable regex patterns for web application reconnaissance and API…
atmoner/twitwork✨ Monitor twitter stream from nodejs electron
y0oshi/project-eyes-onProject Eyes On is a high-speed, multi-threaded surveillance tool by Y0oshi (@rde0) for locating open IP cameras worldwide. Unifies Google Dorking…
rodricios/wxpathwxpath - declarative web crawling with XPath; a Web Query Language (WQL)
talyx66/webfangWEBFANG, is my first CLI, a modular OSINT & Reconnaissance toolkit curated for Ethical Hackers and Red-Teamers. Sink fangs into web targets using a…
bhavsec/autopentest-aiAgentic Pentesting MCP server that discovers, exploits, and reports web application vulnerabilities.
hpthreatresearch/subcrawlModular framework for discovering and analyzing open directories on the web. Crawls URLs, extracts content, applies YARA/ClamAV scanning, and outputs…
offensive-hub/black-widowGUI based offensive penetration testing tool (Open Source)
