#1Web crawling, scraping, spidering, and automated reconnaissance tools for security testing.
Kitploit recommended

OSINT tool - gets data from services like shodan, censys etc. in one app
Universal Reddit Scraper - A comprehensive Reddit scraping/archival command-line tool.

The fastest dork scanner written in Go.

Automated OSINT reconnaissance tool that queries Google and social platforms to gather intelligence on usernames and queries, with proxy rotation and…

Advanced cross-platform web crawler for security professionals, enabling automated reconnaissance, information gathering, and OSINT data collection…


Searches dark web and onion search engines for keywords and returns matching onion links, with optional local database storage and Ahmia blacklist…

Generates targeted password wordlists by combining personal info, applying case/leet transforms, and scraping artist lyrics for enhanced cracking…

tool for generating wordlists or extending an existing one using mutations.

Snoopy v2.0 - modular digital terrestrial tracking framework

OSINT python webscaping framework

Simple, but smart, multi-threaded web crawler for randomly gathering huge lists of unique domain names.

PwnXSS: Vulnerability (XSS) scanner exploit

Tool to find JavaScript files on Websites

Python 3 script to dump/scrape/extract company employees from LinkedIn API

OSINT Tool For Scraping Dark Websites

An OSINT search engine for mapping real-world infrastructure from OpenStreetMap data

Automated white-hat vulnerability scanner that monitors HackerOne and Bugcrowd assets, performs subdomain enumeration, crawling, fingerprinting, and…