#1Web crawling, scraping, spidering, and automated reconnaissance tools for security testing.
Kitploit recommended

Yet another Ransomware gang tracker
A Python library to utilize AWS API Gateway's large IP pool as a proxy to generate pseudo-infinite IPs for web scraping and brute forcing.

Fetch many paths for many hosts - without killing the hosts

A high-speed tool for passively gathering URLs, optimized for efficient and comprehensive web asset discovery without active scanning.

Self-hosted scraping engine — bypasses any JS challenge & captcha: Cloudflare, Turnstile, reCAPTCHA, hCaptcha, GeeTest. FlareSolverr & Byparr…

Port of Wappalyzer (uncovers technologies used on websites) to automate mass scanning.

⡷⠂𝚔𝚊𝚛𝚖𝚊 𝚟𝟸⠐⢾ is a Passive Open Source Intelligence (OSINT) Automated Reconnaissance (framework)

Web application security scanner created by lcamtuf for google - Unofficial Mirror

The Prime Cross Site Request Forgery (CSRF) Audit and Exploitation Toolkit.

The AI toolkit for building reliable browser automations

A python tool used to discover endpoints, potential parameters, a target specific wordlist for a given target and secrets

A Bash script and Docker image for Bug Bounty reconnaissance. Intended for headless use.

declutters url lists for crawling/pentesting

Collects email addresses for a target domain by querying search engines, GitHub API, PhoneBook, Snov, Skymem and other OSINT sources, saving results…

Go-based CLI scanner for web cache poisoning and deception. Supports 10 poisoning techniques, multiple deception methods, built-in crawler, JSON…

A ready to use JSONP endpoints/payloads to help bypass content security policy (CSP) of different websites.

Making Favicon.ico based Recon Great again !

a javascript change monitoring tool for bugbounties