#1Web crawling, scraping, spidering, and automated reconnaissance tools for security testing.
Kitploit recommended

XSSCon: Simple XSS Scanner tool
Extracting URLs of a specific target based on the results of "commoncrawl.org"

Browser automation, web crawling, and iOS + Android device control for AI agents. Zig-native, token-efficient CDP snapshots, HAR recording, native…

GUI based offensive penetration testing tool (Open Source)

OWASP D4N155 - Intelligent and dynamic wordlist using OSINT

🕵️ Python project to crawl for JavaScript files and search for secrets like API keys, authorization tokens, hardcoded credentials, etc.

Crawls web applications to detect second-order subdomain takeover vulnerabilities by collecting URLs and matching configurable rules for non-200…

Extracts URLs from OSINT Archives for Security Insights

Github stargazers information gathering tool

Grawler is a tool written in PHP which comes with a web interface that automates the task of using google dorks, scrapes the results, and stores them…

An automated tool which can simultaneously crawl, fill forms, trigger error/debug pages and "loot" secrets out of the client-facing code of sites.

CLI to download websites' actual JS/CSS/assets (not flattened HTML)

A golang utility to spider through a website searching for additional links.

Advanced search in search engines, enables analysis provided to exploit GET / POST capturing emails & urls, with an internal custom validation…

High-performance OSINT/CTI framework for automated identity pivoting and risk analysis across 120+ sources.

High speed/Low cost CommonCrawl RegExp in Node.js

A simple Burp Suite extension to crawl JavaScript (JS) files in passive mode and display the results directly on the issues

Automated data harvesting tool for extracting sensitive information (backups, clones, address books) from web servers via targeted scanning and…