Crawler

dwisiswant0/galer

A fast tool to fetch URLs from HTML attributes by crawl-in.

xroche/httrack

HTTrack Website Copier, copy websites to your computer (Official repository)

topscoder/nuclei-wordfence-cve

80k+ WordPress Nuclei templates, updated daily from Wordfence intel—filter by severity/tags/CVE and scan in one line. 🚀🔒

l4rm4nd/linkedindumper

Python 3 script to dump/scrape/extract company employees from LinkedIn API

owasp/d4n155

OWASP D4N155 - Intelligent and dynamic wordlist using OSINT

mfmokbel/crawlector

Crawlector is a threat hunting framework designed for scanning websites for malicious objects.

d4vinci/scrapling

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

soxoj/maigret

🕵️‍♂️ Collect a dossier on a person by username from 3000+ sites

s0md3v/xsstrike

Most advanced XSS scanner.

hakluke/hakrawler

Simple, fast web crawler designed for easy, quick discovery of endpoints and assets within a web application

ultrafunkamsterdam/nodriver

Successor of Undetected-Chromedriver. Providing a blazing fast framework for web automation, webscraping, bots and any other creative ideas which are…

harismuneer/ultimate-social-scrapers

🤖 Top-rated tools to scrape all major public sections from Facebook, Instagram, and Twitter (X) including posts (likes/comments), photos/videos,…

thewhiteh4t/finalrecon

Automated web reconnaissance tool providing header analysis, DNS enumeration, subdomain discovery, directory scanning, SSL inspection, and port…

s0md3v/uro

declutters url lists for crawling/pentesting

hackmanit/web-cache-vulnerability-scanner

Go-based CLI scanner for web cache poisoning and deception. Supports 10 poisoning techniques, multiple deception methods, built-in crawler, JSON…

vigolium/vigolium

Vigolium - High-fidelity vulnerability scanner fusing agentic AI with native speed, modularity, and precision

khashashin/ogi

Open Source Link Analysis & OSINT Framework