Crawler

sirbugs/endext

Extracts all possible endpoints, URLs, and paths from JavaScript files using customizable regex patterns for web application reconnaissance and API…

hueristiq/xcrawl3r

Recursively spider webpages to discover all URLs by following links, parsing sitemaps, and robots.txt files. Ideal for security reconnaissance and…

tegal1337/mapsdumper

Dump place details from Google Maps like phone,email,website,and reviews

soxoj/tiktok-scraper

Scrape and download TikTok video posts, user profiles, hashtag trends, and music feed metadata via CLI or Node.js module. Supports batch downloads,…

l4rm4nd/linkedindumper

Python 3 script to dump/scrape/extract company employees from LinkedIn API

h33tlit/jbin-website-secret-scraper

Jbin will gather all the URLs from the website and then it will try to expose the secret data from them such as API keys, API secrets, API tokens and…

josephlai241/urs

Universal Reddit Scraper - A comprehensive Reddit scraping/archival command-line tool.

mhmdiaa/second-order

Crawls web applications to detect second-order subdomain takeover vulnerabilities by collecting URLs and matching configurable rules for non-200…

s4dhulabs/vimana-framework

Vimana is a modular security framework for auditing Python APIs and Web applications. The plugin-based architecture enables security professionals to…

morph3/crawpy

Asynchronous HTTP content discovery tool with auto-calibration, recursive scanning, and report generation for enumerating hidden web directories and…

anouarbensaad/wsvuls

CLI website vulnerability scanner that detects outdated server software, insecure HTTP headers, and extracts Cloudflare IP/port data with performance…

soxoj/socid-extractor

⛏️ The extraction engine behind Maigret: turn any profile URL into a structured OSINT record across 150+ sites

hackmanit/web-cache-vulnerability-scanner

Go-based CLI scanner for web cache poisoning and deception. Supports 10 poisoning techniques, multiple deception methods, built-in crawler, JSON…

r4ygm/dorkscout

Automated Google dork scanner that fetches exploit-db dork lists and scans targets or the entire internet for vulnerable applications, secret files,…

hpthreatresearch/subcrawl

Modular framework for discovering and analyzing open directories on the web. Crawls URLs, extracts content, applies YARA/ClamAV scanning, and outputs…

mishakorzik/allhackingtools

All-in-One Hacking Tools For Hackers! And more hacking tools! For termux.

yogeshojha/rengine

reNgine is an automated reconnaissance framework for web applications with a focus on highly configurable streamlined recon process via Engines,…