Crawler

al0ne/vxscan

python3写的综合扫描工具，主要用来存活验证，敏感文件探测(目录扫描/js泄露接口/html注释泄露)，WAF/CDN识别，端口扫描，指纹/服务识别，操作系统识别，POC扫描，SQL注入，绕过CDN，查询旁站等功能，主要用来甲方自测或乙方授权测试，请勿用来搞破坏。

harismuneer/ultimate-social-scrapers

🤖 Top-rated tools to scrape all major public sections from Facebook, Instagram, and Twitter (X) including posts (likes/comments), photos/videos,…

eldraco/domain_analyzer

Analyze the security of any domain by finding all the information possible. Made in python.

1tayh/noisy

Simple random DNS, HTTP/S internet traffic noise generator

qianlitp/crawlergo

A powerful browser crawler for web vulnerability scanners

paulpierre/informer

A Telegram Mass Surveillance Bot in Python

megadose/onionsearch

OnionSearch is a script that scrapes urls on different .onion search engines.

jaeles-project/gospider

Gospider - Fast web spider written in Go

laramies/metagoofil

Metadata harvester

topscoder/nuclei-wordfence-cve

80k+ WordPress Nuclei templates, updated daily from Wordfence intel—filter by severity/tags/CVE and scan in one line. 🚀🔒

projectdiscovery/wappalyzergo

A high performance go implementation of Wappalyzer Technology Detection Library

anouarbensaad/vulnx

vulnx 🕷️ an intelligent Bot, Shell can achieve automatic injection, and help researchers detect security vulnerabilities CMS system. It can perform…

ge0rg3/requests-ip-rotator

A Python library to utilize AWS API Gateway's large IP pool as a proxy to generate pseudo-infinite IPs for web scraping and brute forcing.

soxoj/socid-extractor

⛏️ The extraction engine behind Maigret: turn any profile URL into a structured OSINT record across 150+ sites

althonos/instalooter

Another API-less Instagram pictures and videos downloader. (defunct)

0xinfection/xsrfprobe

The Prime Cross Site Request Forgery (CSRF) Audit and Exploitation Toolkit.

spidersuite/spidersuite

Advanced cross-platform web crawler for security professionals, enabling automated reconnaissance, information gathering, and OSINT data collection…