Crawler

harismuneer/ultimate-social-scrapers

🤖 Top-rated tools to scrape all major public sections from Facebook, Instagram, and Twitter (X) including posts (likes/comments), photos/videos,…

ksharinarayanan/sourcewolf

Amazingly fast response crawler to find juicy stuff in the source code! 😎🔥

ktzgraph/sarenka

OSINT tool - gets data from services like shodan, censys etc. in one app

0xzdh/bridgekeeper

Scrape, Hunt, and Transform names and usernames

paulpierre/informer

A Telegram Mass Surveillance Bot in Python

xorbit01/webpalm

Multi-threaded web crawler for structured site mapping and regex-based data extraction. Exports results in JSON, XML, or TXT for reconnaissance and…

dragonked2/egyscan

Egyscan The Best web vulnerability scanner; it's a multifaceted security powerhouse designed to fortify your web applications against malicious…

joshkar/x-recon

A utility for detecting webpage inputs and conducting XSS scans.

fetchfox/fetchfox

AI-powered web scraping and data extraction library that extracts structured information from any webpage using natural language instructions, with…

s0md3v/uro

declutters url lists for crawling/pentesting

josh0xa/pantheon

IoT Camera Reconnaissance and Live Viewer

mrcl0wnlab/scanner-inurlbr

Advanced search in search engines, enables analysis provided to exploit GET / POST capturing emails & urls, with an internal custom validation…

sanix-darker/falla

A search-engine-cli-scraper for more than 15 search engines, including Google. duckduckgo, Bing, Ask, etc...

hjunker/twittersearch2gephi

This windows CLI app lets you collect data from twitter via REST API and convert it into a CSV data set that can be used with Gephi. Other social…

hehpollon/instagram-crawler

Non API. Crawling post (photo, likes, comments, date ...) by username, hashtags

althonos/instalooter

Another API-less Instagram pictures and videos downloader. (defunct)

datvance/noinstagram

"follow" instagram users/tags without an IG account