Crawler

s0md3v/goop

Google Search Scraper

krmaxwell/maltrieve

A tool to retrieve malware directly from the source for security researchers.

aancw/belati

The Traditional Swiss Army Knife for OSINT

kgretzky/dcrawl

Simple, but smart, multi-threaded web crawler for randomly gathering huge lists of unique domain names.

dirtyfilthy/freshonions-torscraper

Fresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion

robre/scripthunter

Tool to find JavaScript files on Websites

ni5arga/sightline

An OSINT search engine for mapping real-world infrastructure from OpenStreetMap data

itsmehacker/darkscrape

OSINT Tool For Scraping Dark Websites

vida-nyu/ache

Focused web crawler that uses page classifiers and link prioritization to efficiently collect domain-specific or pattern-matching web pages, with…

c0dejump/hawkscan

Security Tool for Reconnaissance and Information Gathering on a website. (python 3.x)

s0md3v/sqlmate

A friend of SQLmap which will do what you always expected from SQLmap.

kahunalu/pwnbin

Python Pastebin Webcrawler that returns list of public pastebins containing keywords

sensepost/snoopy-ng

Snoopy v2.0 - modular digital terrestrial tracking framework

0xdsm/pinkerton

🕵️ Python project to crawl for JavaScript files and search for secrets like API keys, authorization tokens, hardcoded credentials, etc.

anon-exploiter/sitebroker

A cross-platform python based utility for information gathering and penetration testing automation!

josue87/emailfinder

Search emails from a domain through search engines

justicerage/freedomfighting

A collection of scripts which may come in handy during your freedom fighting activities.