Crawler

koala73/worldmonitor

Real-time global intelligence dashboard. AI-powered news aggregation, geopolitical monitoring, and infrastructure tracking in a unified situational…

d4vinci/scrapling

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

naibowang/easyspider

A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/网页爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。

dgtlmoon/changedetection.io

Best and simplest tool for website change detection, web page monitoring, and website change alerts. Perfect for tracking content changes, price…

qeeqbox/social-analyzer

API, CLI, and Web App for analyzing and finding a person's profile in 1000 social media \ websites

scrapegraphai/scrapegraph-ai

Python scraper based on AI

skyvern-ai/skyvern

Automate browser based workflows with AI

projectdiscovery/nuclei-templates

Community curated list of templates for the nuclei engine to find security vulnerabilities.

s0md3v/photon

Incredibly fast crawler designed for OSINT.

projectdiscovery/katana

A next-generation crawling and spidering framework.

instaloader/instaloader

Download pictures (or videos) along with their captions and other metadata from Instagram.

soxoj/maigret

🕵️‍♂️ Collect a dossier on a person by username from 3000+ sites

daijro/camoufox

🦊 Anti-detect browser

bigbodycobain/shadowbroker

Real-time geospatial OSINT platform aggregating 60+ public telemetry feeds (ADS-B, AIS, satellites, CCTV) into a unified map with server-side recon…

blacklanternsecurity/bbot

The recursive internet scanner for hackers. 🧡

s0md3v/xsstrike

Most advanced XSS scanner.

mishakorzik/allhackingtools

All-in-One Hacking Tools For Hackers! And more hacking tools! For termux.