Crawler

d4vinci/scrapling

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

sqlmapproject/sqlmap

Automatic SQL injection and database takeover tool

microsoft/playwright

Browser automation framework with a single API for Chromium, Firefox, and WebKit. Supports web testing, scraping, screenshots, and network…

naibowang/easyspider

A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/网页爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。

projectdiscovery/nuclei

Fast YAML-based vulnerability scanner with template-driven detection engine for automated security testing across web apps, APIs, networks, DNS, and…

soxoj/maigret

🕵️‍♂️ Collect a dossier on a person by username from 3000+ sites

dgtlmoon/changedetection.io

Best and simplest tool for website change detection, web page monitoring, and website change alerts. Perfect for tracking content changes, price…

scrapegraphai/scrapegraph-ai

Python scraper based on AI

qeeqbox/social-analyzer

API, CLI, and Web App for analyzing and finding a person's profile in 1000 social media \ websites

zaproxy/zaproxy

Open-source web application security scanner for automated vulnerability detection, manual penetration testing, and API security testing with a…

skyvern-ai/skyvern

Automate browser based workflows with AI

projectdiscovery/nuclei-templates

Community curated list of templates for the nuclei engine to find security vulnerabilities.

ffuf/ffuf

Fast web fuzzer written in Go

maurosoria/dirsearch

Web path scanner

s0md3v/photon

Incredibly fast crawler designed for OSINT.

mishakorzik/allhackingtools

All-in-One Hacking Tools For Hackers! And more hacking tools! For termux.