Skip to content
KitploitKITPLOIT
HerramientasExploitsBlog
Log in
Enviar
HerramientasExploitsBlog
Enviar

¡Herramientas de Hacking, PenTest y Ciberseguridad para tu Arsenal de Seguridad!

Kitploit es un directorio de herramientas de hacking, ciberseguridad y pentesting. Descubre las últimas actualizaciones de proyectos para encontrar vulnerabilidades, analizar sistemas, automatizar pruebas y fortalecer tu seguridad.

FeedsContactoPrivacidad© 2026 Kitploit

Directorio de Herramientas

Categorías

Ver todas las categorías
Loading categories
grubcrawler — El rastreador agéntico más rápido del mundo. Recuperado. Reinventado. Listo para la guerra. | Kitploit
Herramientas/GitHubGitHub/deepbluedynamics/grubcrawler
OSINT (Inteligencia de Fuentes Abiertas)ReconocimientoAnálisis Dinámico (Sandboxing)Proxies Web e InterceptaciónRecopilación de InformaciónSeguridad WebPruebas de PenetraciónUtilidades y FrameworksAprendizaje AutomáticoRed TeamingCrawler
34757hace 1 mesRevisado por Kitploit
Compartir
Anti-Bot
GitHubdeepbluedynamics/grubcrawler

grubcrawler

El rastreador agéntico más rápido del mundo. Recuperado. Reinventado. Listo para la guerra.

Ver RepositorioSitio web
Contenido no disponible en el idioma solicitado. Mostrando versión en inglés.
Grub Crawler

License Python FastAPI Playwright MCP Ghost Protocol Live Stream Camoufox Proxy


Agentic web crawler with anti-detection, vision fallback, and peer-to-peer mesh.



Full-stack web crawling engine with JavaScript rendering, Camoufox anti-detect browser, per-request proxy routing, and autonomous agent loops. Converts pages to clean markdown with a native Rust extraction engine. When standard crawling is blocked by Cloudflare, CAPTCHAs, or JavaScript walls, Ghost Protocol captures a screenshot and extracts content via vision AI (Claude, GPT-4o, or Ollama). Supports multi-provider LLM orchestration across OpenAI, Anthropic, and Ollama in a single session. Nodes coordinate over a gossip-based peer-to-peer mesh for distributed crawling.


An open source project from DeepBlue Dynamics, which builds open source agentic tooling for the marine electronics market. DeepBlue's promise: agents and people can ask questions of a boat's instruments, logs and documents, on board and without a connection.

Run it

# Full image: Camoufox (stealth Firefox) + Chromium. linux/amd64 and linux/arm64.
docker run -d --name grub -p 6792:6792 -e DISABLE_AUTH=true deepbluedynamics/grubcrawler:latest

# Lite image: Chromium only, about half the size. Raspberry Pi 5 and other small hosts.
docker run -d --name grub -p 6792:6792 -e DISABLE_AUTH=true deepbluedynamics/grubcrawler:latest-lite

curl http://localhost:6792/health

DISABLE_AUTH=true is for your own machine. Anywhere else, leave it unset and put the service behind nuts-auth tokens (Authorization: Bearer ahp_…), which is how the hosted instance at https://grub.nuts.services works.

Add it to your agent (MCP)

grub is an MCP server (streamable HTTP) at /mcp/ exposing grub_crawl, grub_screenshot and grub_diagnose. Point your harness at http://localhost:6792/mcp/ for your own container, or at https://grub.nuts.services/mcp/ with a bearer token for the hosted one.

Claude Code

claude mcp add --transport http grub http://localhost:6792/mcp/
# hosted
claude mcp add --transport http --header "Authorization: Bearer ahp_yourtoken" grub https://grub.nuts.services/mcp/

Cursor — .cursor/mcp.json

{ "mcpServers": { "grub": { "url": "http://localhost:6792/mcp/",
                          "headers": { "Authorization": "Bearer ahp_yourtoken" } } } }

VS Code (Copilot) — .vscode/mcp.json

{ "servers": { "grub": { "type": "http", "url": "http://localhost:6792/mcp/",
                       "headers": { "Authorization": "Bearer ahp_yourtoken" } } } }

Claude Desktop — claude_desktop_config.json, through mcp-remote

{ "mcpServers": { "grub": { "command": "npx",
    "args": ["-y", "mcp-remote", "https://grub.nuts.services/mcp/", "--header", "Authorization: Bearer ahp_yourtoken"] } } }

Codex CLI — ~/.codex/config.toml

[mcp_servers.grub]
url = "https://grub.nuts.services/mcp/"
bearer_token_env_var = "GRUB_TOKEN"   # export GRUB_TOKEN=ahp_yourtoken

Gemini CLI — ~/.gemini/settings.json

{ "mcpServers": { "grub": { "httpUrl": "http://localhost:6792/mcp/",
                          "headers": { "Authorization": "Bearer ahp_yourtoken" } } } }

The HTTP API has more surface than the MCP tools (batch, PDF pages, jobs, agent runs); see API Endpoints. Scanned-PDF OCR needs a vision provider: Ollama on the host (benhaotang/Nanonets-OCR-s, the docker-compose default) or an ANTHROPIC_API_KEY / OPENAI_API_KEY.

First crawl over HTTP

curl -X POST http://localhost:6792/api/markdown \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "customer_id": "me"}'

# a PDF comes back as markdown, one "## Page N" section per page
curl -X POST http://localhost:6792/api/markdown \
  -H "Content-Type: application/json" \
  -d '{"url": "https://arxiv.org/pdf/1706.03762", "customer_id": "me"}'

Images

TagBrowsersSize, unpacked (compressed pull)Platforms
deepbluedynamics/grubcrawler:latest, :<version>Camoufox (Firefox) + Playwright/Patchright Chromium~5 GB (2.9 GB)linux/amd64, linux/arm64
deepbluedynamics/grubcrawler:latest-lite, :<version>-litePlaywright/Patchright Chromium only~2.4 GB amd64, ~2.8 GB arm64 (1.2 GB)linux/amd64, linux/arm64

Both share one codebase and API. The lite image sets BROWSER_ENGINE=chromium; the full image adds the anti-detect Firefox engine, Xvfb and Ghost Protocol's stronger stealth. Build either yourself with docker build . or docker build -f Dockerfile.lite .; the release workflow publishes both for both architectures on every version tag.


Why Grub

We integrated features from every major crawler — then added what none of them have.

Self-Hosted Crawlers

FeatureCrawl4AIFirecrawlScrapyGrub
JS rendering✅ Playwright✅ Playwright❌ HTTP only✅ Playwright
Anti-detect browserstealth plugin❌❌✅ Camoufox
Ghost Protocol❌❌❌✅ auto fallback
Per-request proxy✅ escalation❌middleware✅ per-request
Stealth patches✅❌❌✅ opt-in
Agent loop✅ agentic✅ /agent❌ spiders✅ bounded SM
Live browser stream✅ WebSocket✅ Live View❌✅ WS + MJPEG
Markdown output✅ Fit Markdown✅ core❌✅ Rust engine
PDF extraction✅ PDF strategy✅ parse❌✅ text layer + OCR fallback
MCP tools✅ community✅ official⚠️ community✅ 15 tools
Multi-provider LLM✅ all LLMs⚠️ Gemini❌✅ OpenAI/Anthropic/Ollama
Mesh P2P❌❌❌✅ gossip protocol
Policy enforcement❌❌❌✅ domain gates + redaction
Prompt injection defense❌❌❌✅ quarantine + visible-text diff
LicenseApache 2.0AGPL-3.0BSDBSD-3-Clause
PricingFreeFree–$333/moFreeSelf-hosted

Cloud / Managed Crawlers

Descargar herramienta

Más Populares

Ver todos →

Descubre las herramientas más usadas por nuestra comunidad.

  • 1

    OsintHunter

    letox4/osinthunter

    🎯 El kit de herramientas OSINT definitivo: grafo de investigación, puntuación de confianza, familias de alias, mapa, línea de tiempo e informes. 49 módulos, más de 400 sitios, sin necesidad de claves API.

    524
  • 2

    ghostlock-app

    yukonga/ghostlock-app

    # Aplicación de Ejecución con un Solo Toque GhostLock (CVE-2026-43499)

    478
  • 3

    ARTEX

    baarai/artex

    Plataforma de pruebas de penetración autónoma multiagente basada en LLM con agentes planificadores/trabajadores, grafos de activos y de exploración, registro de tráfico MITM y reevaluación de vulnerabilidades.

    357
  • 4

    redforge

    cybersecurity91/redforge

    Kit de herramientas de red team del lado del navegador con 102 herramientas offline para reconocimiento, generación de payloads, ataques de credenciales, criptografía, codificación y elaboración de informes de compromiso.

    317
  • 5

    estorides

    grisuno/estorides

    Agrega más de 99 fuentes OSINT gratuitas en paralelo, resuelve entidades en un grafo de conocimiento y exporta STIX 2.1, MISP o GraphML con análisis asistido por LLM.

    305
  • 6

    opencti

    opencti-platform/opencti

    Plataforma abierta de inteligencia de amenazas cibernéticas

    275

Explora todas las herramientas

Explora nuestra colección de herramientas

Ver todas las herramientas →