hahwul/deadfinderFerramenta CLI rápida para escanear sites, sitemaps e listas de URLs em busca de links quebrados, com workers concorrentes, relatório de cobertura e múltiplos formatos de saída.
Encontre dead-links (links quebrados)
Documentação • Instalação • Github Action • Contribuindo • Changelog
Dead link (link quebrado) significa um link dentro de uma página web que não pode ser conectado. Esses links podem ter um impacto negativo no SEO e na Segurança. Esta ferramenta facilita a identificação e a modificação.
Procurando a v1 (Ruby gem)? Ela agora reside no branch
legacy/v1e continua a publicar a gemdeadfinderpara releases de correção de bugs e segurança. Omainhospeda a reescrita em Crystal (v2+).
brew install deadfinder
# https://formulae.brew.sh/formula/deadfinder
A partir da versão posterior à 2.1.0:
choco install deadfinder
docker run ghcr.io/hahwul/deadfinder:latest deadfinder url https://example.com
Baixe o arquivo para sua plataforma na última versão, extraia e coloque deadfinder (deadfinder.exe no Windows) no seu PATH.
nix run github:hahwul/deadfinder
nix profile install github:hahwul/deadfinder
nix develop github:hahwul/deadfinder
Requer Crystal >= 1.19.1 e cmake (para o postinstall do parser HTML lexbor — sem ele, shards install falha com 'cmake': No such file or directory).
# macOS
brew install crystal cmake
# Debian / Ubuntu
sudo apt install crystal cmake
shards install
crystal build src/cli_main.cr -o deadfinder --release
# or: just build
deadfinder sitemap https://www.hahwul.com/sitemap.xml
Fixe uma tag de release específica. @latest não é uma ref válida para Actions.
steps:
- name: Run DeadFinder
uses: hahwul/deadfinder@v2 # tracks the latest 2.x — pin a specific tag (e.g. @2.1.0) for stricter reproducibility
id: broken-link
with:
command: sitemap # url / file / sitemap / pipe
target: https://www.hahwul.com/sitemap.xml
# timeout: 10
# concurrency: 50
# silent: false
# headers: "X-API-Key: 123444"
# worker_headers: "User-Agent: Deadfinder Bot"
# include30x: false
# user_agent: "Apple"
# proxy: "http://localhost:8070"
# proxy_auth: "id:pw"
# insecure: false
# match: ""
# ignore: ""
# limit: 0
# output_format: json
# coverage: true
# visualize: report.png
# fail_on_dead: true # fail the step when a dead link is found
# method: auto # auto / head / get
# retry: 2
# delay: 0 # ms between requests to the same host
# accept_status: "403,999"
# dead_status: ""
# target_concurrency: 10
# check_anchors: false
- name: Output Handling
run: echo '${{ steps.broken-link.outputs.output }}'
Se você encontrou um Dead Link e deseja adicioná-lo automaticamente como uma issue, consulte o artigo "Automating Dead Link Detection".
Usage: deadfinder <command> [options]
Commands:
pipe Scan the URLs from STDIN
file <FILE> Scan the URLs from File (`-` for STDIN)
url <URL> Scan the Single URL
sitemap <SITEMAP-URL> Scan the URLs from sitemap
completion <SHELL> Generate completion script (bash/zsh/fish)
version Show version
Options:
-r, --include30x Include 30x redirections as dead links
-c, --concurrency=N Number of concurrent workers (default: 50)
--target-concurrency=N Targets scanned in parallel; total in-flight
requests stay capped at -c (default: 10)
-t, --timeout=N Timeout in seconds (default: 10)
--method=METHOD Link check method: auto, head, get (default: auto)
--retry=N Retry transient failures N times (default: 2)
--delay=MS Minimum ms between requests to the same host
--accept-status=LIST Statuses to treat as alive (e.g. 200,403,999)
--dead-status=LIST Statuses to treat as dead (e.g. 500-599)
--exclude-status=LIST Alias of --dead-status
-o, --output=FILE File to write results (`-` for stdout)
-f, --output_format=FORMAT Output format: json, yaml, toml, csv, sarif (default: json)
-H, --headers=HEADER Custom HTTP headers for initial request
--worker_headers=HEADER Custom HTTP headers for worker requests
--user_agent=UA User-Agent string
-p, --proxy=PROXY Proxy server (HTTP and HTTPS CONNECT)
--proxy_auth=USER:PASS Proxy authentication
-k, --insecure Skip TLS certificate verification (not recommended)
-m, --match=PATTERN Match URL pattern (regex)
-i, --ignore=PATTERN Ignore URL pattern (regex)
-s, --silent Silent mode
-v, --verbose Verbose mode
--debug Debug mode
--limit=N Limit number of URLs to scan
--check-anchors Verify #fragment targets exist in the linked document
--coverage Enable coverage tracking and reporting
-F, --fail-on-dead Exit with code 2 when anything dead is found
--visualize=PATH Generate visualization PNG
-h, --help Show help
# Scan the URLs from STDIN (multiple URLs)
cat urls.txt | deadfinder pipe
# Scan the URLs from a file
deadfinder file urls.txt
# `-` and other non-regular files (process substitution, /dev/stdin, FIFOs) work too
deadfinder file - < urls.txt
deadfinder file <(subfinder -d example.com -silent | httpx -silent)
# Scan a single URL
deadfinder url https://www.hahwul.com
# Scan the URLs from a sitemap
deadfinder sitemap https://www.hahwul.com/sitemap.xml
pipe / file)Uma URL por linha. A lista é normalizada antes da verificação:
# são ignoradoshttp:// / https:// são reportadas e ignoradas--limit N conta apenas alvos reais e interrompe a leitura da entrada assim que o limite é atingido# production
https://www.hahwul.com
https://www.hahwul.com/cullinan/
# staging
https://staging.hahwul.com
deadfinder sitemap https://www.hahwul.com/sitemap.xml -o output.json
cat output.json | jq