기능

빠르고 완전히 구성 가능한 웹 크롤링
Standard 및 Headless 모드
JavaScript 파싱 / 크롤링
사용자 정의 가능한 자동 폼 채우기
범위 제어 - 사전 구성된 필드 / Regex
지식 베이스 - ML 페이지 유형 / 폼 분류 (자동 다운로드 모델)
사용자 정의 가능한 출력 - 사전 구성된 필드
INPUT - STDIN, URL 및 LIST
OUTPUT - STDOUT, FILE 및 JSON

설치

katana를 성공적으로 설치하려면 Go 1.26+가 필요합니다. 설치 문제가 발생하면 최신 버전의 Go로 시도해 보는 것을 권장합니다. 최소 요구 버전이 변경되었을 수 있기 때문입니다. 아래 명령을 실행하거나 릴리스 페이지에서 미리 컴파일된 바이너리를 다운로드하세요.```console CGO_ENABLED=1 go install github.com/projectdiscovery/katana/cmd/katana@latest

**katana- 설치 / 실행을 위한 추가 옵션**

<details>
  <summary>Docker</summary>

> 최신 태그로 docker를 설치 / 업데이트하려면 -```sh
docker pull projectdiscovery/katana:latest

docker를 사용하여 표준 모드로 katana를 실행하려면 -```sh docker run projectdiscovery/katana:latest -u https://tesla.com

> docker를 사용하여 headless 모드에서 katana를 실행하려면 -```sh
docker run projectdiscovery/katana:latest -u https://tesla.com -system-chrome -headless

Ubuntu

다음 필수 구성 요소를 설치하는 것이 권장됩니다 -```sh sudo apt update sudo apt install zip curl wget git snapd sudo snap refresh sudo snap install golang --classic

sudo install -d -m 0755 /etc/apt/keyrings curl -fsSL https://dl.google.com/linux/linux_signing_key.pub
| sudo gpg --dearmor -o /etc/apt/keyrings/google-chrome.gpg

echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/google-chrome.gpg]
http://dl.google.com/linux/chrome/deb/ stable main"
| sudo tee /etc/apt/sources.list.d/google-chrome.list > /dev/null

sudo apt update sudo apt install google-chrome-stable

> katana 설치 -```sh
go install github.com/projectdiscovery/katana/cmd/katana@latest

사용법```console

katana -h

이것은 도구에 대한 도움말을 표시합니다. 다음은 도구가 지원하는 모든 스위치입니다.```console
Katana is a fast crawler focused on execution in automation
pipelines offering both headless and non-headless crawling.

Usage:
  ./katana [flags]

Flags:
INPUT:
   -u, -list string[]     target url / list to crawl
   -resume string         resume scan using resume.cfg
   -e, -exclude string[]  exclude host matching specified filter ('cdn', 'private-ips', cidr, ip, regex)

CONFIGURATION:
   -r, -resolvers string[]       list of custom resolver (file or comma separated)
   -d, -depth int                maximum depth to crawl (default 3)
   -jc, -js-crawl                enable endpoint parsing / crawling in javascript file
   -jsl, -jsluice                enable jsluice parsing in javascript file (memory intensive)
   -ct, -crawl-duration value    maximum duration to crawl the target for (s, m, h, d) (default s)
   -kf, -known-files string      enable crawling of known files (all,robotstxt,sitemapxml), a minimum depth of 3 is required to ensure all known files are properly crawled.
   -mrs, -max-response-size int  maximum response size to read (default 4194304)
   -timeout int                  time to wait for request in seconds (default 10)
   -aff, -automatic-form-fill    enable automatic form filling (experimental)
   -fx, -form-extraction         extract form, input, textarea & select elements in jsonl output
   -retry int                    number of times to retry the request (default 1)
   -proxy string                 http/socks5 proxy to use
   -td, -tech-detect             enable technology detection
   -H, -headers string[]         custom header/cookie to include in all http request in header:value format (file)
   -config string                path to the katana configuration file
   -fc, -form-config string      path to custom form configuration file
   -flc, -field-config string    path to custom field configuration file
   -s, -strategy string          Visit strategy (depth-first, breadth-first) (default "depth-first")
   -iqp, -ignore-query-params    Ignore crawling same path with different query-param values
   -fsu, -filter-similar         filter crawling of similar looking URLs (e.g., /users/123 and /users/456)
   -fst, -filter-similar-threshold int  number of distinct values before a path position is treated as parameter (default 10)
   -tlsi, -tls-impersonate       enable experimental client hello (ja3) tls randomization
   -dr, -disable-redirects       disable following redirects (default false)
   -pcs, -page-content-similar   enable page content similarity filtering (simhash|tfidf|bm25)
   -pcsm, -page-content-similar-mode string  similarity mode: simhash, tfidf, or bm25 (default simhash)
   -pcsd, -page-content-similar-distance int  simhash max hamming distance (default 3)
   -pcst, -page-content-similar-threshold float  tfidf/bm25 min score 0-1 (default 0.85)
   -pcsn, -page-content-similar-budget int  pages to fully process per similarity cluster (default 1)
   -sdd, -similarity-deduplication  alias for -pcs
   -kb, -knowledge-base          enable knowledge base classification
   -kb-secrets                   enable secrets extractor in the knowledge base
   -kb-validate-secrets          validate detected secrets against their provider (sends live API calls)
   -kb-endpoints                 enable endpoints extractor (classifies REST/GraphQL/SOAP/XHR requests)
   -mdp, -max-domain-pages int   maximum number of pages to crawl per domain (default unlimited)

DEBUG:
   -health-check, -hc        run diagnostic check up
   -elog, -error-log string  file to write sent requests error log
   -pprof-server             enable pprof server

인기

katana

차세대 크롤링 및 스파이더링 프레임워크

기능

설치

사용법```console