Home / Chrome DevTools / Crawler / Devtool / Extractor / Galer / golang / Spider / Url Extractor / Url Parser / Waybackurls / Galer - A Fast Tool To Fetch URLs From HTML Attributes By Crawl-In

Galer - A Fast Tool To Fetch URLs From HTML Attributes By Crawl-In

| Post sponsored by FaradaySEC | Multiuser Pentest Environment


A fast tool to fetch URLs from HTML attributes by crawl-in. Inspired by the @omespino Tweet, which is possible to extract src, href, url and action values by evaluating JavaScript through Chrome DevTools Protocol.


Installation

from Binary

The installation is easy. You can download a prebuilt binary from releases page, unpack and run! or with

             __
   __    _ _(_ )   __  _ __ 
 /'_ '\/'_' )| | /'__'( '__)
( (_) ( (_| || |(  ___| |
'\__  '\__,_(___'\____(_)
( )_) |
 \___/'  @dwisiswant0

from Source

If you have go1.15+ compiler installed and configured:

▶ (sudo) curl -sSfL https://git.io/galer | sh -s -- -b /usr/local/bin

from GitHub
▶ GO111MODULE=on go get github.com/dwisiswant0/galer

Usage

Basic Usage

Simply, galer can be run with:

▶ git clone https://github.com/dwisiswant0/galer
▶ cd galer
▶ go build .
▶ (sudo) mv galer /usr/local/bin

Flags
▶ galer -u "http://domain.tld"


This will display help for the tool. Here are all the switches it supports.

Flag Description
-u, --url Target to fetches (single target URL or list)
-e, --extension Show only certain extensions (comma-separated, e.g. js,php)
-c, --concurrency Concurrency level (default: 50)
--in-scope Show in-scope URLs/same host only
-o, --output Save fetched URLs output into file
-t, --timeout Maximum time (seconds) allowed for connection (default: 60)
-s, --silent Silent mode (suppress an errors)
-v, --verbose Verbose mode show error details unless you weren't use silent
-h, --help Display its helps

Examples

Single URL
▶ galer -h

URLs from list
▶ galer -u "http://domain.tld"

from Stdin
▶ galer -u /path/to/urls.txt

In case you want to chained with other tools:

▶ cat urls.txt | galer

You can use galer as library.

▶ subfinder -d domain.tld -silent | httpx -silent | galer

For example:

▶ go get github.com/dwisiswant0/galer/pkg/galer

TODOs
  • Enable to set extra HTTP headers
  • Provide randomly User-Agent
  • Bypass headless browser
  • Add exception for specific extensions

Help & Bugs

If you are still confused or found a bug, please open the issue. All bug reports are appreciated, some features have not been tested yet due to lack of free time.


License

galer released under MIT. See LICENSE for more details.


Version

Current version is 0.0.2 and still development.


Pronunciation

id_ID • /gäˈlər/ — kalau galer jangan dicium baunya, langsung cuci tangan, bego!


Acknowledgement
  • Omar Espino for the idea, that's why this tool was made!


Galer - A Fast Tool To Fetch URLs From HTML Attributes By Crawl-In Galer - A Fast Tool To Fetch URLs From HTML Attributes By Crawl-In Reviewed by Zion3R on 8:30 AM Rating: 5