SpiderFoot v2.6.1 - Open Source Intelligence Automation
SpiderFoot is an open source
intelligence automation tool. Its goal is to automate the process of
gathering intelligence about a given target.
Purpose
There are three main areas where SpiderFoot can be useful:
- If you are a pen-tester, SpiderFoot will automate the reconnaisance stage of the test, giving you a rich set of data to help you pin-point areas of focus for the test.
- Understand
what your network/organisation is openly exposing to the outside world.
Such information in the wrong hands could be a significant risk.
- SpiderFoot can also be used to gather threat intelligence about suspected malicious IPs you might be seeing in your logs or have obtained via threat intelligence data feeds.
Features
- Utilises a shedload of data sources; over 40 so far and counting, including SHODAN, RIPE, Whois, PasteBin, Google, SANS and more.
- Designed for maximum data extraction;
every piece of data is passed on to modules that may be interested, so
that they can extract valuable information. No piece of discovered data
is saved from analysis.
- Runs on Linux and Windows. And fully open-source so you can fork it on GitHub and do whatever you want with it.
- Visualisations. Built-in JavaScript-based visualisations or export to GEXF/CSV for use in other tools, like Gephi for instance.
- Web-based UI. No cumbersome CLI or Java to mess with. Easy to use, easy to navigate. Take a look through the gallery for screenshots.
- Highly configurable. Almost every module is configurable so you can define the level of intrusiveness and functionality.
- Modular.
Each major piece of functionality is a module, written in Python. Feel
free to write your own and submit them to be incorporated!
- SQLite back-end. All scan results are stored in a local SQLite database, so you can play with your data to your heart’s content.
- Simultaneous scans. Each footprint scan runs as its own thread, so you can perform footprinting of many different targets simultaneously.
- So much more.. check out the documentation for more information.
Data Sources
This
is an ever-growing list of data sources SpiderFoot uses to gather
intelligence about your target. A few require API keys but they are
freely available.
Source | Location | Notes |
---|---|---|
abuse.ch | http://www.abuse.ch | Various malware trackers. |
AdBlock | https://easylist-downloads.adblockplus.org/easylist.txt | AdBlock pattern matches |
AlienVault | https://reputation.alienvault.com | AlienVault’s IP reputation database. |
Autoshun.org | http://www.autoshun.org | Blacklists. |
AVG Site Safety Report | http://www.avgthreatlabas.com | Site safety checker. |
Bing | http://www.bing.com | Scraping but future version to also use API. |
Blocklist.de | http://lists.blocklist.de | Blacklists. |
Checkusernames.com | http://www.checkusernames.com | Look up username availability on popular sites. |
DNS | Your configured DNS server. | Defaults to your local DNS but can be configured to whatever IP address you supply SpiderFoot. |
DomainTools | http://www.domaintools.com | |
DroneBL | http://www.dronebl.org | |
DuckDuckGo | http://www.duckduckgo.com | |
http://www.facebook.com | Scraping but future version to also use API. | |
FreeGeoIP | http://freegeoip.net | |
Github | http://www.github.com | |
http://www.google.com | Scraping but future version to also use API. | |
Google+ | http://plus.google.com | Scraping but future version to also use API. |
Google Safe Browsing | http://www.google.com/safebrowsing | Site safety checker. |
IPCat | https://raw.githubusercontent.com/client9/ipcat/master/datacenters.csv | IP Categorisation. |
http://www.linkedin.com | Scraping but future version to also use API. | |
malc0de.com | http://malc0de.com | Blacklists. |
malwaredomainlist.com | http://www.malwaredomainlist.com | Blacklists. |
malwaredomains.com | http://www.malwaredomains.com | Blacklists. |
McAfee SiteAdvisor | http://www.siteadvisor.com | Site safety checker. |
NameDroppers | http://www.namedroppers.org | |
Notepad.cc | http://www.notepad.cc | |
Nothink.org | http://www.nothink.org | Blacklists. |
Onion.City | http://onion.city | Search engine for the dark web. |
OpenBL | http://www.openbl.org | Blacklists. |
PasteBin | http://www.pastebin.com | Achieved through Google scraping. |
Pastie | http://www.pastie.org | |
PGP Servers | http://pgp.mit.edu/pks/ | PGP public keys. |
PhishTank | http://www.phishtank.org | Identified phishing sites. |
Project Honeypot | http://www.projecthoneypot.org | Blacklists. API key needed. |
PunkSPIDER | http://www.punkspider.org | |
RIPE/ARIN | http://stat.ripe.net/ | |
Robtex | http://www.robtex.com | |
SANS ISC | http://isc.sans.edu | Internet Storm Center IP reputation database. |
SHODAN | http://www.shodanhq.com | API key needed. |
SORBS | http://www.sorbs.net | Blacklists. |
SpamHaus | http://www.spamhaus.org | Blacklists. |
ThreatExpert | http://www.threatexpert.com | Blacklists. |
TOR Node List | http://torstatus.blutmagie.de | |
TotalHash.com | http://www.totalhash.com | Domains/IPs used by malware. |
UCEPROTECT | http://www.uceprotect.net | Blacklists. |
VirusTotal | http://www.virustotal.com | Domains/IPs used by malware. API key needed. |
WayBack Machine | http://www.archive.org | |
Whois | Various | Whois servers for different TLDs. |
XSSposed | http://www.xssposed.org | |
Yahoo | http://www.yahoo.com | Scraping but future version to also use API. |
Zone-H | http://www.zone-h.org | Easy to get black-listed. Log onto the site in a browser from the IP you’re scanning from first and enter the CAPTCHA, then it should be fine. |
SpiderFoot v2.6.1 - Open Source Intelligence Automation
Reviewed by Zion3R
on
6:35 PM
Rating: