200 OK          The request has succeeded.
403 Forbidden   The server understood the request, but is refusing to fulfill it.
404 Not Found   The server hasn't found anything matching the Request-URI.
302 Found       The requested resource resides temporarily under a different URI.
...

sudo setup.py install

sudo apt-get install python3-pip
sudo pip3 install parsero

sudo apt-get update
sudo apt-get install parsero

$ parsero -h
    
usage: parsero.py [-h] [-u URL] [-o] [-sb]

optional arguments:
-h, --help  show this help message and exit
-u URL      Type the URL which will be analyzed
-o          Show only the "HTTP 200" status code
-sb         Search in Bing indexed Disallows
-f FILE     Scan a list of domains from a list

root@kali:~# parsero -u www.example.com -sb

     ____                               
    |  _ \ __ _ _ __ ___  ___ _ __ ___  
    | |_) / _` | '__/ __|/ _ \ '__/ _ \ 
    |  __/ (_| | |  \__ \  __/ | | (_) |
    |_|   \__,_|_|  |___/\___|_|  \___/ 

Starting Parsero v0.75 (https://github.com/behindthefirewalls/Parsero) at 05/22/14 11:12:55
Parsero scan report for example.com
http://example.com/download.php 302 Moved Temporarily
http://example.com/raw.php 302 Moved Temporarily
http://example.com/embed_js.php 200 OK
http://example.com/embed.php 200 OK
http://example.com/print.php 302 Moved Temporarily
http://example.com/diff.php 302 Moved Temporarily
http://example.com/share.php 404 Not Found
http://example.com/report.php 302 Moved Temporarily
http://example.com/embed_iframe.php 200 OK
                                         
[+] 9 links have been analyzed and 3 of them are available!!!
                                         
Searching the Disallows entries in Bing
                                         
http://www.bing.com/search?q=site:http://example.com/download.php
http://www.bing.com/search?q=site:http://example.com/raw.php
- example.com/raw.php/contact?i=KR9c2erd 200 OK
- example.com/raw.php/legal.aspx 302 Moved Temporarily
- example.com/raw.php/points?i=KR9c2erd 200 OK
- example.com/raw.php/image/sqrn11sp3C/zayn-tshirt-one-direction?i=... 302 Moved Temporarily
http://www.bing.com/search?q=site:http://example.com/embed_js.php
http://www.bing.com/search?q=site:http://example.com/embed.php
http://www.bing.com/search?q=site:http://example.com/print.php
http://www.bing.com/search?q=site:http://example.com/diff.php
http://www.bing.com/search?q=site:http://example.com/share.php
http://www.bing.com/search?q=site:http://example.com/report.php
http://www.bing.com/search?q=site:http://example.com/embed_iframe.php
                                         
Finished in 7.290362596511841 seconds