[EN] Modules in your own webscanner - find dirs and files

Durning the projects often the question is 'what tools we use'.
It would be difficult to 'present' the entire list of tools available in distros such as BackTrack,
but sometimes also hard to believe that we can use 'our own tools'.

Today, the idea taken directly from the popular DirBuster (available here). If you don't know it,
take a few minutes to check it out against your server(s).

Sometimes, when we're doing tests 'from shell' (or from console, you name it), we would like to use 'lighter' tool, than Java-based DirBuster.

Python can be the answer here.

With a few lines of code, we can offer a simple solution. A small program written in Python,
reads 'line by line' filenames and/or dir-names listed in the TXT-list-file and after that it
will present status code (of HTTP response) for each file/dirname.

In the directory where you'll put this python-code, let's create the file with a list of the interesting location(s) on a remote server. Sample list could look like this:

/config.php
/config_inc.php
/config/
/configuration/
/configuration.php
/doc/
/api/
/cache/
/template/
/language/
/media/
/modules/
/plugins/
/install/
/users/
/admincp/
/modcp/
/archive/
/archives/
/sitemap.xml
/ckeditor.php
/FCKeditor/editor/filemanager/browser/default/browser.html
/editor/filemanager/browser/default/browser.html
/fckeditor/editor/css/
/wp-admin/
/wp-content/
/wp-includes/
/index.aspx
/manual/
/server-status
/phpinfo.php
/pi.php
/phpMyAdmin/
/phpmyadmin/
/pma/
/panel/
/login/
/register
/contac


Of course a good choice is to use your own list (but for start you can try 'lists' from DirBuster / fuzzdb project).

Save this list to 'dirsToCheck.txt' file. This is of course a sample list of 'most interesting (us)' locations on a remote test-server. Finding those files/dirs can be significant (from 'webapp-test-point-of-view') because their could be used in the future to abuse, or obtaining information that will be useful durning another steps in pentest.

How do I take advantage of it now?

The program, which is below, does the following:
* the previously prepared TXT-file (with names and locations of directories and files), reads a line by line 'name-location'
* those 'names' (locations) will be used to build a full-URL address to remote hosts (as sys.argv[1])
* full-URL now is checking by HTTP GET (by urllib)
* status code (HTTP response) is the answer from each test (for each 'location')

Code is here:

#!/usr/bin/env python
# ---
# try_dirs.py
# this 'module' will check if there is a file/dir at remote host.
# files/dirs can be edited (you will find it at dirsToCheck.txt file).
# ---
# version : 2 @ 19.03.2013
#

import urllib
import sys

# defines:
url = sys.argv[1]
dirsToCheck = open('dirsToCheck.txt','r')
try_dir = dirsToCheck.readlines()

if len(sys.argv) < 2:
  sys.stderr.write('usage: '+sys.argv[0]+' http://localhost/')
  sys.exit(1)
else:
  print '--------------------------------------------------------------'
  print 'Try enumerate files/dirs at this URL: ',url
  print '--------------------------------------------------------------'

  i=0
  for line in try_dir:
    full_url_to_check = url+line
#    print full_url_to_check
    try_page = urllib.urlopen(full_url_to_check)
    i=i+1

    if try_page.getcode() == 200:
      print 'Found location: ', line
      print 'Status: ', try_page.getcode()
      print '------------------------------------------'
    elif try_page.getcode() == 401:
      print 'Found location: ', line
      print 'Seems to be authorized only: ', try_page.getcode()
      print '------------------------------------------'
    elif try_page.getcode() >= 500:
      print 'Found server-side problem: ', line
      print 'Status: ', try_page.getcode()
      print '------------------------------------------'
    elif try_page.getcode() == 403:
      print 'Found but you have no permissions to access: ', line
      print 'Status: ', try_page.getcode()
      print '------------------------------------------'

Now you can re-edit this code to add for example 404-code (what can be useful durning information gathering steps, because sometimes 404-pages responsing with accurate name and server version).

At this stage, all (the results of the program) can be written by a *nix-based 'redirect to a file' using the '> name.txt'.
Another method is to create a larger 'program' and to establish methods of saving it to 'log-file' (eg using. writelines() to generate a simple report in a more elegant way.

How to run it you will find at code. At console you can use
$chmod u+x check_dirs.py
and next:
$ ./check_dirs.py http://our-server.com (with > filelog.txt if you want)

Sample output will look like this:

$ ./check_dirs-2.py http://www.xxx.xx
--------------------------------------------------------------
Try enumerate files/dirs at this URL:  http://www.xxx.xx
--------------------------------------------------------------
Found location:  /plugins/
Status:  200
------------------------------------------
Found location:  /sitemap.xml
Status:  200
------------------------------------------
Found but you have no permissions to access:  /wp-admin/
Status:  403
------------------------------------------
Found location:  /wp-content/
Status:  200
------------------------------------------
Found but you have no permissions to access:  /server-status
Status:  403
------------------------------------------
(...)

(* full code you will find here.)

Enjoy ;)