Web Server Assessment Tool by cmlang

VIEWS: 97 PAGES: 24

									Wikto

Web Server Assessment Tool

20 October 2004

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

Document Information
PUBLIC Completed On: Document Number: Issue: Last Modified: Author:

2004-10-20 Wikto_2004-10 1.1 2004-10-20 Roelof Temmingh

Revision History
Document Version 1.0 1.1 Description First Draft Public release Date 2004-08-10 2004-10-20 Author Roelof Temmingh Roelof Temmingh

SensePost Contact Details
Physical Address Middle View Office Building Ground Floor 138 Middle Street Nieuw Muckleneuk 0181 South Africa Contact E-Mail Addresses General: Training: Reseach: HackRack:
info@sensepost.com training@sensepost.com research@sensepost.com info@hackrack.com www.hackrack.com www.sensepost.com

Postal Address P.O Box 10692 Centurion 0046 South Africa

Contact Number +27 12 460 0880

Fax Number +27 12 460 0885

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

2 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

Table of Contents
IMPORTANT NOTES – please read this: ...............................................................................................4 1 1.1 2 3 4 5 5.1 5.2 6 6.1 6.2 6.3 6.4 7 7.1 7.2 7.3 8 Features/bug fixed added in Version 1.51: .............................................................................4 Google Hack section .............................................................................................................. 4 Introduction..................................................................................................................................5 Uses ..............................................................................................................................................6 AI/Fuzzy logic.............................................................................................................................10 Googler .......................................................................................................................................12 Setup ..................................................................................................................................... 12 Starting a run......................................................................................................................... 13 Back-End miner.........................................................................................................................15 Adding/Editing suspect directories....................................................................................... 15 Starting a run......................................................................................................................... 15 During mining ........................................................................................................................ 16 Exporting results ................................................................................................................... 17 Wikto CGI checker ....................................................................................................................18 Setup ..................................................................................................................................... 18 During the scan ..................................................................................................................... 20 Exporting results ................................................................................................................... 23 System wide settings ...............................................................................................................23

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

3 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

IMPORTANT NOTES – please read this:
1. Wikto uses the .NET framework – it will not work if the .NET framework is not loaded. The framework can be obtained by doing a Google search for: “.NET framework” download 2. Wikto is provided without any warrantee etc. 3. Feedback/suggestion can be addressed to research@sensepost.com. 4. The newest version of Wikto is always available at http://www.sensepost.com/research/wikto 5. Wikto is released with full source. If you want to contribute to Wikto please contact us. All help is appreciated.

1
    

Features/bug fixed added in Version 1.51:
Wikto understands the Google Hack DB, and can run all tests automatically Manual testing for Google Hacks enabled Sockets in CLOSE_WAIT is cleaned up using forced Garbage collection Proxy support for Wikto section works BackEnd Miner shows minimum and maximum AI values

1.1

Google Hack section

The Google Hack section is pretty straight forward. Johnny Long told me he will make the GHDB (Google hack database) publicly available for download. With this in mind Wikto now supports the DB, and can do all tests automatically. The setup is rather straight forward:

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

4 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

Simply enter the target site (or domain, or even TLD), load the XML database, and click on “Start GH”. Make sure you have a valid GoogleAPI key set in the SystemConfig section. Results appear in the bottom window, descriptions of the issue in the middle window. If the mouse enters the top window (shaded in the screenshot), the descriptions will update as they are tested. You may also click on the results – if it’s a Google query then its description will appear in the middle window – else it will show the URL (so you can copy and paste it – we’ll get automatic IE surfing in the next release…promise). You can also do a manual test – if you need to check something quickly. Wikto does not add the “site:” functionality to manual queries – so its just a good as a normal Google search. Manual testing can be done at any stage – even when a scan is running (results will be appended to the results screen). It’s not yet clear how the Google Hack Database (GHDB) will be distributed – but you should be able to find a recent one quite easily. Automatic updates for the DB will be provided in upcoming releases. The rest works pretty much as you would expect it to work.

2

Introduction

In the last couple of years the testing of web servers has been largely “standardized”. It has come to the point where testers are reluctant to spend too much time on web servers and rather spend time on web applications. This is largely due to the fact that there have been little or no Version: 1.0 SensePost Research (research@sensepost.com) Page: 5 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

advances in web server testing tools, while new web application testing tools are appearing like mushrooms after a rainy night. This trend is totally understandable – these days very few web servers are exploitable using network application level vulnerabilities. System administrators tend to patch servers, and secure back-end interfaces. Why then spend more time on building yet another web server testing tool? 1. Testing web servers properly has been limited to people who are comfortable in the Unix environment – tools like Nikto and Nessus (the defaco open source tools) are native to the Unix environment. Porting these tools to the Windows world is totally possible (Newt is appearing on the schene at the time of writing this, and using Active State Perl you could port Nikto to Windows), but not wide spread. 2. Very few tools solve the issue of false positives. While assessment tools have been standing still the last couple of years, web server administrators are keeping with the times. Today there are many servers that do not return proper error messages (404s, 200s). Testing these servers with traditional testing tools is difficult and in some cases outright impossible. Later in this document we will see how we address this problem. 3. In order to really understand where developers make mistakes you need to be able to think like a developer. SensePost recently sent 3 engineers on C# training in order to learn how things really sit together in .NET environment. This tool came out as a byproduct of this training. 4. The idea is that this tool is the definitive web server assessment tool – this tool should be all you ever need to perform assessment on web servers (important note – tools could NEVER replace the eye of an experienced security analyst. Also – this tool is not applicable for web application level testing – nor does it try to play in that space at all).

3
  

Uses

The tool that we build is called Wikto. It is apparent that the tool provides the same functionality as the Nikto tool. But it goes a little further. There are 3 main sections of the tool. These are: Back-End miner Nikto-like functionality Googler

This document will discuss these 3 sections and their interactions in detail. But let’s first look at some screenshots:

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

6 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

The Back-End Miner:

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

7 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

Wikto CGI checker:

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

8 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

Googler tool

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

9 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

4

AI/Fuzzy logic

Wikto has a small checkbox in the Wikto and Back-End section named “Use AI”. This checkbox dramatically change the way that Wikto works. If this checkbox is unchecked, Wikto will use standard error codes to determine if a file or directory is present. In the Back-End section of the tool these error codes can be manually set. In the Nikto-like section codes are determined by what ever is contained in the Nikto scan database (usually 200). The problem with inspecting error codes is that many sites do not return clean error codes – these are sites that will (for instance), redirect to a friendly error page. The “error” codes returned for files that do not exist therefore are 200 – making the scanner believe that the file exists. Nikto typically responds like this:

… + Over 30 "OK" messages, this may be a by-product of the + + …
Nessus performs a little bit better dealing with “friendly 404s”, but not a lot. When checking the “use AI” checkbox, Wikto ignores error codes altogether and activates its fuzzy logic / AI modules (call it what ever you want – bottom line is that it doesn’t look at error codes anymore). Wikto creates a signature based on three elements:    File extension (e.g. asp, pl, cgi etc.) Location (e.g. /scripts, /admin) Method (e.g. POST, GET, SEARCH)

server answering all requests with a "200 OK" message. You should manually verify your results.

Before doing any check (be that for a file in the Nikto part, or for a directory in the Back-End miner), it requests a non-existent file with the same extension, method and location. The tool then builds a fingerprint of the request. Next it requests the real file/directory and compares the fingerprint. The function that compares the signatures/fingerprints returns a value of how close the two prints match. The higher the value, the closer they match – the lower the value, the more they differ. Theoretically, an exact match will result in a value of 1 while a complete mismatch results in 0, but in real life values higher than 1 have been seen… The analyst (the user of the tool), can decide on which value to trigger a hit. In the Nikto-like section of the tool the user can do this at any time/in real time – Wikto saves the result of every request. This means the analyst can run the tool and after it has completed modify the trigger levels to see which requests fall within the limit.

When the “Update” button is clicked after modifying the value, Wikto will display the new results. The “Show all” button simply sets the value to a large number (1000), and updates the display. When the “Reset” button is clicked the previous value is restored, and the display is updated. These buttons can be clicked at any time. The Back-End tool (currently) does not keep a history of results, but the trigger level can be set in real time. Version: 1.0 SensePost Research (research@sensepost.com) Page: 10 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

Both the Back-End tool and the Nikto-like tool’s fingerprint DB can be cleared at any time. When engaging another target, these databases should be cleared. Keep in mind that the AI/Fuzzy logic only looks at the similarity in responses. This could sometimes lead to unexpected results. When requesting “/global.asa” on an IIS server, ACLs on the server denies the request. When requesting “/nothere.asa” however the ACL does not kick in and a normal 404 error is returned. For the AI component these are two different responses, and the file is thus marked as present. It is important to understand how the AI/Fuzzy logic works in order to interpret results.

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

11 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

5

Googler

The Googler tool’s primary function is to obtain additional directories for use in the Nikto-type tool and the Back-End miner tool. As an interesting by product it sometimes list files found on web servers that should be there.

5.1

Setup

The first step to use the Googler tool is to get a Google API key. This is a painless process – having a Google API key is a Good Thing ™, and it’s free! Simply visit http://api.google.com and fill in the form.

Once you have a valid Google key enter it in the Google Key field found in the “SystemConfig” tab:

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

12 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

Note – the key that’s hard coded in the application DOES NOT WORK! YOU NEED YOUR OWN KEY! As you can easily exhaust your Google key’s daily limit you might want to increase or decrease the number of pages to download. By default, Google provides results in batches of 10, with a maximum of 100 pages (or 1000 results). The default should be a good start. The Googler tool’s logic works like this: 1. Perform a search for “<Google keyword> filetype: <filetype> site: <Site/Domain>” 2. Find directories in results. 3. Add new directories to “Mined directories” These steps are performed for every file type. By modifying the “Google keyword”, “filetype” and “site/domain” very interesting results can be found. The Googler tool automatically updates the “Google keyword” section as the “site/domain” input field is modified, but this can be changed before the search is performed. Finding interesting information by changing these values is left as an exercise to the reader…

5.2

Starting a run

When everything has been set up (again – the defaults should be a good start), the analyst can click on “Start Googling”. Full URLs will be returned in the main windows. Currently clicking on these will have no result.

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

13 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

Directories (and its parents) are immediately populated in the “mined directories” section. This is a normal text box, and can be edited. The small “cl” button clears the list. Note that subsequent runs will append to the list. These directories can be imported to either the Wikto CGIDIRs list, or to the Back-End Miner’s suspect list. See later sections for more information on this.

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

14 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

6
6.1

Back-End miner
Adding/Editing suspect directories

The Back-End miner section is used to find interesting files and directories on a web server.

The suspect directories, file types and file names can be set manually by simply loading the files into the tool.

By default, these files, directories and file types are appended to the list. The lists can be cleared of all entries by clicking on the small “Cl” button at each list:

If the Googler tool has been run, the resultant directories for the search can be imported into the list by simply clicking on the “Import from Googler” button.

Again, directories are appended to the list.

6.2

Starting a run

To start a run, enter the web site’s DNS name into the IP/DNS name field. If the server is running on a non-standard port, the port can only specified:

The tool automatically updates the HTTP header field to accommodate virtual hosts. The header can be further modified to include Cookies, different Agent types etc. The HTTP header is located in the “SystemConfig” tab:

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

15 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

At this stage the analyst has to decide if she wants to use AI or not (see section on AI/fuzzy logic for more information). If AI is not used the trigger error codes for both directories and files can be set – the defaults should provide a good first iteration:

Next the analyst needs to decide to use GETs or HEADs. Normally HEADs are faster, but some servers could complain about this method and spew unexpected result codes. By default the HEAD method is used. To switch this to the GET method, check the GET/HEAD checkbox:

If using AI/fuzzy logic, and this is the second time the tool is used it could be that the analyst wants to clear the fingerprint database. This can be done by simply clicking the “Clear fingerprint DB” button. At this stage all is set to start the scan, and the analyst can click on the “Start Mining button”:

Directories found are returned in the “Directories (results)” text box. Note that this is a plain text box and can be edited if needed. Files found are returned in the “Files (results)” box. By default the root “/” is populated in the directory results. If the “preserve results” box is checked, the tool will not remove directories and files found. This is useful when the directories are known; the analyst can populate the resultant directories and click on “Skip to files” – see the next section for more details on this functionality.

6.3

During mining

The logic of the Back-End mining tool is as follows: 1. Check all suspect directories to see which exists. 2. Recursively checks within the directories found for the suspect directories (e.g. if it finds /admin it will check if /admin/cgi, /admin/scripts etc. exists). 3. If no more directories are found check each directory for each filename, and file type. It is clear that this process can take quite a while. Imagine 12 directories, 10 file types and 100 filenames – this results in 12 x 10 x 100 requests, or 12000 requests. For the impatient, the Wikto tool provides a way to manually skip parts of the checks. If the “Skip directory” button is clicked while the tool is checking directories it will skip the current directory and start with the next directory in the list. If the tool is busy with file checking it will SensePost Research (research@sensepost.com)

Version:

1.0

Page:

16 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

skip the currently directory and move to the next. If the “Skip to files” button is clicked, the tool immediately skips all directory checks and proceeds directly to checking files.

In combination these two buttons allow the analyst to do an in-depth test, or just a quick glance over. If the “Stop mining” button is clicked the tool will stop after completing the current request. The directory progress bar is zero-ed for each run – this means it will be reset for each directory. The file progress bar attempts to give some indication of how far the file scan has progressed, but has been found to be slightly inaccurate...

6.4

Exporting results

Results can be exported by clicking on the “Export Results” button.

Results are saved in CSV format and include the files and directories found.

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

17 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

7
7.1

Wikto CGI checker
Setup

The Wikto tool tries to provide the same functionality as Nikto.

Wikto uses the same scan database as Nikto version 1.X. This database can be obtained from www.cirt.net (under the Nikto Web Scanner -> Plugins & DB section):

At this stage Wikto does not have an automatic database update function (but watch this space). The scan database can be simply downloaded (scan_database.db) locally. Once the database is downloaded, it can loaded in the Wikto tool, by simply clicking on the “Load DB” button:

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

18 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

Once loaded, the all the checks appear in a list box:

The list box shows two elements per check (tab separated) – the trigger (be that a specific string or error code) and the request. Clicking on any of the test provides more information about the selected check:

NOTE: Wikto tries to identify and ignore XSS test! This is not a bug but by design. After the database is loaded, the target can be configured. Again, if the server runs on a nonstandard port, it can be configured here:

Like with the Back-End miner, the HTTP header is automatically updated, and can be edited manually. Note that if the Wikto tool and the Back-End Miner share the HTTP header field. If these two tools are in use at the same time the target should also be the same. Nikto makes use of a variable called @CGIDIRS (as well as @ADMIN and a few others). Some tests will look for a certain script or CGI within a @CGIDIRS – this basically means that the tool should look through all possible CGI directories for the specific script. The Wikto tool populates this variable in three ways: 1. Using user input (e.g. manual entering of CGI directory names) 2. Importing from the Googler tool 3. Importing from the Back-End miner tool. While some of the directories found in 2. and 3. are clearly not traditional CGI directories (e.g. they are not set to executable), you might find interesting results if you leave them there..

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

19 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

To import CGI directories from the other tools, simply click on the relevant import button:

Next, the analyst needs to decide if AI/Fuzzy logic should be used. See the section on AI/Fuzzy logic for information on this functionality. Wikto can be configured to optimize scans – this basically means that tests for CGIs located in directories that are within the Back-End miner tool suspect list (the list of directories we test for), but that was not found are ignored. The logic here is that if the directory was tested and not found, a CGI in the directory will also not be found.

7.2

During the scan

If the mouse pointer is inside the boundaries of the scan database window (highlighted in blue), Wikto will show the current HTTP request and reply, and highlight the current test in the listbox:

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

20 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

If the mouse is outside of the boundary, the best way to know if the scan is still running is to look at the progress bar, or the single request progress bar (the tiny 3 bar progress in the right hand bottom corner). Results are displayed in the right hand top window. The following elements are listed – Weight, trigger and request. Again, clicking on an item in the result window will display details in the bottom (“Description”) window.

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

21 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

The results window is populated with the following logic:  If using AI/Fuzzy logic (see section on this for more details): o o  If the fingerprint comparison is lower than the value specified on the UI, and the check uses error codes as trigger. If the check does not use error codes (but strings), and the reply contains the trigger string a value of 0.01 is given as fingerprint comparison.

If AI/Fuzzy Logic is switched off: o o If the check uses error codes and the trigger error code is returned a value of 0.01 is retruned If the check uses strings and the trigger string is returned a value is 0.01 is returned

While this could seem confusing at the time of reading it is best explained when actually using the tool. Keep in mind that the fingerprint comparison is stored with every result – this means that the analyst can change the value during the scan, or even when the scan is done.

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

22 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

7.3

Exporting results

Results can be exported at any time. All results in the results window will be exported. Keep in mind that the data in this window is dependant on AI/Fuzzy logic level (if used at all). Clicking on the Export button exports results to CSV file

Exported fields are:      Site name/IP number Request Trigger Type/Family Descriptions

8

System wide settings

These settings affect all tabs/tools within Wikto. Most of the settings are located in the “SystemConfig” tab. With the exception of the Googler tool all other tools is proxy aware. A proxy can be configured by simply specifying the proxy IP/DNS name followed by a colon and the port:

The time that Wikto will wait before registering a time-out as well as the number of retries when a time-out occurs can be configured in the “Timing” section:

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

23 of 24

Wikto - Web Server Assessment Tool Document Number: Wikto_2004-10 Date: 2009.12.08

9
     

Shouts
To Gareth @ SensePost for his initial backend recursion idea (where this all begun) and beta testing. To Samil Shah because he promised me he’ll incorporate HTTPprint (server identification) functionality in Wikto. To Daniel @ deeper for beta testing and feedback. To Johnny Long and everyone that contributed to the GHDB. To the guys at CIRT for building Nikto and maintaining the scan database.

To everyone that ever made me tea.

Version:

1.0

SensePost Research (research@sensepost.com)

Page:

24 of 24


								
To top