The OWASP Testing Framework

Document Sample
The OWASP Testing Framework Powered By Docstoc
					    The OWASP Testing

(Based on the “new OWASP Testing Guide” presentation by Matteo Meucci and Alberto Revelli)

                                  Austin OWASP - 8/28/2007                                   1
Who is Josh Sokol?
 On the Web Systems Team at National Instruments

 UNIX/Linux System Administrator ~10 years

 Cisco Certified Network Associate

 SANS GIAC in Web Application Security (GWAS)

 Working on an initiative to bring a more security
  oriented mindset to the developers at NI.

                     Austin OWASP - 8/28/2007         2
   The OWASP Testing Framework
   The Testing Methodology: How to
   Reporting: How to Evaluate the Risk
    and Write a Report
   Time Pending: Q&A
   Time Pending: Tools and Resources
                 Austin OWASP - 8/28/2007   3
The OWASP Testing Framework
The problem of insecure software: companies next challenge
      “It's impossible to underestimate the importance of having this guide
       available in a completely free and open way”– Jeff Williams (OWASP
Principles of Testing: comparing the state of something against a
  set of criteria defined and complete.
      We want security testing not be a black art
Testing Techniques:
      Manual Inspections & Reviews
      Threat Modeling
      Code Review
      Penetration Testing

                               Austin OWASP - 8/28/2007                        4
The OWASP Testing Framework
Phase 1: Before Development Begins

Before application development has started:

 Test to ensure that there is an adequate SDLC where security is
 Test to ensure that the appropriate policy and standards are in
  place for the development team.
 Develop Measurement and Metrics Criteria (Ensure

                          Austin OWASP - 8/28/2007                  5
The OWASP Testing Framework
Phase 2: During Definition and Design
Before application development has started:
 Security Requirements Review:
       User Management (password reset etc.), Authentication, Authorization, Data Confidentiality, Integrity,
        Accountability, Session Management,Transport Security, Privacy

 Design an Architecture Review
 Create and Review UML Models
       How the application works

 Create and Review Threat Models
       Develop realistic threat scenarios

                                          Austin OWASP - 8/28/2007                                               6
The OWASP Testing Framework
Phase 3: During Development
 Code Walkthroughs:
      high-level walkthrough of the code where the developers can explain the
       logic and flow.

 Code Reviews:
      Static code reviews validate the code against a set of checklists:
           CIA Triad
           OWASP Top10, OWASP Code Review
           Sox, ISO 17799, etc…

                                Austin OWASP - 8/28/2007                         7
The OWASP Testing Framework
Phase 4: During Deployment
   Application Penetration Testing
        Focus of the OWASP Testing Framework Guide
   Configuration Management Testing
        The application penetration test should include the checking of how the
         infrastructure was deployed and secured.

Phase 5: Maintenance and Operations
   Conduct operational management reviews
   Conduct periodic health checks
   Ensure change verification

                                   Austin OWASP - 8/28/2007                        8
Web Application Penetration Testing
   What is Web Application Penetration Testing?
        The process involves an active analysis of the application for any weaknesses, technical flaws or
        Will never be an exact science where a complete list of all possible issues that should be tested can be

   What is a vulnerability?
        A weakness on a asset that makes a threat possible

   OWASP’s approach in writing the guide
        Open
        Collaborative
        Based on a “Black Box” approach

   Defined testing methodology
        Consistent
        Repeatable
        Under quality

                                            Austin OWASP - 8/28/2007                                                9
          Testing Paragraph Template
 Brief Summary
  Describe in "natural language" what we want to test. The target of this section is non-
  technical people (e.g.: client executive)

 Description of the Issue
  Short Description of the Issue: Topic and Explanation

 Black Box testing and example
         How to test for vulnerabilities:
         Result Expected:
 Gray Box testing and example
         How to test for vulnerabilities:
         Result Expected:
 References
         Whitepapers
         Tools

                                       Austin OWASP - 8/28/2007                             10
            Black Box vs. Gray Box
                      The penetration tester does not have any information
Black Box              about the structure of the application, its components
                       and internals

                      The penetration tester has partial information about the
Gray Box               application internals. E.g.: platform vendor, sessionID
                       generation algorithm

White box testing, defined as complete knowledge of the application internals,
is beyond the scope of the Testing Guide and is covered by the OWASP Code
Review Project

                                 Austin OWASP - 8/28/2007                         11
Penetration Testing

      Austin OWASP - 8/28/2007   12
                    Testing Model
The test is divided into 2 phases:

Passive mode: in the passive mode the tester tries to understand
  the application's logic, plays with the application; a tool can be
  used for information gathering such as an HTTP proxy to
  observe all the HTTP requests and responses. At the end of this
  phase the tester should understand all the access points (gates) of
  the application (e.g. Header HTTP, parameters, cookies). A
  spreadsheet with the directory tree of the application and all the
  access points is created for use with the second phase.

Active mode: in this phase the tester begins to test using eight
  distinct sub-phases of security assessment.

                           Austin OWASP - 8/28/2007                13
        Passive Mode: Example

Use an HTTP proxy to observe all the HTTP
requests and responses.

   WebScarab (OWASP)
   TamperData (Firefox Extension)

                   Austin OWASP - 8/28/2007   14
                    Active Mode
 OWASP split the set of tests in 8 sub-categories (for a
  total amount of 48 controls):
      Information Gathering
      Business logic testing
      Authentication Testing
      Session Management Testing
      Data Validation Testing
      Denial of Service Testing
      Web Services Testing
      AJAX Testing

                        Austin OWASP - 8/28/2007            15
             Information Gathering
 The first phase in security assessment is of course focused on collecting all
   the information about a target application.

 Using public tools it is possible to force the application to leak information by
   sending messages that reveal the versions and technologies used by the

 Available techniques include:
       Raw HTTP Connections (netcat)
       The good ol' tools: nmap, amap, ...
       Web Spiders
       Search engines (“Google Dorking”)
       SSL fingerprinting
       File extensions handling
       Backups and unreferenced files

                                Austin OWASP - 8/28/2007                          16
 Information Gathering: Example
 Application Fingerprint
  Knowing the version and type of a running web server allows testers to determine
  known vulnerabilities and the appropriate exploits to use along the tests. Netcat is the
  tool of choice for this very well known technique

    $ nc 80
    HEAD / HTTP/1.0
    HTTP/1.1 200 OK
    Connection: close
    Date: Mon, 27 Aug 2007 22:36:11 GMT
    Server: Microsoft-IIS/6.0
    X-Powered-By: ASP.NET
    X-AspNet-Version: 2.0.50727
    Set-Cookie: ASP.NET_SessionId=atu011455ailys3tuk2hasqh; path=/; HttpOnly
    Set-Cookie: amSessionId=17361177068; path=/
    Cache-Control: no-cache
    Pragma: no-cache
    Expires: -1
    Content-Type: text/html; charset=utf-8
    Content-Length: 9550

                 ...But what if the “Server:” header is obfuscated ?

                                                   Austin OWASP - 8/28/2007             17
Information Gathering: Example
Other hints can be found by sending the server a malformed request, for
instance a “GET / HTTP/3.0”
 HTTP/1.1 400 Bad Request
Date: Sun, 15 Jun 2003 17:12: 37 GMT
Server: obfuscated :P
Connection: close                              Apache 1.3.23
Transfer: chunked
Content-Type: text/HTML; charset=iso-8859-1

 HTTP/1.1 505 HTTP Version Not Supported
Server: obfuscated :P
Date: Mon, 16 Jun 2003 06:04: 04 GMT
Content-length: 140                            Netscape Enterprise 4.1
Content-type: text/HTML
Connection: close

HTTP/1.1 200 OK
Server: obfuscated :P
Date: Fri, 01 Jan 1999 20:14: 02 GMT
Content-Type: text/HTML                                   IIS 5.0
Accept-Ranges: bytes
Last-Modified: Fri, 01 Jan 1999 20:14: 02 GMT
ETag: W/e0d362a4c335be1: ae1                             ...But what if the application simply
Content-Length: 133                                           returns a generic error page ?
                                              Austin OWASP - 8/28/2007                        18
Information Gathering: Example
The good news is that each server has a favorite way to order
headers !
Here are the results for some common web servers when responding
to a “HEAD / HTTP/1.0” command:

 Apache 1.3.23    IIS 5.0            Netscape Enterprise 4.1 SunONE 6.1
 Date             Server             Server                  Server
 Server           Content-Location   Date                    Date
 Last-Modified    Date               Content-Type            Content-Length
 ETag             Content-Type       Last-Modified           Content-Type
 Accept-Ranges    Accept-Ranges      Content-Length          Last-Modified
 Content-Length   Last-Modified      Accept-Ranges
 Connection:      ETag               Connection
 Content-Type     Content-Length

                           Austin OWASP - 8/28/2007                           19
           Business Logic Testing
 In this phase, we look for flaws in the application business logic rather
 than in the technical implementation. Areas of testing include:

 Rules that express the business policy (such as channels, location,
  logistics, prices, and products)
 Workflows that are the ordered tasks of passing documents or data
  from one participant (a person or a software system) to another

    One of the most common results in this step of the analysis are
   flaws in the order of actions that a user has to follow: an attacker
   could perform them in a different order to get some sort of

   This step is the most difficult to perform with automated tools, as it
   requires the penetration tester to perfectly understand the business
       logic that is (or should be) implemented by the application

                             Austin OWASP - 8/28/2007                        20
Business Logic Testing: Example
Summer 2005 Issue of 2600:
In this article he describes a flaw that became apparent to him within a newly released
BlackJack game on the Paradise Poker website. In BlackJack, when the dealer is
showing an ace, the dealer offers the players the option to purchase insurance. This is a
way for the players to pay to cut their losses should the dealer have ten (10, Jack,
Queen, or King) in the hole.

On this particular online game, he noticed that when the dealer did have a pocket ten,
there would be a noticeable pause before he was prompted with the Insurance request.
When there wasn’t a pocket ten, the prompt appeared immediately.

What do you think happened next?

                                   Austin OWASP - 8/28/2007                                 21
Business Logic Testing: Example
After doing some quick calculations, he realized this bit of information gave him an
edge over the house. He ended up playing the next seven hours exploiting this bug and
made a nice chunk of change during that time.

Obviously we don’t know what caused the flaw in the game, but a good guess is that
there was some calculation the system needed to make to determine whether or not to
offer insurance. That calculation may have taken more time to perform in the situation
the dealer had a ten.

Let’s pretend that we’re right and think about that for a sec. The code itself may have
been completely correct in the sense that it did what it was supposed to do. It was the
amount of time the code needed to execute that ended up being the tell. No different
than when a poker player twitches when holding a great hand.

The fix may have been to change the execution profile of the code so that it made the
same pause no matter what was in the hole. Talk about a challenge for game
developers. Not only does the code need to be bug free in syntax and semantics, but
                                 execution profile for
they now need to worry about the Austin OWASP - 8/28/2007 their games.                22
            Authentication Testing
Testing the authentication scheme means understanding how the application
checks for users' identities and using that information to circumvent that
mechanism and access the application without having the proper credentials

Tests include the following areas:
  • Default or Guessable Accounts
  • Brute-force
  • Bypassing Authentication
  • Directory Traversal / File Include
  • Vulnerable “Remember Password” and Password
  • Logout and Browser Cache Management

                              Austin OWASP - 8/28/2007                       23
Authentication Testing: Example

Using Brutus to brute force HTTP Basic Authentication
on a demonstration website.

                    Austin OWASP - 8/28/2007            24
   Session Management Testing
Session management is a critical part of a security test, as every application
has to deal with the fact that HTTP is by it’s nature a stateless protocol.
Session Management broadly covers all controls on a user from
authentication to leaving the application

Tests include the following areas:
   Analysis of the session management scheme
   Cookie and session token manipulation
   Exposed session variables
   Cross Site Request Forgery (CSRF)
   HTTP Exploiting

                             Austin OWASP - 8/28/2007                            25
              Data Validation Testing
  In this phase we test that all input is properly sanitized before being
  processed by the application, in order to avoid several classes of
  attacks. This is the most common web application security weakness.
Cross site scripting (XSS)
         Test that the application filters JavaScript code that might be executed by the
        victim in order to steal his/her cookier

HTTP Methods and XST
         Test that the remote web server does not allow the TRACE HTTP method

SQL Injection
         Test that the application properly filters SQL code embedded in the user input

Other attacks based of faulty input validation...
       LDAP/XML/SMTP/OS injection
       Buffer overflows

                                     Austin OWASP - 8/28/2007                               26
Data Validation Testing: Example
XSS Exploit on

                    Austin OWASP - 8/28/2007   27
            Denial of Service Testing
DoS are types of vulnerabilities within applications that can allow a malicious
user to make certain functionality or sometimes the entire website unavailable.
These problems are caused by bugs in the application, often resulting from
malicious or unexpected user input

   Locking Customer Accounts
   User Specified Object Allocation
   User Input as a Loop Counter
   Writing User Provided Data to Disk
   Failure to Release Resources
   Storing too Much Data in Session

    Usually not performed in performed on production environments

                               Austin OWASP - 8/28/2007                       28
            Denial of Service: Example
At a former employer there was an application that queried a database for a few thousand
rows of data when a user clicked a “submit” button. This request could take several minutes
to process and dramatically increased CPU utilization. The application did not stop the user
from clicking “submit” again while it was processing and each time the user did that it
spawned another process which further drove up CPU utilization. An impatient user could
easily cause the server to lock up and cause a denial of service.

                                       Austin OWASP - 8/28/2007                           29
            Web Services Testing
Webservice “clients” are generally not user web front-ends but
other backend servers. The vulnerabilities in web services are
similar to other vulnerabilities such as SQL injection, information
disclosure and leakage etc but web services also have unique
XML/parser related vulnerabilities.

   XML Structural Testing
   XML Content-Level Testing
   HTTP GET parameters/REST Testing
   Naughty SOAP attachments
   Replay Testing

                          Austin OWASP - 8/28/2007                    30
                Web Services Testing
The vulnerabilities are similar to other “classical” vulnerabilities such as SQL
  injection, information disclosure and leakage etc but web services also have
  unique XML/parser related vulnerabilities.

WebScarab (available for free at provides a plug-in
  specifically targeted to Web Services. It can be used to craft SOAP messages
  that contains malicious elements in order to test how the remote system
  validates input

                                 Austin OWASP - 8/28/2007                           31
Web Services Testing: Example 1
 XML Structural Testing
  In this example, we see a snippet of XML code that violates the hierarchical
   structure of this language. A Web Service must be able to handle this kind of
   exceptions in a secure way
   <?xml version="1.0" encoding="ISO-8859-1"?>
   <note id="666">
   <heading>I am Malformed </to>
   <body>Example of XML Structural Test</body>

                              Austin OWASP - 8/28/2007                         32
Web Services Testing: Example 2
 XML Large payload
  Another possible attack consists in sending to a Web Service a very large
  payload in an XML message. Such a message might deplete the resource of a
  DOM parser
       <Hehehe>I am a Large String (1MB)</Hehehe>
       <Hehehe>I am a Large String (1MB)</Hehehe>
       <Hehehe>I am a Large String (1MB)</Hehehe>…

                            Austin OWASP - 8/28/2007                      33
            Web Services Testing : Example 3
 Naughty SOAP attachments
  Binary files, including executables and document types that can contain
  malware, can be posted using a web service in several ways
 POST /Service/Service.asmx HTTP/1.1
 Host: somehost
 Content-Type: text/xml; charset=utf-8
 Content-Length: length
 SOAPAction: http://somehost/service/UploadFile
 <?xml version="1.0" encoding="utf-8"?>
 <soap:Envelope xmlns:xsi=""
 <UploadFile xmlns="http://somehost/service">

                                  Austin OWASP - 8/28/2007                        34
                                   AJAX Testing
 AJAX (Asynchronous JavaScript and XML) is
    a web development technique used to create
    more interactive web applications.
 XMLHttpRequest object and JavaScript to make
    asynchronous requests for all communication
    with the server-side application.
 Main security issues:
        AJAX applications have a greater attack
         surface because a big share of the
         application logic is moved on the client
        AJAX programmers seldom keep an eye
         on what is executed by the client and what
         is executed by the server
        Exposed internal functions of the
        Client access to third-party resources with
         no built-in security and encoding
        Failure to protect authentication
         information and sessions
                                             Austin OWASP - 8/28/2007   35
        AJAX Bridging
                      AJAX Testing
 While in traditional web applications it is very easy to enumerate
  the points of interaction between clients and servers, when
  testing AJAX pages things get a little bit more complicated, as
  server-side AJAX endpoints are not as easy or consistent to
 To enumerate endpoints, two approaches must be combined:
      Look through HTML and Javascript (e.g: look for XmlHttpRequest
      Use a proxy to monitor traffic
      Tools: OWASP Sprajax or Firebug add-on for Firefox
 Then you can test it as described before (SQL Inj, etc..)
 ...and don't forget AJAX potential in prototype hijacking and
  resident XSS !
                            Austin OWASP - 8/28/2007                    36
  AJAX Testing

 With firebug it is possible
to efficiently inspect AJAX
           Austin OWASP - 8/28/2007   37
         AJAX Testing: Example
The Samy and Spaceflash worms both spread on MySpace,
changing profiles on the hugely popular social-networking Web site.
In Samy attack, the XSS Exploit allowed <SCRIPT> in profile. AJAX was used to inject a virus into the
MySpace profile of any user viewing an infected page and forced
any user viewing the infected page to add the user “Samy” to his
friend list. It also appended the words “Samy is my hero” to the
victim’s profile.

                         Austin OWASP - 8/28/2007                38
Penetration Testing

      Austin OWASP - 8/28/2007   39
              Testing Report: Model
 The OWASP Risk Rating Methodology
      Estimate the severity of all of these risks to your business
      This is not universal risk rating system: vulnerability that is critical to one
       organization may not be very important to another

 Simple approach to be tailored for every case
      standard risk model: Risk = Likelihood * Impact

Step 1: identifying a risk
   You'll need to gather information about:
      the vulnerability involved
      the threat agent involved
      the attack they're using
      the impact of a successful exploit on your business.
                                   Austin OWASP - 8/28/2007                              40
          Testing Report: Likelihood
Step 2: factors for estimating likelihood
   Generally, identifying whether the likelihood is low, medium, or high is sufficient.

 Threat Agent Factors:
    Skill level (0-9)

    Motive (0-9)

    Opportunity (0-9)

    Size (0-9)

 Vulnerability Factors:
    Ease of discovery (0-9)

    Ease of exploit (0-9)

    Awareness (0-9)

    Intrusion detection (0-9)    Austin OWASP - 8/28/2007                          41
            Testing Report: Impact
Step 3: factors for estimating impact

Technical impact:
      Loss of confidentiality (0-9)
      Loss of integrity (0-9)
      Loss of availability (0-9)
      Loss of accountability (0-9)

 Business impact:
      Financial damage (0-9)
      Reputation damage (0-9)
      Non-compliance (0-9)
      Privacy violation (0-9) Austin OWASP - 8/28/2007   42
   Testing Report: Value the Risk
Step 4: determining the severity of the risk

 In the example above, the likelihood is MEDIUM, and the technical impact is HIGH, so
  from a technical standpoint, the overall severity is HIGH. But business impact is
  actually LOW, so the overall severity is best described as LOW as well.
                                 Austin OWASP - 8/28/2007                           43
Testing Report: Decide What to Fix
 Step 5: Deciding What To Fix
    As a general rule, you should fix the most severe risks first.
    Some fix seems to be not justifiable based upon the cost of
    fixing the issue but may be reputation damage from the fraud
    that could cost the organization much more than implement a
    security control

 Step 6: Customizing Your Risk Rating Model
     Adding factors

     Customizing options

     Weighting factors

                           Austin OWASP - 8/28/2007                  44
               Writing the Report
 I. Executive Summary
 II. Technical Management Overview
 III Assessment Findings
 IV Toolbox

                         Austin OWASP - 8/28/2007   45
      OWASP Penetration Testing
         Guide: Summary
                       A structured approach to the testing activities
                       A checklist to be followed
                       A learning and training tool

                       A tool to understand web vulnerabilities and their impact
                       A way to check the quality of the penetration tests they

 More in general, the Guide aims to provide a pen-testing standard that creates
     a 'common ground' between the pen-testing industry and it’s clients.
 This will raise the overall quality and understanding of this kind of activity and
           therefore the general level of security in our infrastructures.

                                  Austin OWASP - 8/28/2007                            46

  Austin OWASP - 8/28/2007   47
Tools and Resources
 A list of tools which are free and/or
             Open Source.

             Austin OWASP - 8/28/2007    48
             OWASP Resources
   OWASP AppSec FAQ Project FAQ covering many
    application security topics
   OWASP Guide Project a massive document covering all
    aspects of web application and web service security
   OWASP Legal Project a project focused on contracting
    for secure software
   OWASP Testing Guide a project focused on application
    security testing procedures and checklists
   OWASP Top Ten Project an awareness document that
    describes the top ten web application security

                   Austin OWASP - 8/28/2007                49
    Black Box Testing Tools
   OWASP WebScarab –
   OWASP CAL9000 –
   OWASP Pantera –
   SPIKE –
   Paros –
   Burp Proxy –
   Achilles Proxy –
   Odysseus Proxy –
   Webstretch Proxy –
   Firefox LiveHTTPHeaders, Tamper Data and Developer Tools –
   Sensepost Wikto (Google cached fault-finding) -

                      Austin OWASP - 8/28/2007                     50
           Testing Ajax

              Austin OWASP - 8/28/2007   51
Testing for SQL Injection
   Multiple DBMS Sql Injection tool – [SQL Power Injector]
   MySql Blind Injection Bruteforcing, – [sqlbftools]
   Antonio Parata: Dump Files by sql inference on Mysql –
   Sqlninja: a SQL Server Injection & Takeover Tool –
   Bernardo Damele and Daniele Bellucci: sqlmap, a blind SQL
    injection tool –
   Absinthe 1.1 (formerly SQLSqueal) –
   SQLInjector –
   Bsqlbf-1.2-th –

                      Austin OWASP - 8/28/2007                         52
         Testing Oracle
   TNS Listener tool (Perl) -
   Toad for Oracle -

             Austin OWASP - 8/28/2007   53
           Testing SSL
   Foundstone SSL Digger -

              Austin OWASP - 8/28/2007   54
Testing for Brute Force Password
   SensePost Crowbar –
   THC Hydra –
   John the Ripper -
   Brutus –
   Medusa -

                  Austin OWASP - 8/28/2007            55
Testing for HTTP Methods
   NetCat -

              Austin OWASP - 8/28/2007   56
    Testing Buffer Overflow
   OllyDbg: "A windows based debugger used for analyzing
    buffer overflow vulnerabilities" -
   Spike, A fuzzer framework that can be used to explore
    vulnerabilities and perform length testing -
   Brute Force Binary Tester (BFB), A proactive binary
    checker -
   Metasploit, A rapid exploit development and Testing
    frame work -

                    Austin OWASP - 8/28/2007               57
   OWASP WSFuzzer -

              Austin OWASP - 8/28/2007   58
   Foundstone Sitedigger (Google cached
    fault-finding) -

              Austin OWASP - 8/28/2007   59
     Source Code Analyzers
   PMD –
   FlawFinder –
   Microsoft’s FXCop –
   Split –
   Boon –
   Pscan –
   FindBugs -

                  Austin OWASP - 8/28/2007             60
          Acceptance Testing
   WATIR – - A Ruby based web testing framework that
    provides an interface into Internet Explorer. Windows only.
   HtmlUnit – - A Java and JUnit based framework that
    uses the Apache HttpClient as the transport. Very robust and configurable and is used
    as the engine for a number of other testing tools.
   jWebUnit – - A Java based meta-framework that uses
    htmlunit or selenium as the testing engine.
   Canoo Webtest – - An XML based testing tool that
    provides a facade on top of htmlunit. No coding is necessary as the tests are
    completely specified in XML. There is the option of scripting some elements in
    Groovy if XML does not suffice. Very actively maintained.
   HttpUnit – - One of the first web testing frameworks,
    suffers from using the native JDK provided HTTP transport, which can be a bit
    limiting for security testing.
   Watij – - A Java implementation of WATIR. Windows only because
    it uses IE for it's tests (Mozilla integration is in the works).
   Solex – - An Eclipse plugin that provides a graphical tool
    to record HTTP sessions and make assertions based on the results.
   Selenium - - JavaScript based testing framework,
    cross-platform and provides a GUI for creating tests. Mature and popular tool, but the
    use of JavaScript could hamper certain security tests.

                            Austin OWASP - 8/28/2007                                     61
           Site Mirroring
   wget –
   curl –
   Sam Spade –
   Xenu -

                Austin OWASP - 8/28/2007        62

Shared By: