CHALLENGES IN SECURITY TESTING OF WEB APPLICATIONS
Authors: Rajendra Gokhale and Susheel Kumar Sharma Aztecsoft-itest
Abstract Security testing for web-based applications is fundamentally different from functional testing in a number of ways. This fact should impact how we test web applications for security. In order to devise an effective methodology for Web Application Security Testing, we must first understand its unique challenges. This paper attempts to identify these challenges in the hope that this information will serve as useful input for developers of security testing tools and test managers for security projects. Introduction A number of developments in the last decade or so have contributed to software security’s moving from the obscurity of Usenet newsgroups into mainstream media, from front-page articles analyzing the worldwide impact of viruses and worms, to Hollywood movies about hackers and the people who bring them to justice. While this has led to commercial and public enterprises allocating substantial funds to secure their network and computing infrastructure, Web Application Security continues to remain an elusive goal. It is estimated that over 80% of security breaches have occurred because of weak protection on external corporate websites. Web Application Security is therefore one of the weakest links in the security chain today. As a result, organizations engaged in software development have been increasing attention to application security issues during the SDLC. Several initiatives propose a secure development methodology that is baked into the software development process itself. For example, Microsoft now requires that its development teams include “Security Threat Model Analysis” in the design and coding phase. The Open Web Application Security Project (“OWASP”), a non-profit organization, regularly maintains a list of “Top 10 Web Application Security vulnerabilities” on its websites that developers can attempt to guard against. Organizations have become sensitized to the need for a dedicated effort designed to test web applications specifically for security issues. These are very positive developments but it is our thesis that these efforts can succeed only when accompanied by improvements in the methodology and tools used in Web Application Security Testing. In this paper, we identify the unique challenges of Web Application Security Testing, and suggest approaches that may be beneficial for security testing teams as well as for developers developing tools for Web Application Security Testing. The Challenges Significantly larger search space
itest.aztecsoft.com
Page 1 of 6
A security tester or a security automation tool has to deal with an exponentially larger test space than a functional tester or functional test automation tool. For a tester it means being able to automate whatever part can be automated, and doing the rest as smartly as possible. For a tool developer it means building an interface that allows an intelligent user to guide it in context-specific tasks, , as well as equipping the application with algorithms to address an NP-complete problem (complete path traversal) in as practically efficient a manner as possible. Speed of traversal is a key selection criterion precisely because there is so much to do: Most vulnerability is high-priority While doing functional testing it is possible to make trade-offs in resources and coverage. As part of the planning phase, the test analyst can narrow the scope of testing by focusing on those parts of the application that are most critical from a business point of view, plus those that are most frequently used. While testing all parts of the application would be ideal, budgetary constraints force trade-offs: the product is released with certain non-critical bugs identified, but not fixed. In security testing, however, testers do not enjoy this luxury. Any trapdoor, however obscure, has the potential for compromising the application. A vulnerability that is present in a rarely used part of the application is just as likely to cause damage as one on the application’s log-in page. Need to test hidden parts of the application A functional tester is primarily concerned with testing what is exposed by an application’s interface. Additionally, he may have to test the application’s backend interfaces. A Performance Tester may need to make sure that the system’s backend can stand up to the Load levels it is likely to be subjected to in deployment. In all these cases, the test target is defined by the application itself. That is not the case in security testing. The security tester must defend against a variety of unspecified attacks: • an SQL injection attack through UI controls (e.g. textboxes, radio-buttons, drop-downs, etc.) • a hidden POST parameter • a GET parameter • a cookie value Need to protect application from willful damage In the functional testing phase, testers must consider whether the application under test will continue to function as expected if the user were to perform normal or at worse random actions; they are not expected to validate or verify the application’s behavior following a user’s modifying the value of a cookie. A security tester, on the other hand, must consider all the ways that a user might willfully damage the application under test. Naturally, this results in a manifold increase in the number of areas that need to be considered during security testing. Standard Software Release Model is not designed for Security Testing In the standard software release model, the code to be tested is accompanied by a set of build release notes, which list the feature changes included in the current release. The functional testing team performs a few Build Verification Tests (BVTs) before repeating the regression test suite from the last test pass. BVTs are not concerned with changes in implementation details when carrying out the regression but only with ensuring that the primary functionality has not been altered, and that code changes have not introduced catastrophic failures. This approach cannot work for the security test team. Consider, for example, a real scenario wherein the application under test maintained the user profile on the server and used the session-id as a key for looking up the user’s data. A developer wanted to save an additional round-trip to the server, so in his next check-in, he stored data on the client side. It worked, but it also introduced a serious vulnerability! Not surprisingly, this change was not recorded in the build release notes since it did not involve any feature change. Had the security
itest.aztecsoft.com
Page 2 of 6
tester not been alert, the vulnerability might have been made it into the released product, to be discovered only when a customer’s private data was compromised through a hostile attack. We propose that in an ideal scenario, implementation details that could impact security should be documented as part of the release notes, including: • the names of form parameters (hidden or otherwise), cookies etc. being used on various pages • the purpose of each of these entities (is this used for a database lookup? Does this get used on other pages?) • a list of entities that have undergone any changes since the last release (e.g. information could get passed using a cookie rather than a form parameter). The problem here is that the software release process has not yet matured enough to the point where these aspects are documented by default. Until this state of affairs changes, security testers will have to continue to find this information themselves. Difficulty in Automating Security Testing It is much harder to write tools that automate the task of testing web application security than for testing application functionality. Some tasks are at first glance not difficult: • ensure that the application rejects potentially malicious characters in the input (e.g. various SQL metacharacters that could be used for carrying out SQL injection) • ensure that the application executes htmlencoding or urlencoding of special input characters that it echoes out. However, a naïve approach can result in false positives and false negatives. For example, it is certainly reasonable for a web application to accept user input consisting of strings such as “%” and “--” if they are never going to be used for constructing dynamic SQL queries. But an automation tool is very likely to flag this as a vulnerability. For an example of a false negative, consider a web-based email application that allows users to compose and read mails online. It would clearly be an error for such an application to display a mail without htmlencoding it. However, an automated tool may never catch this vulnerability since the “Read Mail” page never gets generated as a direct result of cross-site injection that the tool might try on the “Compose Mail” page. Indeed, the ability to minimize the number of false positives and false negatives should be one of the more important criteria in selecting a security test automation tool. We believe that the right approach to building tools for this space would be to marry the computer’s ability to carry out large numbers of repetitive tasks leveraging the intelligence, creative genius and domain knowledge of a security tester. We believe that any tool in this space should allow the security tester to model the application in some manner that can be understood by the tool. The tool can then use this model in order to design its tests. We will illustrate this by means of our email application example. This application can be represented by the following schematic:
itest.aztecsoft.com
Page 3 of 6
If the tool were to enable a tester to create such a model, it would be possible to represent this internally in the form of a directed graph. The tool could then use this graph to infer a correlation between the “Compose Mail” and the “Read Mail” links and design its cross-site scripting attacks accordingly. Another example of this would be a model that allows developers to specify how different inputs will be used. For example, it would be pointless to try out SQL injection attacks on fields that are never used directly or indirectly for accessing a database. By allowing human beings to supply this information, a tool can drastically reduce the overall search space as well as the amount of analytical output that the tester must read and process. In other words, the tool would use the tester’s intelligence to assist in abstracting a logical subset of tasks from the security test space, rather than giving the tester a long list of tasks to select or de-select. Unfortunately, most tools we have seen tend to underplay the importance of the role of the human tester in the security testing activity. Perhaps dictated by the exigencies of marketing, they paint the picture of a wizard-based tool which when pointed at the application under test does a thorough check of the application and produces a beautiful report that will satisfy management. One of the challenges
itest.aztecsoft.com
Page 4 of 6
we need to address is to figure out how to change the customer mindset so that tool vendors are encouraged to add features that seek to enable rather than eliminate the security tester.
Difficulty in finding skilled testers with the right competencies As pointed out above, automation tools can only help, not replace the Security Tester. right set of employees for this job is therefore a major challenge. Finding the
Security Testing is a very complex and hard-to-master skill. It involves being able to understand implementation details, to be able to look under the hood. Apart from this, a security tester must also be endowed with a number of qualities that are hard to find in one person: • Working knowledge of a large number of technologies: He needs to be familiar with a number of web-servers in order to exploit web-server specific vulnerabilities. He needs a good understanding of various RDBMS systems (including their different dialects, their security models, etc.), a good understanding of frameworks such as J2EE and .Net, Operating systems and their Security models, and of course security fundamentals. Understanding of internal implementation details– A security tester needs to be able to understand how an application might have been written before he can try to test it. For example a tester seeking to test for the “broken session management” vulnerability needs a very good understanding of the stateless nature of the HTTP protocol and how it can be made stateful at the application level. He also needs to know how this is achieved using various frameworks such as .NET in order to see if this feature has been implemented securely. Similarly, in order to test for SQL injection, the tester needs to understand interactions between the front-end and the database. Understanding of Web interface internals: The security tester needs to be familiar with the HTTP model including concepts such as GET and POST parameters, hidden fields, cookies, sessions, redirects etc. He needs to understand how to tamper with these invisible entities in order to expose vulnerabilities in the system. Experience with scripting: You can’t undertake comprehensive security testing without knowing how to automate context-specific tasks. For example, a 3rd party security test tool allows the tester to add his own tests in the form of new “policies” written in Javascript. A Security Tester using this tool has to be able to add such policies in order to create tests specific to the application at hand. Similarly, the security tester needs to be able to use command-line tools and write scripts in languages such as Perl to glue together different tools. Currency in changing threats and technologies: There are three dimensions to this problem: o newer technologies bring with them newer vulnerabilities o newer versions of underlying technologies result in new bugs getting introduced o interaction between ever-growing numbers of technologies introduces more and more bugs.
•
•
•
•
The security tester has to constantly update his knowledge-base and tool-chest for these new varieties of security flaws. Some of them may need custom tools and/or innovative approaches to discover. The “bad guys” are evolving, and they are constantly discovering new ways of
itest.aztecsoft.com
Page 5 of 6
compromising web applications. This has been facilitated by the exponential rate of growth of the Internet that has made access to information more "affordable" for crackers. To quote just one statistic, CERT has already issued 2874 advisories in the first half of 2005! A security tester needs to be able to at least keep abreast of the latest techniques used by the attackers to be effective at his profession. Also, it is now possible to access an application through a plain-vanilla web browser, a mobile phone (using SMS or a WAP gateway) and a hand-held device to name but a few interfaces. A good security tester needs a good understanding of all of these so as to be able to judiciously decide what tests need to be run on which platforms.
Conclusion In this paper we have attempted to identify the main challenges in creating the right methodology and set of tools for doing Web Application Security Testing. We believe that a focus on the following will yield significant dividends, justifying substantial investment: • design tools that involve the security tester and the developers in modeling the test application and designing the test strategy • design tools that allow the security tester to logically prune the search space during automated security testing. • Incorporate security-related implementation information into the software release phase • develop interactive training material that will help build up a sufficiently large pool of skilled Security Test professionals. One example of such an effort is OWASP’s tool “Web Scarab”.
References • • • • • is a good description http://msdn.microsoft.com/en-us/library/aa561499.aspx of the “Threat Model Analysis” process used at Microsoft Corporation. The website of the “Open Web Application Security Project” (OWASP) http://www.owasp.org has a lot of useful information related to Web Application Security “Abstracting application-level web security”. David Scot and Richard Sharp Proceedings of the 11th international conference on World Wide Web, 2002. http://www.cert.org has a lot of useful statistics related to Security incidents reported since 1988. Websites of some Application Security Tool vendors: o WatchFire http://www.watchfire.com has a tool called “AppScan” o Cenzic (http://www.cenzic.com) has a tool called “Hailstorm” o SpyDynamics (http://www.spidynamics.com) has a tool called “WebInspect”
itest.aztecsoft.com
Page 6 of 6