Software Assurance of Web-based Applications

Software Assurance of Web-based Applications 2nd Annual OSMA Software Assurance Symposium Wednesday, September 4, 2002 Tim Kurtz SAIC/GRC Risk Management Office Tim.Kurtz@grc.nasa.gov Roadmap Introduction Overview and History of Web-apps Research Plan Initial Results/Proposed Methodologies What’s Next A Look Back 6/8/2009 Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility 2 Introduction Internet, initially used for an information channel, has grown into a commercial channel  Enormous amount of business takes place on the internet  Consumer purchases from online retailers totaled $53B in 2001, non-travel site sales were up 20% from 2000  Averages - $155 million weekday, $97 million weekend day  $321.6 million - Wed., Dec. 12 – highest sales day of the year  Effect of an order entry system that processed orders but forgot to bill customers for a week NASA uses web-based apps to control combustion experiments Effects of failure of a NASA web-app…    6/8/2009 Wouldn’t bankrupt Lost money, resources, science, possible injury BadKurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA publicity Tim IV&V Facility and GSFC 3 Introduction DoD, software industry recognized Software Crisis in the 80’s resulting in    Software development standards Software QA standards Certification processes NASA employs these standards and processes Geared towards large development efforts requiring large resources and months/years to develop Don’t specifically address web-app development 6/8/2009 Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility 4 Overview and History Evolution of the Web Initial web content consisted of static documents containing   Text, pictures and graphics Links to other static pages Used mainly to provide information Today, content includes dynamic pages    Database reports, search results, financial transactions Sound/video files Interactive pages Environmental control Commerce Micro gravity experiment control Data collection Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility Web is used for     6/8/2009 5 Overview and History NASA - Technologies WITS (Web Interface for Telescience) interface can be used by scientists from their home institutions to participate in planetary rover missions by viewing downlink data and generating rover commands. A similar system could be used to command space science instruments or spacecraft. The Goal Performance Evaluation System (GPES) helps automate the process of employee (and organization) performance evaluation/planning. The KSC Electronic Documentation System (KEDS), an engineering drawing viewing/printing software application, was implemented as a state-of-the-art WWW intranet application, providing networked viewing and printing of KSC released engineering drawings from any MS Windowsbased PC WWWorkflow, developed at JPL for the computer mediation of work through an organization, exploits an opportunity created by organization intranets to provide a common user interface across heterogeneous platforms. On-Line Test Procedure, an effective combination of wireless technology, and internet access to electronic test procedure data. 6/8/2009 Ref. http://technology.nasa.gov search for web-based, funded by NASAweb interface Tim Kurtz SAIC/GRC Risk Management OfficeResearch web control OSMA and GSFC IV&V Facility 6 Overview and History NASA – Success Stories The Web Interactive Training (WIT) project. Several WIT-based training courses were developed for the Safety and Mission Assurance Directorate at KSC to efficiently and effectively train a large base of NASA workers using state-of-the-art technologies delivered over the Internet through a Web browser interface Tempest Embedded Web Server  originally developed to support the Manned Space Flight Program for Shuttle and Station experiment remote control.  This technology is currently being used in the Virtual Interactive Classroom( VIC) at NASA Glenn Research Center.  Researchers no longer need to be at the test site in order to collect data. Launchpad to Learning: KSC's Web-Based Engineering Career Education Ref. http://technology.nasa.gov search for web-based, funded by NASAweb interface Tim Kurtz SAIC/GRC Risk Management OfficeResearch web control OSMA and GSFC IV&V Facility 6/8/2009 7 Overview and History NASA – Program Areas An Intelligent Case-based Help Desk: Web-based support for EOSDIS customers 1997 Teacher Tutorials: Teacher training and tools for web-based science, math and technology, etc. A Web-based Distribution of Ionoshperic Thermal Plasma Data from the DMSP Spacecraft Testbed Web-based Tool Development to Involve Non-professionals in Space Science Research Assist in the Development of a new Automated, Web-based Change Tracking System for the Launch Processing System-Configuration Management (LPS-CM) Paper Trail 6/8/2009 Ref. http://technology.nasa.gov search for web-based, funded by NASAweb interface Tim Kurtz SAIC/GRC Risk Management OfficeResearch web control OSMA and GSFC IV&V Facility 8 Research Plan 3 year effort to determine:    How much is NASA using web-apps and how much will they be used in the future? What is NASA doing to assure the quality of the web-apps they are developing and using right now? What should NASA be doing? Surveys, results and resources available on web site Use the tools and techniques on pilot projects Assumptions   Web-apps need to be defined and classified to determine level and type of SA and testing needed Web SA and testing methodologies need to be identified Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility 6/8/2009 9 Research Plan W eb-app P apers S urvey R esults C onferences description/ classification D evelop standard description/ classification S oftw are assurance N asa Industry A cadem ia D eterm ine B est P ractices S oftw are assurance tools P ilot S tudyR esults practices P ilot S tudy M ethods R esources T echnology T ransfer U pdated B est P ractices, G uidebook S A W bA W eb S ite 6/8/2009 Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility http://osat-ext.grc.nasa.gov/rmo/sawba 10 Research Plan Web Site   http://osat-ext.grc.nasa.gov/rmo/sawba What's New - information about the latest happenings at the SAWbA web and research. Schedule - contains research tasks completed last month, in process this month and planned tasks for next month. Events related to the research. Milestones and deliverables and their status. Archives - collection of documents and software developed during the research and links to tools we found useful. Biblio - books, articles and web resources found during the research. FAQ page - frequently asked questions and answers related to web-based applications Surveys/Communities of Practice – post surveys and questionnaires to web site & news groups. Analyze responses. Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility     6/8/2009 11 Research Plan Research Schedule Task Research plan and Web site Survey and investigate current practices Best Practices (draft) Pilot Studies, Report Best Practices (final), Training materials, Guidebook Dates Dec 01 Jun 02 Sep 02 Jun 03 Sep 03 6/8/2009 Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility 12 Research Plan Pilot Projects Micro-gravity Combustion project    Control and conduct gas/fluid combustion experiment Data collection Development begins 2002 CMM level 2 pilot projects 6/8/2009 Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility 13 Research Plan Characteristic Primary Aim Typical Size Timeline Technology Used Characterize Development Modes Traditional Development Build products at minimum cost Medium to large 10 to 100+ eng. 12 – 18 months OOT, CASE tools, generators, C++, etc. Web Development Bring products to market quickly Small 3 to 10 eng 3 – 6 months CBSE, frameworks, java, multi-media, etc NASA Webbased Process CMM-based Ad Hoc, death marches Ref.:’Donald J. Reifer, Web Development: Estimating Quick-to-Market Software, 15th International Forum on COCOMO and Software Estimation Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility 6/8/2009 14 Research Plan Characteristic Products Characterize Development Modes Traditional Development Code-based systems, done in-house, mostly new, many external interfaces, often complex Software engineers. Web Development Object-based systems, multi-media, done outhouse, many reusable parts, few external interfaces, often simple Graphics designers, software engineers, etc. Size: ??? (web objects) Resources: ad hoc or WBS estimate NASA Webbased Development staff Estimating technology Size: SLOC or fp Resources: models or WBS estimate Ref.:’Donald J. Reifer, Web Development: Estimating Quick-to-Market Software, 15th International Forum on COCOMO and Software Estimation Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility 6/8/2009 15 Initial Results QA and Testing SA and testing of static pages consists of    Checking spelling, grammar and anchors (links) Validating code Finding orphaned files Coding standards Automated tools (test scripts) Error detection and prevention Component testing Site testing Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility Dynamic pages require much more effort      6/8/2009 16 Initial Results QA Static and Dynamic QA/Tests Tests Coding standards Web box testing Site testing Regression testing Proof readers Spell and grammar checkers HTML validator ADA validator Configuration audits Checklists 6/8/2009 Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility 17 Initial Results Methodology - Planning Use:  Tailor planning activities to development effort, risks  Correlate SA activities with schedule and milestones  Identify necessary resources/skills SA activity:  Generate Software Assurance plan 6/8/2009 Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility 18 Initial Results Methodology - Coding Standards Use: Implemented for each language used in the project, i.e. HTML, XML, JavaScript, VBScript, etc.   May be separate standards or combined Tailored to each project, environment and requirements. Reduces the opportunity for making errors. Ensure browser compatibility. SA activity: Check code and enforce the standards. 6/8/2009 Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility 19 Initial Results Methodology - Web Box Testing Use: Verify component functionality and integration. Verifies outputs. Establish infrastructure for building, publishing and testing programs and scripts. Set up tool checks for programs and scripts. SA activity: Witness selected tests Check code and enforce coding standards. Inspect output pages for correct results and compliance to coding standards. Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility 6/8/2009 20 Initial Results Methodology - Site Testing Use: Determine if web-app will crash during:   Normal use Abnormal use Map default set of paths through site. Test critical paths’ functionality using default set of paths. Verify creation and display of all static and dynamic pages/dynamic data. Verify back-end applications (servers, databases) are robust SA activity: Verify tests are completed successfully Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility 6/8/2009 21 Initial Results Methodology - Regression Testing Use: Determine if changes have introduced errors. Repeat each previously successful white box, black box and web box test cases which might have been affected by the changes. SA Activity: Witness or verify all affected tests successfully completed Inspect changed code and output pages for correct results and compliance to coding standards. 6/8/2009 Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility 22 Initial Results Methodology – Safety/Security Use:  Identify safety/security issues  Implement controls to reduce/eliminate  Test controls SA Activity:  Review/provide input to safety/security issues  Monitor development and testing of controls 6/8/2009 Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility 23 Initial Results Use: Methodology - Metrics Assist project planning Determine project status SA Activity: Collect, review and analyze metrics 6/8/2009 Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility 24 Initial Results Specification  Methodology – Candidate Metrics User commands Database files Class definitions Object oriented Function points Lines of source code Complexity Coding status Testing status   Design   Program   Progress   6/8/2009 Ref: http://www.mmhq.co.uk/my-complexity/measures-software.shtml Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA IV&V Facility and GSFC 25 What’s Next? We need to answer some questions What is the current and future extent of the use of web based applications in NASA projects? Take the Web-app usage survey – http://osat-ext.grc.nasa.gov/rmo/sawba/UsingSurveyphp.htm  What is NASA currently doing to assure the quality of web based applications? Take the Web-app usage survey – http://osat-ext.grc.nasa.gov/rmo/sawba/AssuranceSurveyphp.htm  6/8/2009 Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility 26 A Look Back Introduction Overview and history Research plan Overview of web application SA and testing activities for static and dynamic web sites Specific types of testing and SA        Planning Coding standards Web box testing Site testing Regression testing Safety/Security Metrics Tim Kurtz SAIC/GRC Risk Management OfficeResearch funded by NASA OSMA and GSFC IV&V Facility Need survey information from NASA/commercial projects 6/8/2009 27