software_testing 
The Groundwork Books Project Software Testing Yekaterina Tsipenyuk, Juliana Wong, Joshua Tran, Chun-Chung Lui, and David Shuffelton http://www-cse.ucsd.edu/~ytsipeny/home/cs210/project.html Introduction The project has progressed through requirements analysis, design, and development and is now nearing completion. Development is still ongoing, but a framework for testing is in place. The structure and emphasis of this testing framework was driven by the risks still present in the project. The test plan is devised in three general parts, two of which are informal and one is specified formally. The first part of the test plan is unit testing of functional units immediately after module completion and peer review of code between members of the project team. This limited white-box testing can catch many errors before they propagate. No formal results of this testing exist as the process has been ongoing for the life of the development cycle. The second part of the test plan is specified formally as functional tests of the various parts of the system. Limited results of this testing are available and attached. Finally, the project team has enlisted the aid of several outsiders to help with general usability testing. The results of this testing are somewhat more freefoorm examples are attached. Risks and Resolution Generally speaking, the most important risk at this point in the project is that some part of the product will not function correctly upon installation. A somewhat less important risk is that the product will not be easy enough to use to convince both customers and the Groundwork staff to use it. The first risk is rather broad and can be broken down into several smaller risks, some of which are more important than others. The most important risk in this respect is that the product will do the wrong thing when presented with correct input. This would be a catastrophic failure and would probably inhibit anyone from actually using the system. A less important risk is that the product will fail in some unfriendly way when presented with incorrect input (for instance, by someone typing HTML tags into an input field which are not noticed by the code). Although irritating, this can generally be resolved by hitting the "back" button on the browser and trying again, so the project team has deemed this less of a risk. With a view toward these two risks, a test plan has been formulated that addresses each of the risks in turn. For each piece of functionality in the system, a description is given of what the module should do when given correct input and when given incorrect input. A specification for what correct input is is also given for each input field. It is then possible to test with respect to each of the risks above; first, testing correct inputs to verify that the correct action is taken, and then with incorrect inputs to verify that a suitable error message is shown or the correct error action is taken. By allowing the testing to be focused on either correct or incorrect input, the risks can be addressed in order of importance. To address the usability risk, the project team has arranged for several people to attempt to use the (still somewhat unfinished) web site as a normal customer would. A member of the project team observes how the user interacts with the site and records any comments they have about the experience. Since this testing is user-driven, there is no test script to specify exactly what steps are to be taken. Instead, the user simply tries to perform a task and comments on how easy or hard it was. The results can then be analyzed by the project team and possibly used to make minor adjustments to the product, helping to alleviate the usability risk. Finally, a further risk exists whose importance is difficult to gauge. Development has been done on one machine, but the system will actually be deployed on a different machine. The software used (Linux, Apache, MySQL, PHP, and sendmail) are all available on the deployment machine, but the environment is slightly different. This risk will be addressed by performing a second round of functional testing once the product is installed at the deployment site. If there are any anomalies, they should be easy to detect by comparing the results of the first round of functional testing to the second. Steps can then be taken to correct (or work around) any difficulties presented by the different environment. Test Plan Following is the functional test plan, roughly in the order that a user would encounter these functionalities upon using the system. Afterwards a short example of a testing session is given for the first few functions. Login link Should appear on the menu only if the user is not logged in. When the link is clicked, the user is taken to the login.php page. Logout link Should appear only if the user is logged in. When the link is clicked, all session variables are unset, the session is destroyed, and the user is shown a "Successful logout" message. Registration Inputs Required? Format required username yes an email password yes 6 -10 characters confirm password yes same as previously entered password name yes user's name; no constraint work phone yes must be digits; 10 digits total other phone no must be digits; 10 digits total When the submit button is clicked, if all the required fields are entered as specified in the above table, the user is taken to the welcome.php page. If any of the required fields are not entered, a corresponding error message is displayed. If any input is not of the required format, a corresponding message is displayed. Upon successful registration, the user becomes a registered user of the system with "regular user" privilege. Login Inputs Required? Format required username yes none password yes none If the username and password pair entered is correct, the user is taken to the welcome.php page. Otherwise, a "Login failed" error message is displayed. Forgot your password Inputs Required? Format required username yes an email When the submit button is pressed, it is checked that the email exists in the database and that the account has an active status. If either is not true, an "Unknown user" error message is displayed. Otherwise, and e-mail containing the password is sent to the user, and the user is automatically redirected to the login.php page. If there is a problem connecting to the database, a corresponding error message is displayed. If there is a problem sending an email, a corresponding error message is displayed. Placing Special Order Inputs Required? Format required home phone no must be digits; 10 digits total other phone no must be digits; 10 digits total author yes none title yes none publisher no none isbn no none quantity yes can only be one or two If a user attempts to directly access this page without logging in, the user is automatically redirect to the login page. When the submit button is clicked, if all the required fields are entered, the user is taken to a confirmSOrder.php page where their order information is redisplayed for confirmation. If any required fields is not provided, an error message is displayed. If any input is not in its required format, a corresponding error message is displayed. At least one title is required. Special Order Confirmation If a user attempts to directly access this page without logging in, the user is automatically redirect to the login page. There is no input on this page. It merely serves as a confirmation for the user before their order is placed and saved in the database. If the user clicks the submit button, the order will be saved and the user will be taken back to the welcome.php page. If the user clicks cancel, the order will not be saved in the database and the user will be taken back to the welcome.php page. If the user wishes to make changes to the order, s/he can use the browser's back button to go back to the special order form. Order Book for a Course Link Should appear on the page only if the user is logged in as a professor. When the link is clicked, the user is taken to the Get Information page. Get Information Has two text fields: required book number and suggested book number. For required book number, the input must be positve integer. For suggested book number, the input can be non-negative integer or blank. Inputs Required? Format required number of required yes positive integer number of suggested no non-negative integer When the next button is clicked, the user is taken to Professor Order. Professor Order Inputs Required? Format required campus phone yes area-pre-post phone number home phone no area-pre-post phone number quarter yes Fall, Winter, Spring, Summer year yes 4 char string all numbers department yes 15 char string course yes 10 char string, (number)*(letter)? course listed as yes 40 char string est enrollment yes positive integer for required each book: author yes 100 char string title yes 200 char string publisher no 100 char string isbn no 30 char string type yes TRD, PAP for suggested each book: author yes 100 char string title yes 200 char string publisher no 100 char string isbn no 30 char string quantity yes positive interger type yes TRD, PAP When the submit button is pressed, it will pass all data to and bring up Confirm Professor Order. Confirm Professor Order Check if all required fields is filled (no type checking yet), and if there is an order already been placed for a class in the database that has the same quarter, year, department and course number (temp turned off for testing purpose). no type checking is done at this point. Echo the phone number(s) as the user input them in pOrder.php, echo back all books info in tables. Professor View Past Order List ALL the orders this user has placed in the past with Term (quarter + year), course number (department + course_num), course name, and ordered date, followed by a link "View". Ideally, professor can view their class order and special order in different tables. When the user click on the "View" link after each order, they will be taken to Display Order. Display Order The contact info and class info is echo nicely on the screen (don't mind the text box for "other phone", i will take it away by this week) All books in this order is listed at the bottom. Process Order There are two types of forms on this page, one is a per title form to allow administrator to process each title separately and the other is a per order form to allow administrator to process the entire order. per title form: Inputs Required? Format required new in GW yes must be a number new need to ord yes must be a number new avail no must be "Yes", "No", or "Never" if "No", a date is required new p.o. number no none new ord no must be a number new date no must be in a date format (YYYY-MM-DD) new re-ord no must be a number new re-ord date no must be in a date format (YYYY-MM-DD) used In GW yes must be a number used need to ord yes must be a number used avail no must be "Yes", "No", or "Never" if "No", a date is required used p.o. number no none used ord no must be a number used date no must be in a date format (YYYY-MM-DD) If a user attempts to directly access this page without logging in or is not logged in with "administrator" privilege, the user is automatically redirect to the login page. The administrator can either "process" or "cancel" each title. If the administrator chooses to "process" a title, all the required fields are checked and all inputs are validated. If an error occurs, a corresponding message is displayed. Upon successful checking, the title is updated in the database. If the administrator chooses to "cancel" a title, the title's status is updated in the database and s/he is taken back to the "process order form". per order form: Inputs Required? Format required Limit yes must be a number pre-enrollment no must be a number 1st week no must be a number 3rd week no must be a number If a user attempts to directly access this page without logging in or is not logged in with "administrator" privilege, the user is automatically redirect to the login page. The administrator can either "process" or "cancel" each order. If the administrator chooses to "process" an order, all the required fields are checked and all inputs are validated. In order to "process" an order, all titles in the order must also be processed or cancelled. In other words, no titles can be "unprocessed". If the administrator is successful in processing the order, the order's status is updated in the database and s/he is taken back to the "list of orders" page. If the administrator chooses to cancel an order, the order's status is updated in the database and s/he is taken back to the "list of orders" page. Administrator Viewing All Orders This page can only be viewed by an administrator. If a non-administrator tries to access the page, an error message is displayed that says that only administrators can view the page. The page outputs two tables: one for new orders and another for processed orders. Each table has four columns: order number, course number, instructor name, and action. If course number is an empty string (this order is a special order), "N/A" is displayed instead. If any of the fields contain HTML tags, they need to be displayed correctly on the page without destroying the HTML format of the page. When the "View" link is clicked, the administrator is taken to displayOrder.php page, where s/he can process an order. After the order is processed and when the administrator returns to the orders.php page, the corresponding order needs to be displayed in the "Processed orders" table rather than the "New orders" table as before. Odd rows of the table need to be colored with light gray, and even --with dark gray. If one of the tables does not have any entries, then only the header of the table is displayed. If there is a problem connecting to the database, a corresponding error message is displayed. Administrator Upgrading/Downgrading Regular User Account to Professor Status and Vice Versa This page can only be viewed by an administrator. If a non-administrator tries to access the page, an error message is displayed that says that only administrators can view the page. The page outputs two tables: one for professor accounts and another for regular user accounts. Only active accounts are displayed. Each table has four columns: user number, user name, email, and action. If any of the fields contain HTML tags, they need to be displayed correctly on the page without destroying the HTML format of the page. When "Downgrade" link is clicked, the corresponding professor account becomes a regular user account, disappears from the "Professor accounts" table, and gets moved to the "Regular user accounts" table. When "Upgrade" link is clicked, the corresponding regular user account becomes a professor account, disappears from the "Regular user accounts" table, and gets moved to the "Professor accounts" table. Odd rows of the table need to be colored with light grey, and even --with dark grey. If one of the tables does not have any entries, then only the header of the table is displayed. If there is a problem connecting to the database, a corresponding error message is displayed. Administrator Downgrading/Upgrading Administrator Account to Regular User Status and Vice Versa This page can only be viewed by an administrator. If a non-administrator tries to access the page, an error message is displayed that says that only administrators can view the page. The page outputs two tables: one for administrator accounts and another for nonadminiistrato accounts. Only active accounts are displayed. Also, only administrators who are not logged in this session are displayed, which means that at least one administrator will remain in case all others are downgraded. Each table has four columns: user number, user name, email, and action. If any of the fields contain HTML tags, they need to be displayed correctly on the page without destroying the HTML format of the page. When "Downgrade" link is clicked, the corresponding administrator account becomes a regular user account, disappears from the "Administrator accounts" table, and gets moved to the "Non-administrator accounts" table. When "Upgrade" link is clicked, the corresponding non-administrator account becomes an administrator account, disappears from the "Non-administrator accounts" table, and gets moved to the "Administrator accounts" table. Odd rows of the table need to be colored with light gray, and even --with dark gray. If one of the tables does not have any entries, then only the header of the table is displayed. If there is a problem connecting to the database, a corresponding error message is displayed. Administrator Activating/Deactivating Accounts This page can only be viewed by an administrator. If a non-administrator tries to access the page, an error message is displayed that says that only administrators can view the page. The page outputs two tables: one for active accounts and another for inactive accounts. Only accounts that are not used for the current session can be deactivated, which guarantees that at least one administrator will remain in case all others are deactivated. Each table has four columns: user number, user name, email, and action. If any of the fields contain HTML tags, they need to be displayed correctly on the page without destroying the HTML format of the page. When "Deactivate" link is clicked, the corresponding account gets deactivated, disappears from the "Active accounts" table, and gets moved to the "Inactive accounts" table. When "Activate" link is clicked, the corresponding account becomes active, disappears from the "Inactive accounts" table, and gets moved to the "Active accounts" table. Odd rows of the table need to be colored with light gray, and even --with dark gray. If one of the tables does not have any entries, then only the header of the table is displayed. If there is a problem connecting to the database, a corresponding error message is displayed. Testing Results Example Function: Registration Inputs (correct): username: dshuffel@cs.ucsd.edu password: mypass confirm password: mypass name: David Shuffelton work phone: 8585551212 other phone: Result: Successfully registered; taken to the welcome page and prompted for further action. Function: Registration Inputs (incorrect): username: blahblah password: mypass confirm password: mypass name: David Shuffelton work phone: 8585551212 other phone: Result: Taken to a page which said "Username entered doesn't appear to be an email address." Function: Registration Inputs (incorrect): username: dshuffel@cs.ucsd.edu password: mypass confirm password: mypass1 name: David Shuffelton work phone: 8585551212 other phone: Result: Taken to a page which said "The passwords entered do not match." Function: Registration Inputs (incorrect): username: dshuffel@cs.ucsd.edu password: mypass confirm password: mypass name: David Shuffelton work phone: 858555121a other phone: Result: Taken to a page which said "The work phone number entered is invalid." Function: Login Inputs (correct): username: dshuffel@cs.ucsd.edu password: mypass Result: Successfully logged in; taken to the welcome page and prompted for further action. Function: Login Inputs (incorrect): username: dshuffel@cs.ucsd.edu password: mypass1 Result: Taken to a page which said "unknown user". Function: Login Inputs (incorrect): username:
password: Result: Taken to a page which said "unknown user". Function: Forgot Your Password Inputs (correct): username: dshuffel@cs.ucsd.edu Result: Redirected to the login page; an e-mail was sent with my (correct) password. Function: Forgot Your Password Inputs (incorrect): username: dshuffel@cs.ucsd.edu1 Result: Taken to a page which said "Unknown user. Please use the browser's back button to try again." Usability Testing Results Example login page: • most links are broken • required fields are not marked as required (flagging with an '*' or red text labels is conventionally done to denote this) • tabbing from username field goes to "call groundwork store" link • 'other phone' fields not inputted into the database, or not retrieved for sOrder form sOrder page: • most links are broken • contact information not precompleted after submitting an order • "next" button not working in textbook order process stage 1 (number of copies) for administrator account • inconsistent labeling of "other phone" appears as "home phone" in textbook order page • after attempting to order a book, and providing a home phone number, the following error is given: 1231231234 1231231234 insert into OrderTable values(0, 13, 1, '1234', 'hi', 'hi', 'hi', 1, now(), 2, 0, 0, 0, 0) select order_number from OrderTable where user_number=13 AND quarter=1 AND year='1234' AND department='hi' AND course_number='hi' insert into BookTable values(0, 53, 'bar', 'foo', '', 'baz', 0, 0, 0, 0, 0, 0, 0, 0, null, 0, 1, 0, 0, 'TRD') insert into BookTable values(0, 53, '', '', '', '', 0, 0, 0, 0, 0, 0, 0, 0, null, 0, 1, 0, , 'TRD') Warning: Cannot add header information -headers already sent by (output started at /var/home/ytsipeny/public_html/cse210/mockups/handlepOrder.php:25) in /var/home/ytsipeny/public_html/cse210/util.php on line 22 • after logging out, hitting back button triggers: Fatal error: Call to undefined function: redirect() in /var/home/ytsipeny/public_html/cse210/mockups/getInfo.php on line 5 • hitting back button after submitting an order can cause a duplicate order submission in IE (form data automatically reposted) • downgrading an account from professor to regular user does not remove order placing capabilities from active sessions: -I log in as a professor -you make me a regular user -I can still place orders