Peer Review Procedures

Guidelines for Agency Peer Reviews Office of Information Technology Services (ITS) Enterprise Project Management Office (EPMO) Effective Date: 05/09/2000 Revised Date: 03/10/2006 _________________________________________________________________________ 1 The Review Process This process defines the peer review guidelines for the Office of Information Technology Services, Enterprise Project Management Office (ITS/EPMO), State agency Project Office (PO) / Program Management Office (PMO) staff to ensure that any work document created within the department will be reviewed by the subject matter experts and other department members prior to release. The primary goal of the peer review is to detect and remove work document defects as early as possible in the development process. The secondary goals are:  Comply with IEEE Standard 1028, Standard for Software Reviews.         Ensure correctness, completeness, consistency, and accuracy of work documents for all life cycle activities within the development process. Educate project staff and enhance their insight into project specific work documents and processes. Provide a technically correct base for the next phase of development. Increase product quality. Achieve lower life-cycle cost. Provide a first indication of program maintainability. Increase project productivity. Encourage entry/exit-criteria project management.  Improve project development processes. Documents that may require a review include the following:  Process documents  Procedures documents  Checklists  Templates  Project and work related documents such as request for proposals, requirements, design, software architecture, statement of work, project plan, and test plan, etc. 1.1 Purpose and Scope The purpose of this document is to define and control the work document review process. The objective of the peer review is to evaluate the work document for correctness, completeness, and adherence to standards and requirements. It is important that the review process be understood and adhered to by all participants. This document describes:  Methodology for review process D:\Docstoc\Working\pdf\29b53c35-c5ac-4878-aba6-6cabbdd3b2da.doc Page 1 of 8 10/22/09 2:05 AM Guidelines for Agency Peer Reviews   Review entry and exit criteria Dispositions for the review meeting This process begins when a draft of a work document (example: a process document) is ready for review. Appendix B contains a “Peer Review Checklist” and “Summary Log” to aid the moderator, author and review participants in carrying out peer review activities. 1.2 Definitions Following are definitions of commonly used terms:    Document Control: The organization responsible for documentation filing procedures. Item: A thing written down at the peer review meeting. This consists of issues, improvement suggestions and questions of intent to the author. Meeting: A forum used to formally present the work document where there is a face to face or verbal interaction between all participants. The work document being presented must be reviewed with all participants able to communicate and interact with each other at the same time. The meeting can take place using televideo, teleconference, shared computer, or hard copy viewing. Moderator: The person who leads the meeting and is responsible for the overall review meeting process. Peer: A member of the State agency staff who is of equal standing with the author of the work document, and is a participant of the review meeting. Peer is especially familiar with the techniques used by the author, and ensures that the work document is correct. Review: A formal peer meeting held to allow discussion of issues, alternatives, options, and for the work document approval. Scribe: A peer review meeting participant who records the items identified at the meeting. Work Document: The material to be reviewed.      1.3 Meeting Participants, Roles, and Responsibilities The peer review team members include the moderator, the author, and the reviewers. The review meeting participants roles and responsibilities are discussed below. 1.3.1 Moderator The moderator plays an important role in peer reviews to ensure the compliance with and overall effectiveness of the process. Moderator’s responsibilities are as following:  Attend and participate in all reviews; is the KEY person in a successful meeting.  Is an independent participant (a participant that is not directly working on the work document under review, and does not have a vested interest in the work document).  Is neither the author of the work document nor the manager of the work document.  May also be the recorder if the total attendance for the meeting is three participants.  Verifies the entry criteria are met. D:\Docstoc\Working\pdf\29b53c35-c5ac-4878-aba6-6cabbdd3b2da.doc Page 2 of 8 10/22/09 2:05 AM Guidelines for Agency Peer Reviews      Moderates the meeting; keeps meeting focused; controls the pace of the meeting; offers leadership. Provides impartial arbitration at the meeting. Summarizes the open issues at the end of the review meeting and polls the participants to determine the disposition of the review; if the team can not agree on a disposition, the moderator makes the decision on what the disposition will be. Verifies that all exit criteria are met. Verifies that the appropriate approvals are obtained. 1.3.2 Author The author has well-defined responsibilities, both before and after the review meeting as listed below:  Prepare the work document under review.  Work with management to ensure that the necessary participants attend the meeting and to ensure that the necessary approvals are obtained.  Selects the moderator.  Notifies the participants about the review within the appropriate time frame.  Delivers the product package to the participants within the appropriate time frame.  Resolves open issues recorded at the meeting.  Ensures that all exit criteria are met. 1.3.3 Reviewer The reviewer has the following responsibilities:  Prepares for and participates in a review.  Identifies issues, defects, improvements, interactions, and other pertinent information required for the success of the work document.  Praises good work. 1.3.4 Scribe The scribe’s responsibilities are:  Is a reviewer, author, or moderator.  Should be appointed when work document package is made available.   Brings the appropriate “summary log” and forms to the meeting. Records all identified issues, defects, and suggested improvements at the meeting. 1.4 Entry Criteria for Reviews The purpose of having entry criteria to the review process is to ensure that the time spent in reviewing the work document and associated documents is well spent. Following is the entry criteria for the review process: D:\Docstoc\Working\pdf\29b53c35-c5ac-4878-aba6-6cabbdd3b2da.doc Page 3 of 8 10/22/09 2:05 AM Guidelines for Agency Peer Reviews    Completed and edited work document using a standard template as appropriate Supporting documentation Moderator selected by author 1.5 Review Process Phases The document may be partitioned into units to allow for incremental reviews. The work document reviews are complete after all issues have been discussed and after all incremental reviews have been held. A peer review process consists of the following three phases: planning, meeting, and rework. 1.5.1 Planning Phase The objective of the planning phase is organization and preparation. The author delivers the work document to the moderator. The moderator reviews the work document. If the moderator determines that the work document is ready for review, the moderator continues with the planning phase of the review process. If the work document is not ready for review the document is returned to the author for rework. The moderator consults with the author to assist him/her with planning. This plan includes:    Identify and invite participants to review the work document. Divide the work document into reviewable units. Establish mutually convenient meeting times for the review team and schedule the first review meeting.  Ensure that team members get copies or have access to electronic copies of all relevant documents at least three working days prior to the first meeting date.  Reserve suitable meeting room and/or arrange for teleconferencing. The review team may consist of as many reviewers as necessary to achieve review objectives. It is recommended that the review team consist of the work document subject matter experts. The minimum number of participants for the review team is three. All participants must prepare for the review before the meeting. Adequate preparation is a key ingredient to an effective and successful review meeting. 1.5.2 Review Meeting Phase The review meeting reviews the work document to:    Discuss improvements and suggestions identified by reviewers prior to the meeting. Discuss additional improvements and suggestions identified during the meeting. Record improvement suggestions, and questions to be resolved by the author. The Summary Log (in Appendix B) may be used to record defects, improvement suggestions, issues and action items that are identified at the meeting. D:\Docstoc\Working\pdf\29b53c35-c5ac-4878-aba6-6cabbdd3b2da.doc Page 4 of 8 10/22/09 2:05 AM Guidelines for Agency Peer Reviews 1.5.2.1  Meeting Recommendations The meeting should last no longer than two hours. If two hours is insufficient time to review the work document, then another meeting should be scheduled. No more than two meetings should be scheduled in one day. All comments are to be directed to the group as a whole (the work document is the subject of the meeting, not the author). The meeting is attended by the author, the moderator, and the other selected reviewers only. To avoid any chance that the meeting might take on the trappings of a performance review, management is specifically excluded. The Review Meeting Disposition   1.5.2.2    The review team recommends one of the following work document dispositions: Accept: The document is approved. Accept with Recommended Changes: If all changes can be easily made and the rework will not significantly alter parts of the work document, the author makes all changes and the moderator verifies the changes. Review Rework: If changes that significantly alter the direction of the work document are required, the document enters the rework phase and the same team will be asked to review the reworked document. If additional expertise is required, new members may be added to the review team. 1.5.3 Rework Phase During rework, the author is responsible for ensuring resolution of all open issues and completing the rework. All changes to the document are to be made using revision mode (make revision while editing, i.e., activate Track Changes in Word documents) so that reviewers can easily identify and verify the changes. Any changes to this version of the work product must be communicated to the review team. In case of major changes, another review of the work document may be required. 1.6 Exit Criteria for Reviews In order to closely manage the process, the exit criteria for the process must be clearly defined. The exit criteria for the review process include:  Open issues logged during the meeting are resolved    Changes, edits, and improvements suggested during the meeting are completed Approval Form in Appendix A is completed and signed The work document was placed under configuration management system. Note: Any exception to the above procedure requires approval by the author’s manager. D:\Docstoc\Working\pdf\29b53c35-c5ac-4878-aba6-6cabbdd3b2da.doc Page 5 of 8 10/22/09 2:05 AM Guidelines for Agency Peer Reviews APPENDIX A: Approval Form Document Name: Author: Author Phone Number: Revision Number/Issue Date: Reason for Revision: Approval: ______________________________________________ Author’s Manager Date D:\Docstoc\Working\pdf\29b53c35-c5ac-4878-aba6-6cabbdd3b2da.doc Guidelines for Agency Peer Reviews APPENDIX B: Peer Review Checklist and Summary Log Done # 1 2 3 4 Verify entry criteria. Action Identify review participants. Schedule review meeting and send meeting notice. Owner moderator author and moderator moderator Introduce and state objectives of review meeting; and moderator summarize the previous meeting minutes and status of open issues. Determine adequate meeting attendance. Review the work document using applicable standards. Record work document defects issues. moderator participants author and moderator 5 6 7 8 9 10 11 Record open issues when the team cannot readily author and moderator decide if the item in question is a defect or not. Review the open issues for the team at the end of the moderator meeting. Reach a team disposition. consensus about the meeting moderator Close the meeting. If a significant amount of moderator controversy or arguments have occurred during the meeting, the moderator may wish to spend a few moments on team building before closing. Correspondingly if the meeting was especially effective, the team should be complemented. Verify that defects and open issues have been author and moderator addressed. Verify that exit criteria have been met. moderator 12 13 D:\Docstoc\Working\pdf\29b53c35-c5ac-4878-aba6-6cabbdd3b2da.doc Page 7 of 8 Guidelines for Agency Peer Reviews Summary Log Page ____ of ____ Date _____/____/____ Work Document Name/ID ______________ Version _________ Recorder: ___________________ Moderator: ___________________ Verify Closure Section/ Paragraph Line #/ Page # Item Description D:\Docstoc\Working\pdf\29b53c35-c5ac-4878-aba6-6cabbdd3b2da.doc Page 8 of 8