Docstoc

Adaptation of GQM Method for Evaluating the Performance of Software Project Manager

Document Sample
Adaptation of GQM Method for Evaluating the Performance of Software Project Manager Powered By Docstoc
					                                                             (IJCSIS) International Journal of Computer Science and Information Security,
                                                                                                            Vol. 8, No. 9, December 2010

        Adaptation of GQM Method for Evaluating the
          Performance of Software Project Manager
                         Nagy Ramadan Darwish                                         Mahmoud M. Abd Ellatif
              Department of Computer and Information Sciences,                   Department of Information Systems,
                 Institute of Statistical Studies and Research,                 Faculty of Computers and Information,
                                Cairo University                                         Mansoura University
                                   Cairo, Egypt                                            Mansoura, Egypt
                              drnagyd@yahoo.com                                         drmmlatif@yahoo.com

Abstract—This paper is concerned with evaluating the                        For example, scope management includes the following
performance of software project manager using Goal Question          activities:
Metrics (GQM) method. It clarifies the Software Project                 Identifying the project background.
Management (SPM) domains and the performance metrics of each            Assessing the initial feasibility of the project.
domain. This paper presents the basic concepts of GQM method.
Based on a combination of statistical techniques, this paper            Defining project scope and deliverables.
presents how to apply GQM method to evaluate the performance            Listing project assumptions and constraints.
of a software project manager. A software company can use the           Verifying the project scope.
proposed approach to track, evaluate, control, correct, and             Establishing procedures for tracking project progress.
enhance the performance of software project managers to increase        Assessing the project feasibility.
the success rate of software projects.
                                                                        Managing project scope changes.
   Keywords Software ; Project Manager; Performance; Evaluation         Tracking project progress.
- GQM – Metrics – Performance Report
                                                                          System improvement requires measurement and analysis
         I.     INTRODUCTION AND PROBLEM DEFINITION                  [21]. Performance measurements are used in project
     SPM is the on-going activities for planning, organizing,        management and quality processes to determine and
directing, and controlling progress to develop an acceptable         communicate status and accomplishments measured against
system, i.e. conform to the quality standards within the             specific objectives, schedules, and milestones. These
allocated time and budget [11]. The mismanaged projects may          measurements extend to include delivery of desired products
lead to: unfulfilled or unidentified requirements, uncontrolled      and services to customers, whether external or internal [3].
change of project scope, uncontrolled change of technology,          Performance measurement can be useful to improve future
uncontrolled risk of the project, uncontrolled subcontracting        work estimates [15]. Performance measurement is the ongoing
and integration, cost overruns, and/or late delivery [12]. The       monitoring and reporting of project accomplishments,
failure rate of large software projects is larger than the success   particularly progress towards pre-established goals.
rate [16]. Therefore, there is a need to track, evaluate, control,   Performance measures may address: the type or level of project
correct, and enhance the performance of software project             activities conducted, the direct products and services delivered
managers to increase the success rate of software projects.          by a program, and/or the results of those products and services
                                                                     [2].
     GQM method can be used to evaluate the performance of
software project managers. This method was developed for                  A metric is a quantitative measure of the degree to which a
multi-purpose evaluation of software. GQM method consists of         system, component, or process possesses a given attribute [7].
three steps: determination of a Goal, construction of sets of        Performance metrics should be objective, timely, simple,
Questions which have possible answers. The last step is              accurate, useful, and cost-effective. The performance metrics
analytic of the set of metrics, which consists of weight             can be divided into three basic categories [2]: measures of
coefficient for each set of answers [22].                            efforts, measures of accomplishments, and measures that relate
                                                                     efforts to accomplishments.
       II.     SPM DOMAINS AND PERFORMANCE METRICS
                                                                          The researchers elaborated a set of performance metrics
     SPM activities can be organized in nine domains [9]:            for scope management. These elaborated performance metrics
integration management, scope management, schedule                   can be calculated at specific time check points such as project
management, costs management, quality management, human              milestones. In addition, they can be calculated for a given time
resources management, communications management, risk                interval such as a week, month, and so on. Therefore, they can
management, and procurement management. Each domain                  be calculated weekly as an integral part of project progress
includes a set of activities related to a specific field in SPM      report. The elaborated performance metrics for scope
practices.                                                           management include:



                                                                  304                             http://sites.google.com/site/ijcsis/
                                                                                                  ISSN 1947-5500
                                                           (IJCSIS) International Journal of Computer Science and Information Security,
                                                                                                          Vol. 8, No. 9, December 2010
  Percentage of users involved in defining scope and             adaptation of GQM method to fit the performance evaluation of
   deliverables vs. planned.                                      software project manager. Each software company includes a
  Percentage of project deliverables achieved vs. planned.       quality group to evaluate and track the performance of the
  Percentage of project deliverables reviewed and approved       software project managers.
   vs. achieved.                                                       Calculating metrics is a simple process because it depends
  Percentage of major milestones met vs. planned.                on simple or known statistical or mathematical formulas such
  Percentage of project team meetings achieved vs. planned.      as Return On Investment (ROI), payback, percentage, ratio,
  Percentage of scope change requests subjected to               cost deviation, and time deviation (in hours, days, weeks, or
   feasibility studies vs. all requests.                          months).
  Percentage of scope change requests subjected to
   integration tests vs. all requests.                                   Goal                                                      Metrics
                                                                                                  Question
  Percentage of scope change requests subjected to
   configuration management tests vs. all requests.
  Percentage of preliminary feasibility studies achieved vs.
   required at the initiation phase.
  Return On Investment (ROI) calculated for the potential            Goal = SPM
                                                                                                                                       Metric 1

   preliminary project’s costs at the initiation phase.                Domain
                                                                                                   Question 1                      Metric 2
  ROI calculated for the potential detailed project’s costs at
                                                                        Goal 1:
   the planning phase.                                                Integration                  Question 2                      Metric n
  Payback period calculated for the potential preliminary            Management
   project’s costs at the initiation phase.                                                        Question n

  Payback period calculated for the potential detailed
   project’s costs at the planning phase.                                                           Question 1
  Percentage of detailed feasibility studies achieved vs.
   required at the planning phase.                                     Goal 2:                     Question 2
                                                                       Scope
           III.   GOAL QUESTION METRICS METHOD                       Management
                                                                                                   Question n

     Victor Basili and et al at Maryland University developed a
goal oriented approach for measurement [5]. This approach                                           Question 1
depends on three steps:
                                                                       Goal 3:
  Set goals specific to needs in terms of purpose,                   Schedule
                                                                                                   Question2

    perspective, and environment.                                    Management
  Refine the goals into quantifiable questions that are                                           Question n

    tractable.
  Deduce the metrics and data to be collected (and the means
    for collecting them) to answer the questions.

      In GQM method, each goal generates a set of quantifiable
questions that attempt to define and quantify this goal. The
question can only be answered relative to, and as completely                                       Question 1
as, the available metrics allow. In GQM, the same question can
be used to define multiple goals. Also, metrics can be used to         Goal 9:
                                                                                                   Question 2                      Metric 1
                                                                     procurement
answer more than one question. Unfortunately, this approach
                                                                     Management
suits to mature and well-understood problem areas.                                                  Question                       Metric 2
                                                                                                       n

      IV. THE PROPOSED APPROACH FOR EVALUATING                                                                                     Metric n
      PERFORMANCE OF SOFTWARE PROJECT MANAGER
                                                                            Figure (1): GQM Method and SPM domains.
     GQM method can be used to evaluate the performance of
software project managers. Each SPM domain has a goal, for              Figure (2) illustrates a general flowchart that presents the
example schedule management aims to administrate and              proposed approach. The proposed approach includes the
control of the finite resource of time. The three conventional    following main procedures:
measures of project success are budget, schedule, and                Define the goals, questions, and performance metrics.
functionality. The project manager must manage the schedule
                                                                     Calculate the value of each performance metric and
carefully for preventing or correcting any slippages. Each goal
                                                                       compare it with the accepted range of the metric value.
is decomposed into several questions, and each question can be
                                                                     Interpret and analyze the performance report.
answered by a metric or more. Figure (1) illustrates the




                                                               305                              http://sites.google.com/site/ijcsis/
                                                                                                ISSN 1947-5500
                                                                                       (IJCSIS) International Journal of Computer Science and Information Security,
                                                                                                                                      Vol. 8, No. 9, December 2010

                                                                                                     Table (1) illustrates an elaborated list of questions and
A. Define the Goals, Questions, and Performance Metrics
                                                                                                performance metrics for scope management.
    Each SPM domain have a goal, therefore there are nine
goals: integration management, scope management, schedule
                                                                                                         Question                                  Metric
management, costs management, quality management, human                                               Q1: Is the users      1.   Percentage of users involved in defining scope
resources management, communications management, risk                                               involved in defining         and deliverables vs. planned.
management, and procurement management. Each goal is                                                     scope and
decomposed into several questions, and each question can be                                            deliverables?
answered by a metric or more. The definition of each metric                                                                 1.   Percentage of project deliverables achieved vs.
should include the mathematical or statistical techniques for                                   Q2: Is the project               planned.
                                                                                                deliverables planned,       2.   Percentage of project deliverables reviewed and
calculating this metric.                                                                        achieved, reviewed,              approved vs. achieved.
                                                                                                and approved.
                                      Start
                                                                                                      Q3: Is the major      1.   Percentage of major milestones met vs. planned.

                     Define the goals, questions, and
                                                                                                      milestones of the
                          Performance metrics                                                           project met?

                                                                                                    Q4: Are the project     2.   Percentage of project team meetings achieved
                                                                                                      team meetings              vs. planned.
         Calculate the value of each performance metric and compare it
                   with the accepted range of the metric value                                          achieved?
                                                                                                                            1.   Percentage of scope change requests subjected
                                                                                                                                 to feasibility studies vs. all requests.
             Interpret and analyze the performance report
                                                                                                      Q5: Are the scope     2.   Percentage of scope change requests subjected
                                                                                                    change requests dealt        to integration tests vs. all requests.
                                                                                                           well?            3.   Percentage of scope change requests subjected
                                     End                                                                                         to configuration management tests vs. all
                                                                                                                                 requests.
Figure (2): The proposed approach for Evaluating Performance                                                                1.   Percentage of preliminary feasibility studies
                 of Software Project Manager.                                                                                    achieved vs. required at the initiation phase.
                                                                                                                            2.   Return On Investment (ROI) calculated for the
                                                                                                                                 potential preliminary project’s costs at the
In addition, the definition of each metric should include the                                                                    initiation phase.
accepted range of the metric value. The time check points for                                                               3.   ROI calculated for the potential detailed
calculating performance metrics should be determined. These                                     Q6: Are the feasibility          project’s costs at the planning phase.
                                                                                                     studies do?            4.   Payback period calculated for the potential
metrics can be calculated weekly as a part of project progress                                                                   preliminary project’s costs at the initiation
report. The project manager should be involved in this process.                                                                  phase.
The quality group should present the SPM performance metrics                                                                5.   Payback period calculated for the potential
to the project manager and deal with his objections by                                                                           detailed project’s costs at the planning phase.
                                                                                                                            6.   Percentage of detailed feasibility studies
clarifying, negotiating, or modifying these metrics. Previous                                                                    achieved vs. required at the planning phase.
experience from similar projects can be useful in this process.
In addition, this process can be achieved with the assistance of                                      Table (1): The elaborated list of questions and performance
external consultants to define and validate the SPM                                                                 metrics for scope management.
performance metrics. Figure (3) illustrates the steps of this
procedure.                                                                                      B. Calculate the Performance Metrics and compare it with the
                                                                                                    accepted range
1.    Select a goal from the list of goals of SPM domains (nine goals).
2.    Define a list of questions related to the selected goal.                                       The second procedure of the proposed approach is
3.    Select a question from the list of questions.
                                                                                                calculating the performance metrics for the questions of a
4.    Define a list of performance metrics related to the selected question.
                                                                                                specific goal. Calculating performance metrics is a simple
5.    Select a performance metric to be defined in detail.
                                                                                                process because it depends on simple or known statistical or
                                                                                                mathematical formulas. So, the researchers don't focus on
6.    Define the mathematical or statistical technique for calculating the selected metric.
                                                                                                calculating performance metrics. After calculating the value of
7.    Define time check points for calculating the selected metric.
                                                                                                the performance metric, the quality groups compare this value
8.    Define the accepted range of the metric value of the selected metric.
                                                                                                with the accepted range of the metric value. If the metric value
9.    Negotiate the project manager and deal with his objections by clarifying or
                                                                                                is out of the accepted range, the quality groups add a deviation
      modifying the performance metric.                                                         note to the performance report. The quality groups prepare a
10.   Check the list of performance metrics. If it is not empty, then go to step 5.             performance report that must include the metric value, accepted
11.   Check the list of questions. If it is not empty, go to step 3.                            range, and any deviation notes. Some performance metrics may
12.   Check the list of goals. If it is not empty, go to step 1.                                be Not Applicable (NA) in some specific cases [19]. So, during
                                                                                                computing the value of the performance metrics, the NA
 Figure (3): The definition of the Goals, Questions, and Performance Metrics.




                                                                                              306                                   http://sites.google.com/site/ijcsis/
                                                                                                                                    ISSN 1947-5500
                                                                   (IJCSIS) International Journal of Computer Science and Information Security,
                                                                                                                  Vol. 8, No. 9, December 2010
metrics are eliminated. Figure (4) illustrates the steps of this                  www.crim.ca/Publications/2010/documents/plein_texte/ASD_AliE_al_
procedure.                                                                        QAOOSE_2010.pdf
                                                                           [5]    Fenton, Norman, Robin Whitty and Yoshinori lizuka, ―Software
C. Interpret and Analyze the Performance Report                                   Quality Assurance and Measurement‖, International Thomson
                                                                                  Computer Press, 1995.
     The third procedure of the proposed approach is                       [6]    Girish H. Subramanian, James J. Jiang, and Gary Klein, ―Software
interpreting and analyzing the performance report. The quality                    quality and IS project performance improvements from software
group should report their interpretation to their top                             development process maturity and IS implementation strategies‖, the
                                                                                  Journal of Systems and Software 80, 2007, pages: 616–627
management. If the performance deviations is not accepted, top             [7]    IEEE Standard 610,‖Glossary of Software Engineering‖, 1990.
management may take corrective actions or inform the project               [8]    Ince, ―Software Quality Assurance - A Student Introduction‖,
manager to take corrective actions. The value of performance                      McGraw-hill international (UK) limited, 1995.
deviations should be analyzed to discover the weaknesses and               [9]    Information Systems Audit and Control Foundation (ISACF),‖Project
                                                                                  Management: Skill and Knowledge Requirements in an Information
strengths of project management practices. This analysis can be                   Technology Environment‖, 2002.
used to reduce or avoid many risks or obstacles that may be                [10]   ISO 10006,‖Quality Management Systems – Guidelines for Quality
encountered in later points of time or in next software projects.                 management in Projects‖, Second Edition, 2003.
                                                                           [11]   Jeffrey A. Hoffer, Joey F. George,and Joseph S. Valacich, ―Modern
                         V.    CONCLUSION                                         System Analysis and Design‖, Addison Wesley Longman, Inc, 1999.
                                                                           [12]   Jeffrey L. Whitten, Lonnie D. Bentley and Kevin C.Dittman, ―System
      The main objective of this paper was proposing an                           Analysis and Design Methods‖, Fifth edition, Mc Graw Hill Companies,
approach for evaluating the performance of software project                       Inc, 2001.
                                                                           [13]   Joel Henry,‖Software Project Management – A Real-World Guide to
manager using GQM method. In GQM method, each goal
                                                                                  Success‖, Pearson Education, Inc, 2004.
generates a set of quantifiable questions that attempt to define           [14]   Jolyon E. Hallows, ―Information Systems Project Management: How to
and quantify this goal. The question can only be answered                         Deliver Function and Value in Information Technology Projects‖,
relative to, and as completely as, the available metrics allow.                   AMACOM, a division of American Management Association, 1998.
                                                                           [15]   Karl E. Wiegers, ‖A Software Metrics Primer‖, Process Impact, 1999,
Evaluating the performance of software project manager is                         http://www.processimpact.com/articles/metrics_primer.html
helpful for increasing capability level and productivity,                  [16]   Paul Dorsey, ‖Top 10 Reasons Why Systems Projects Fail‖, 2000,
improving quality, tracking project progress, and assessing                       http://www.duclcian.com/papers/top%2010%20reasons%20why%20sy
project status.                                                                   stems%20projects%20fail.htm
                                                                           [17]   Paul Goodman, ―Software Metrics: Best Practices for Successful IT
      We conclude that the roles of quality group are very                        Management‖, Rothstein Associates, 2004.
important in software projects. They can use the list of                   [18]   Robert T. Futrell, Donald F. Shafer, and Linda I. Safer, ―Quality
performance metrics and the proposed approach to evaluate and                     Software Project Management‖, Prentice Hall PTR, 2002.
track the performance of the software project manager. In                  [19]   William E Perry, ―Quality Assurance for Information Systems:
                                                                                  Methods, Tools, and Techniques‖, QED technical publishing Group,
addition, we conclude that special emphasis must be given to
                                                                                  1991.
performance metrics in software projects to discover and avoid             [20]   Yael Dubinsky and Orit Hazzan, ―Using a role scheme to derive
the weaknesses of practices. On the other hand, the strengths of                  software project metrics‖, Journal of Systems Architecture 52 pages:
project management practices must be utilized and encouraged.                     693–699, 2006.
                                                                           [21]   Zahedi, Fatemeh, ―Quality Information Systems‖, Boyd & Fraser
                       VI.    FUTURE WORK                                         Publishing Company - a Division of International Thomson Publishing
                                                                                  Inc., 1995.
     For future work, the following points are expected to be              [22]   Zdena Dobešová1 and Dagmar Kusendová2, ―Goal-Question-Metric
focused:                                                                          method for evaluation of cartographic functionality in GIS software‖,
                                                                                  proceedings GIS Ostrava, 2009,
 Improvements in integration and scope management of                             www.geoinformatics.upol.cz/app/visegrad/images/GISOstrava.pdf
    software projects.
 Improvements in schedule and cost management of
    software projects.
 Achieving higher levels in Capability Maturity Model
    Integration (CMMI) for IT companies.
 Enhancements in the quality of e-government projects.
                             REFERENCES
[1]   Chang E. Koh, Victor R. Prybutok, and Xiaoni Zhang, ―Measuring e-
      government readiness‖, Journal of Information & Management 45
      pages: 540–546, 2008.
[2]   Department of Energy (DOE), ‖Basic Performance Measures for
      Information Technology Projects‖, 2002,
      http://cio.doe.gov/ITReform/sqse/download/PE-WI-V3-011502.doc
[3]   Department of Energy (DOE), ‖IT Project Management Review
      Process Guide‖, 2003,
      http://cio.doe.gov/ITReform/sqse/download/QR-CP-F3-011403.doc
[4]   El Hachemi Alikacem and Houari A. Sahraoui, ―Rule-Based System
      for Flaw Specification and Detection in Object-Oriented Programs‖,
      2010,




                                                                       307                                 http://sites.google.com/site/ijcsis/
                                                                                                           ISSN 1947-5500

				
DOCUMENT INFO
Description: The International Journal of Computer Science and Information Security (IJCSIS) is a well-established publication venue on novel research in computer science and information security. The year 2010 has been very eventful and encouraging for all IJCSIS authors/researchers and IJCSIS technical committee, as we see more and more interest in IJCSIS research publications. IJCSIS is now empowered by over thousands of academics, researchers, authors/reviewers/students and research organizations. Reaching this milestone would not have been possible without the support, feedback, and continuous engagement of our authors and reviewers. Field coverage includes: security infrastructures, network security: Internet security, content protection, cryptography, steganography and formal methods in information security; multimedia systems, software, information systems, intelligent systems, web services, data mining, wireless communication, networking and technologies, innovation technology and management. ( See monthly Call for Papers) We are grateful to our reviewers for providing valuable comments. IJCSIS December 2010 issue (Vol. 8, No. 9) has paper acceptance rate of nearly 35%. We wish everyone a successful scientific research year on 2011. Available at http://sites.google.com/site/ijcsis/ IJCSIS Vol. 8, No. 9, December 2010 Edition ISSN 1947-5500 � IJCSIS, USA.