Tight rope (PowerPoint)

Document Sample
Tight rope (PowerPoint) Powered By Docstoc
					Walking the Tight-Rope
    Software Project Management
            in Real Life

       Urbán Zoltán, Várszegi György
                   2004
                                    Introduction
 There are well established theories about project
  management. In practice they are easiest to
  follow for
      » small-sized independent (demo) projects
      » government financed mega-projects
 The real challenge is to manage multiple projects
  in parallel in a competitive environment under
  time, budget and resource pressure
 Using well-established general practices can still
  reduce the risks

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi   2
                                   Who we are
 ScanSoft, Inc. is a public company based in
  Boston, MA. With nearly 800 employees
  worldwide, it is the market-leading supplier of
  speech and imaging solutions.
 ScanSoft-Recognita Rt. in Budapest is its only
  imaging development hub. Size of its engineering
  is about 90 heads
 The ratio between development and QA is close
  to 2:1


Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi   3
                                   Who we are
 Our product portfolio is:
      »   OmniPage: stand-alone OCR
      »   PDF Converter Pro: opening and creating PDF files
      »   PaperPort: document management system
      »   OmniPage SDK: imaging development toolkit
      »   OmniForm: form designer and data acquisition
          solution




Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi   4
                       Engineering Process
 We have two basic lines of work:
      » Retail (boxed) products
             – a classic project management concept to deliver major
               product versions
      » Customization projects
             – a relatively high number of minor projects controlled by
               available resources and priorities defined by sales
 We will concentrate on the first type in this
  presentation



Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi              5
                                            Roles
 For retail product projects the customer is not the
  contracting party for the development
      » Product Delivery Team (PDT)
             – Defines and delivers the product
             – A cross-functional group of area representatives
                    Program manager: the coordinator to drive toward team
                     consensus
                    Product manager, marketing, international
                    Project manager, QA Lead, documentation, engineering
                    Technical support
                    Manufacturing
      » Product Approval Committee (PAC)
             – Approves, finances and supervises the project
             – Senior management of the company

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi                 6
     Engineering Phases / Milestones
 Strategic vision
      » Kick-off PAC review
 Product definition
      » Product and market definition PAC review
 Product design
      » Design PAC review
 Coding
      » Alpha acceptance




Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi   7
     Engineering Phases / Milestones
 Alpha
      » UI freeze
      » Beta readiness PAC review
      » Beta acceptance
 Beta
      » First release candidate (RC0)
      » Launch PAC review
      » Release to manufacturing (RTM)
 Sustaining
      » RTM of localized versions, patches, point releases

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi   8
                                  PAC reviews
 Regular PAC reviews
      » Planned delimiters of the phases
      » PDT demonstrates the current state and the successful
        termination of the previous phase
      » PDT presents a detailed plan for the rest of the project
             – Engineering: deliverables (25 varieties), schedule (100 date
               points), resource plan, technical overview, quality plan,
               documentation, localization, 3rd party components, risks
      » PAC decision can be
             –   GO (contract for the next phase)
             –   Redirect (major change in the course of the project)
             –   Retry the review (inadequate input from PDT)
             –   Cancel the project

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi                  9
                                  PAC reviews
 Exception reviews
      » Any material change in the course of the project that
        affects the “contract” for the current phase should
        prompt an exception review
             – Unexpected major market changes
             – Missed milestones
             – Budget overrun
      » Usually PDT (Program Manager) initiates them
      » The review materials and the PAC decision criteria
        are basically the same as at the regular reviews



Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi    10
          Life Cycle / Definition, Design
 Our approach to these phases is iterative
 Initial Marketing Requirements Document
      » Product marketing creates it. MRD0 for a 60 man-year
        project is typically less than 10 pages with about 60
        new product features with very few details.
 Functional Specification
      » Engineering has pretty wide freedom in
        implementation details
      » Feature-based (short and comprehensive descriptions,
        technical details, licensed components, test
        methodology, benchmarking, use cases, risks)
      » Incremental for existing product lines

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi   11
          Life Cycle / Definition, Design
 Feature matrix
      » Important communication and decision-making tool
             – To set realistic project goals (feedback to MRD)
             – To follow the coding progress until Alpha (weekly updates)
      » Required resources for each feature
             – Rows: features with owners and links to the feature
               descriptions
             – Columns: resource names
             – Cells: necessary man weeks (special skills considered)
      » Available resources
             – Considering holidays, other projects, project management
      » Padding 25-50%
             – for target features, unforeseen events, underestimated features

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi                12
          Life Cycle / Definition, Design
 Feature matrix (cont.)
      » Priority of the feature
             – Minimal (committed) / Target (may be realized)
             – Dropped (by marketing or a target feature during coding)
 Feature meetings
      » Useful to understand the feature implementation
        details and their effects
      » 3-4 features discussed daily with
             –   Project manager
             –   Director
             –   Developer(s) implementing the feature
             –   QA Lead
             –   Technical writer
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi              13
                         Life Cycle / Coding
 Coding (Implementation)
      » High risk items first
             – New 3rd party items or new versions of them should be
               definitely taken care of first
      » Experts with unique knowledge
             – Very effective and very vulnerable
      » Technology test group within development
             – But unit tests are not a general rule
      » No obligatory coding standard
      » Currently code review only for new hires
             – Extension under preparation
             – Planned after Alpha, based on bug statistics
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi           14
                         Life Cycle / Coding
 Coding (cont.)
      » Unified installer configurations
      » Avoid sharing components run-time between separate
        products
             – It increases complexity a lot
 Version control
      » Using unified folder structures
 Build process
      » In-house automated nightly build process with
        labeling, binary versioning and error reporting
        (including automated test script results after Alpha)
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi    15
                         Life Cycle / Coding
 QA preparation
      » Test Case List
             – A list of all the project-related planned / implemented Test
               Cases
      » Test Case
             – One suite of manual test steps to check a specific item of
               product functionality
                    We write them in the form of test automation scripts
      » Test Matrix
             – Plan / report about the performed test steps
                    who performed the test, which test case, when, how long it
                     took, operating system, build id, bugs reported


Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi                      16
                         Life Cycle / Coding
 QA preparation (cont.)
      » Number of Test Cases
             – No theoretical upper limit. A higher number yields better
               coverage, but raises costs.
             – We plan weekly test cycles. Test cases are sized to fit all
               functionality tests under one operating system into one week
               for one tester.
             – We usually test on 5 operating systems
      » Basically finalized by mid-Alpha
             – Usually tuned even later based on the test results, external
               beta test reports, random tests



Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi                  17
                         Life Cycle / Coding
           600

           500

           400
                                                               minimum
           300
                                                               target
           200

           100

             0
               02

               02

               03

               03

               03

               03

               03

               03

               03

               03

               03
            20

            20

            20

            20

            20

            20

            20

            20

            20

            20

            20
         7/

         1/

         4/

         8/

         1/

         5/

         1/

         5/

         8/

         2/

         6/
       /1

       /3

       /1

       /2

       /1

       /2

       /1

       /2

       /0

       /2

       /0
     12

     12

     01

     01

     02

     02

     03

     03

     04

     04

     05
The trend of the number of man-weeks necessary to implement
     the minimum and target features predicts Alpha date

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi             18
                           Life Cycle / Alpha
 Milestones
      » Alpha acceptance test
             – Run on Alpha-candidate(s)
             – Test cases to check feature availability
             – Benchmarked parameters with Alpha thresholds
                    About 25 parameters: accuracies, speeds, stability, leak etc.
      » Marketing review
             – Last-minute call for marketing to tune the UI and the features
             – Resources reserved for an iteration cycle
      » UI Freeze
             – Finalize program resources then user guide and help
                    These items are usually on the critical path for the localized
                     releases
             – Localization starts
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi                          19
                   Life Cycle / Alpha, Beta
 The primary goal is to detect and fix bugs
 Based on years of experience in three different
  companies, both the Alpha and the Beta phase
  should be 10 weeks long
 Compression disorganizes the process and finally
  results in at least the same amount of time
 Bug track database is used with a bug fixing
  workflow
      » Allowed state transitions per role, change history, on-
        line reports on the intranet


Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi   20
                   Life Cycle / Alpha, Beta
 Tools / QA
      » Usually 3 computers for each tester
      » Disk images for each platform and hardware
      » Test automation
             – It has become more relevant recently
             – Tests are run on about 20 machines 24 by 7
             – Script development
                    Starts only in Alpha because it is very sensitive to structural
                     changes
                    Implemented test steps are commented out from the test cases
                     so manual testers do not perform them
                    Pretty expensive to prepare them for all kinds of failure
                     situations and to give a meaningful test report
                    During script development a high portion of bugs is detected
                    Scripts pay off in the regression tests of the sustaining phase
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi                           21
                   Life Cycle / Alpha, Beta
 Tools / Developers
      » Test machines with disk images and multi-OS
        emulators are used to reproduce bugs without
        disturbing QA
      » Special software tools to fix hard-to-isolate problems
             – Memory over- and underwrites
             – Memory leaks
             – Threading problems
      » Using common components for
             – Logging (run-time selectable level)
             – Setting debug variables run-time
             – Letting system-level exceptions be caught in JIT debugger

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi               22
                             Life Cycle / Beta
 Beta acceptance test
      » Run on Beta candidate(s)
      » Using all test cases
      » Benchmarked parameters with Beta thresholds
 External Beta cycles
      »   Managed by technical support
      »   Base test scripts prepared by QA
      »   We usually have 2-3 external beta cycles
      »   Important to get test results from
             – Real-life, non-clean software environments
             – Machines with software and hardware components not
               available inhouse

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi        23
                             Life Cycle / Beta
                25.00%
                20.00%
                15.00%
                10.00%
                 5.00%
                                                                   Actual
                 0.00%
                                                                   Criteria
                 -5.00% 1      4    7 10 13 16 19 22 25 28 31 34
               -10.00%
               -15.00%
               -20.00%
               -25.00%


The trend of benchmarked parameters
      » Predicts dates when Alpha/Beta/RTM thresholds can be met
      » Especially useful considering historical data trends
        e.g.: improvement in the last 10 weeks
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi                  24
                                   Life Cycle / Beta
                                             OP14 Oct 31 RTM

              1200
              1000
  Open bugs




               800                                                                         OP11
               600                                                                         OP12
               400                                                                         OP14
               200
                 0
                     25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
                                               Weeks until RTM



Statistical analysis of the bugs
               » Most useful before RC
               » Comparison with historical data results in more reliable estimates
                 e.g. turning point in open bugs happened 12 weeks before RTM
                 for the previous version
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi                                 25
                             Life Cycle / Beta
 Bug meetings
      » Low-priority bugs are requested to be deferred to
        reduce the number of changes and the associated risk
      » The PDT (mainly technical support) decides to accept
        or refuse the request
      » Daily meetings in the last test cycle(s)
 Release candidate
      » Very important milestone
             – Does every participant believe that the build, with all its
               known problems, could be released, if the next test cycle does
               not detect any new bug?
      » Check-in only through the project manager
      » Tests performed from the final media
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi               26
                    Life Cycle / Sustaining
 Deliverables
      » Localized versions, trials, point releases, patches
 Team
      » Separate small team of 2-3 developers
      » Very QA-intensive period. Typically 5 operating
        systems and max. 2-3 languages tested in parallel.
 Archival
      » Seems to be simple so long as no-one tries to recreate
        the build (tools, instructions, environment)
      » Current version control system is not reliable enough
      » Sources on DVD and masters on CD kept safe in 2
        copies geographically separated
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi   27
                                Project control
 ISO 9001:2000 and Tick’IT certification
      » External audits twice a year
      » Internal audits 2-3 times a year
      » Control documents on the intranet
 Software Development Handbook
      » Describes the development process
      » Specifies locations for tools, documents, source code,
        defect database etc.
      » Very useful for new hires
 Our project quality plan is expressed in a life-
  cycle form on the intranet with all the milestones
  and the placeholders for the required documents
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi   28
                                Project control
 Build reports
      » Informs development and QA about the location and
        status of the current build and the known issues
 Test reports
      » QA summarizes its findings in the form of a test report
        about important builds (Alpha, Beta, RC)
 PDT conference calls once a week
 Detailed project status reports every second week



Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi   29
                                     Deviations
Schedule compression
      »Usually a time-to-market requirement
Losing resources
      »Mainly to a higher priority project
Creeping features
      »Changes in market conditions or late discoveries about
      cross-effects may cause features to creep
3rd parties and outsourcing
      »For economical reasons suppliers with much inferior
      quality systems may be selected

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi   30
                                     Deviations
Dealing with deviations
      »Cutting administrative corners “temporarily”
             –No process revisions
             –Skipping reports
             –Not updating project documents
             –Putting archival tasks aside
      »Design
             –Feature bargaining
      »Coding
             –Using padding and dropping (target) features
      »Alpha and Beta
             –Shortening test cycles (but not reducing their number)
             –Hiring more temps (typically QA)

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi           31
                                   Conclusions
 Our engineering process is far from being
  optimal from a project management point of view
      » Other presentations try to describe such ideal methods
      » Listening to these, you’d probably feel guilty as we did for not
        following all their instructions and advice
 However, we believe our approach is closer to
  optimal economically
 The evidence for this may be the number of our
  successfully delivered projects
 We know that behind the implementation and
  operation of each small step towards an ideal
  plan there lies “blood, sweat and tears”
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi               32

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:4
posted:5/24/2012
language:English
pages:32