Docstoc

Tight rope

Document Sample
Tight rope Powered By Docstoc
					Walking the Tight-Rope
Software Project Management in Real Life
Urbán Zoltán, Várszegi György 2004

Introduction
 There are well established theories about project management. In practice they are easiest to follow for
» small-sized independent (demo) projects » government financed mega-projects

 The real challenge is to manage multiple projects in parallel in a competitive environment under time, budget and resource pressure  Using well-established general practices can still reduce the risks
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 2

Who we are
 ScanSoft, Inc. is a public company based in Boston, MA. With nearly 800 employees worldwide, it is the market-leading supplier of speech and imaging solutions.  ScanSoft-Recognita Rt. in Budapest is its only imaging development hub. Size of its engineering is about 90 heads  The ratio between development and QA is close to 2:1

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi

3

Who we are
 Our product portfolio is:
» » » » » OmniPage: stand-alone OCR PDF Converter Pro: opening and creating PDF files PaperPort: document management system OmniPage SDK: imaging development toolkit OmniForm: form designer and data acquisition solution

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi

4

Engineering Process
 We have two basic lines of work:
» Retail (boxed) products
– a classic project management concept to deliver major product versions

» Customization projects
– a relatively high number of minor projects controlled by available resources and priorities defined by sales

 We will concentrate on the first type in this presentation

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi

5

Roles
 For retail product projects the customer is not the contracting party for the development
» Product Delivery Team (PDT)
– Defines and delivers the product – A cross-functional group of area representatives
 Program manager: the coordinator to drive toward team consensus  Product manager, marketing, international  Project manager, QA Lead, documentation, engineering  Technical support  Manufacturing

» Product Approval Committee (PAC)
– Approves, finances and supervises the project – Senior management of the company
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 6

Engineering Phases / Milestones
 Strategic vision
» Kick-off PAC review

 Product definition
» Product and market definition PAC review

 Product design
» Design PAC review

 Coding
» Alpha acceptance

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi

7

Engineering Phases / Milestones
 Alpha
» UI freeze » Beta readiness PAC review » Beta acceptance

 Beta
» First release candidate (RC0) » Launch PAC review » Release to manufacturing (RTM)

 Sustaining
» RTM of localized versions, patches, point releases
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 8

PAC reviews
 Regular PAC reviews
» Planned delimiters of the phases » PDT demonstrates the current state and the successful termination of the previous phase » PDT presents a detailed plan for the rest of the project
– Engineering: deliverables (25 varieties), schedule (100 date points), resource plan, technical overview, quality plan, documentation, localization, 3rd party components, risks

» PAC decision can be
– – – – GO (contract for the next phase) Redirect (major change in the course of the project) Retry the review (inadequate input from PDT) Cancel the project
9

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi

PAC reviews
 Exception reviews
» Any material change in the course of the project that affects the “contract” for the current phase should prompt an exception review
– Unexpected major market changes – Missed milestones – Budget overrun

» Usually PDT (Program Manager) initiates them » The review materials and the PAC decision criteria are basically the same as at the regular reviews

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi

10

Life Cycle / Definition, Design
 Our approach to these phases is iterative  Initial Marketing Requirements Document
» Product marketing creates it. MRD0 for a 60 man-year project is typically less than 10 pages with about 60 new product features with very few details.

 Functional Specification
» Engineering has pretty wide freedom in implementation details » Feature-based (short and comprehensive descriptions, technical details, licensed components, test methodology, benchmarking, use cases, risks) » Incremental for existing product lines
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 11

Life Cycle / Definition, Design
 Feature matrix
» Important communication and decision-making tool
– To set realistic project goals (feedback to MRD) – To follow the coding progress until Alpha (weekly updates)

» Required resources for each feature
– Rows: features with owners and links to the feature descriptions – Columns: resource names – Cells: necessary man weeks (special skills considered)

» Available resources
– Considering holidays, other projects, project management

» Padding 25-50%
– for target features, unforeseen events, underestimated features
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 12

Life Cycle / Definition, Design
 Feature matrix (cont.)
» Priority of the feature
– Minimal (committed) / Target (may be realized) – Dropped (by marketing or a target feature during coding)

 Feature meetings
» Useful to understand the feature implementation details and their effects » 3-4 features discussed daily with
– – – – – Project manager Director Developer(s) implementing the feature QA Lead Technical writer
13

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi

Life Cycle / Coding
 Coding (Implementation)
» High risk items first
– New 3rd party items or new versions of them should be definitely taken care of first

» Experts with unique knowledge
– Very effective and very vulnerable

» Technology test group within development
– But unit tests are not a general rule

» No obligatory coding standard » Currently code review only for new hires
– Extension under preparation – Planned after Alpha, based on bug statistics
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 14

Life Cycle / Coding
 Coding (cont.)
» Unified installer configurations » Avoid sharing components run-time between separate products
– It increases complexity a lot

 Version control
» Using unified folder structures

 Build process
» In-house automated nightly build process with labeling, binary versioning and error reporting (including automated test script results after Alpha)
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 15

Life Cycle / Coding
 QA preparation
» Test Case List
– A list of all the project-related planned / implemented Test Cases

» Test Case
– One suite of manual test steps to check a specific item of product functionality
 We write them in the form of test automation scripts

» Test Matrix
– Plan / report about the performed test steps
 who performed the test, which test case, when, how long it took, operating system, build id, bugs reported

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi

16

Life Cycle / Coding
 QA preparation (cont.)
» Number of Test Cases
– No theoretical upper limit. A higher number yields better coverage, but raises costs. – We plan weekly test cycles. Test cases are sized to fit all functionality tests under one operating system into one week for one tester. – We usually test on 5 operating systems

» Basically finalized by mid-Alpha
– Usually tuned even later based on the test results, external beta test reports, random tests

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi

17

Life Cycle / Coding
600 500 400 300 200 100 0 minimum target

The trend of the number of man-weeks necessary to implement the minimum and target features predicts Alpha date
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 18

12 /1 7/ 20 12 02 /3 1/ 20 01 02 /1 4/ 20 01 03 /2 8/ 20 02 03 /1 1/ 20 02 03 /2 5/ 20 03 03 /1 1/ 20 03 03 /2 5/ 20 04 03 /0 8/ 20 04 03 /2 2/ 20 05 03 /0 6/ 20 03

Life Cycle / Alpha
 Milestones
» Alpha acceptance test
– Run on Alpha-candidate(s) – Test cases to check feature availability – Benchmarked parameters with Alpha thresholds
 About 25 parameters: accuracies, speeds, stability, leak etc.

» Marketing review
– Last-minute call for marketing to tune the UI and the features – Resources reserved for an iteration cycle

» UI Freeze
– Finalize program resources then user guide and help
 These items are usually on the critical path for the localized releases

– Localization starts
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 19

Life Cycle / Alpha, Beta
 The primary goal is to detect and fix bugs  Based on years of experience in three different companies, both the Alpha and the Beta phase should be 10 weeks long  Compression disorganizes the process and finally results in at least the same amount of time  Bug track database is used with a bug fixing workflow
» Allowed state transitions per role, change history, online reports on the intranet

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi

20

Life Cycle / Alpha, Beta
 Tools / QA
» Usually 3 computers for each tester » Disk images for each platform and hardware » Test automation
– It has become more relevant recently – Tests are run on about 20 machines 24 by 7 – Script development
 Starts only in Alpha because it is very sensitive to structural changes  Implemented test steps are commented out from the test cases so manual testers do not perform them  Pretty expensive to prepare them for all kinds of failure situations and to give a meaningful test report  During script development a high portion of bugs is detected  Scripts pay off in the regression tests of the sustaining phase
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 21

Life Cycle / Alpha, Beta
 Tools / Developers
» Test machines with disk images and multi-OS emulators are used to reproduce bugs without disturbing QA » Special software tools to fix hard-to-isolate problems
– Memory over- and underwrites – Memory leaks – Threading problems

» Using common components for
– Logging (run-time selectable level) – Setting debug variables run-time – Letting system-level exceptions be caught in JIT debugger
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 22

Life Cycle / Beta
 Beta acceptance test
» Run on Beta candidate(s) » Using all test cases » Benchmarked parameters with Beta thresholds

 External Beta cycles
» » » » Managed by technical support Base test scripts prepared by QA We usually have 2-3 external beta cycles Important to get test results from
– Real-life, non-clean software environments – Machines with software and hardware components not available inhouse
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 23

Life Cycle / Beta
25.00% 20.00% 15.00% 10.00% 5.00% 0.00% -5.00% 1 -10.00% -15.00% -20.00% -25.00% 4 7 10 13 16 19 22 25 28 31 34 Actual Criteria

The trend of benchmarked parameters
» Predicts dates when Alpha/Beta/RTM thresholds can be met » Especially useful considering historical data trends e.g.: improvement in the last 10 weeks
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 24

Life Cycle / Beta
OP14 Oct 31 RTM
1200 1000

Open bugs

800 600 400 200 0 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 Weeks until RTM

OP11 OP12 OP14

Statistical analysis of the bugs
» Most useful before RC » Comparison with historical data results in more reliable estimates e.g. turning point in open bugs happened 12 weeks before RTM for the previous version
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 25

Life Cycle / Beta
 Bug meetings
» Low-priority bugs are requested to be deferred to reduce the number of changes and the associated risk » The PDT (mainly technical support) decides to accept or refuse the request » Daily meetings in the last test cycle(s)

 Release candidate
» Very important milestone
– Does every participant believe that the build, with all its known problems, could be released, if the next test cycle does not detect any new bug?

» Check-in only through the project manager » Tests performed from the final media
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 26

Life Cycle / Sustaining
 Deliverables
» Localized versions, trials, point releases, patches

 Team
» Separate small team of 2-3 developers » Very QA-intensive period. Typically 5 operating systems and max. 2-3 languages tested in parallel.

 Archival
» Seems to be simple so long as no-one tries to recreate the build (tools, instructions, environment) » Current version control system is not reliable enough » Sources on DVD and masters on CD kept safe in 2 copies geographically separated
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 27

Project control
 ISO 9001:2000 and Tick’IT certification
» External audits twice a year » Internal audits 2-3 times a year » Control documents on the intranet

 Software Development Handbook
» Describes the development process » Specifies locations for tools, documents, source code, defect database etc. » Very useful for new hires

 Our project quality plan is expressed in a lifecycle form on the intranet with all the milestones and the placeholders for the required documents
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 28

Project control
 Build reports
» Informs development and QA about the location and status of the current build and the known issues

 Test reports
» QA summarizes its findings in the form of a test report about important builds (Alpha, Beta, RC)

 PDT conference calls once a week  Detailed project status reports every second week

Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi

29

Deviations
Schedule compression
»Usually a time-to-market requirement

Losing resources
»Mainly to a higher priority project

Creeping features
»Changes in market conditions or late discoveries about cross-effects may cause features to creep

3rd parties and outsourcing
»For economical reasons suppliers with much inferior quality systems may be selected
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 30

Deviations
Dealing with deviations
»Cutting administrative corners “temporarily”
–No process revisions –Skipping reports –Not updating project documents –Putting archival tasks aside

»Design
–Feature bargaining

»Coding
–Using padding and dropping (target) features

»Alpha and Beta
–Shortening test cycles (but not reducing their number) –Hiring more temps (typically QA)
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 31

Conclusions
 Our engineering process is far from being optimal from a project management point of view
» Other presentations try to describe such ideal methods » Listening to these, you’d probably feel guilty as we did for not following all their instructions and advice

 However, we believe our approach is closer to optimal economically  The evidence for this may be the number of our successfully delivered projects  We know that behind the implementation and operation of each small step towards an ideal plan there lies “blood, sweat and tears”
Walking the Tight Rope, 2004 / Zoltan Urban, Gyorgy Varszegi 32


				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:38
posted:11/14/2009
language:English
pages:32