Scrum Burn Down Template

Document Sample
Scrum Burn Down Template Powered By Docstoc
					 How to be a Successful Software
Development Engineer at Microsoft
                      or
       Things You Didn’t Learn in School



             Bill Hanlon
   Software Development Engineer
            Visual Studio
             Microsoft
                    Agenda
•   Test-Driven Development
•   Agile Techniques
•   Design Patterns: Beyond the "Gang of Four"
•   How to Write Maintainable Code
•   “Working Effectively with Legacy Code”
Test-Driven Development


             a.k.a.
  “Continuous Refactoring”
  “Test-First Development”
     “Emergent Design”
            Wild Claims about TDD
•   TDD reduces bug density for new code…
    •   … by 77% (from MS VS Core experience)
    •   … by 50% (from IBM study)
    •   … by 90% (my gut)


•   TDD generates better, cleaner code
•   TDD raises developer confidence
•   TDD is fun!
                    TDD - Step 1
• for the new software intent, before you write
  any source code, write a test
  – how big is an “intent”?
     • definitely not bigger than a method call
     • possibly several intents in each method (depending on
       parameters or state)
     • “baby steps” are good, especially early
  – Example tests for a stack:
     •   create an empty stack, verify size == 0
     •   push a single object, verify size == 1
     •   push and pop a single object, verify size == 0
     •   push and pop an object, verify got same object, etc.
                   TDD - Step 2
• Write just enough code for the test to Fail
  – why?
      • Occasionally the test will pass
      • Failing the test tests the test
  – never write a test that succeeds the first time:
      • process of changing from failure to success
        validates a test
  – stub out the call
      • throw an exception;
      • return a bad value or null; or,
      • do nothing
                   TDD - Step 3
• Write just enough code for the test to pass
  – don’t write too much source yet, because...
     • ... you don’t have tests in place for them yet!
  – you’ll be writing more tests shortly
     • add more source only then
  – don’t design too much future functionality into
    the code - YAGNI
                 TDD – Step 4
• Refactor all your source
  – not just for large architectural changes
  – removing duplication in code drives refactoring
  – Martin Fowler lists 72 different refactorings
  – poorly factored code is hard to test
     • feedback promotes loose coupling, tight cohesion
     • increasingly better design and code
  – Well-factored code tends to be more flexible and
    maintainable
                  TDD – Step 5
• goto 1
  – elapsed time < 10 minutes, often <<
  – a large refactoring may take > 10 minutes
     • but you do them in small steps, each of which is << 10
       minutes
     • “rapid unhurriedness” – Fowler
  – an “if” statement implies at least two tests
                   TDD Summary
•       write a test
•       write enough source to make the test Fail
•       write enough source to make the test Pass
•       refactor – all tests must Pass
                                       NUnit
    –     “Red, Green, Refactor”:
    –     the TDD rhythm
    –     “isolation testing”
        How & Why TDD Works
• Quickly, easily generate a live, up-to-date
  suite of regression tests
• close to 100% code coverage
• often as much test code as source code
  – tests must drive the code down all code paths
• tests make great sample code!
• rather than wasting time debugging, just
  throw away the last few minutes of coding
  and take a smaller step
     How & Why TDD Works (2)
• You have control, security and freedom to
  improve your designs at any time
  – how often is your first design your best idea?
• In what kind of component would you rather
  fix an old bug, a component with or without a
  huge suite of unit tests?
            Psychology of TDD
• Tests must run fast; else they won’t be run at
  all
• In non-TDD programming,
  – tests can only bring bad news;
  – in TDD, tests help with your designs
• having all tests pass after major refactoring
  feels great
  – you just made your code and design better
            TDD: Guidance/Advice
• Try it!
   – New skill
   – Get good at it


• “Do as much design (up front) as makes you
  feel comfortable, and TDD the rest.”
              Agile Techniques
• http://AgileManifesto.org

• Project Management
  – “Why should I care?”
  – Scrum
     • Certified ScrumMaster
           Problem Statement
• Software Projects Slip a lot
  – Scheduling is hard
  – Cost Estimation is hard
  – Control is hard
  – We can’t tell we’re slipping until late
  – Team gets randomized
                    Agile Manifesto
•   Individuals and interactions over processes and tools
•   Working software over comprehensive documentation
•   Customer collaboration over contract negotiation
•   Responding to change over following a plan

    – “While there is value in the items on the right, we value the
      items on the left more.”
    – http://AgileManifesto.org
  How Software Projects are Typically
             Managed
• use MSProject
• Dev Estimates are really bad - why?
  – incomplete knowledge
  – little or no feedback
• Sum of really bad estimates (completion date)
  is also really bad
               What Scrum isn’t
•   a design methodology
•   a programming methodology
•   a quality control methodology
•   a documentation methodology

    – you are free to use whatever you like to address
      these concerns. Scrum is meant to coexist with
      existing practices.
  Defined vs. Empirical Processes
• Defined – think “assembly line”
  – well-known
  – predictable
  – repeatable
• Empirical
  – chaotic
  – unpredictable
  – non-repeatable
• Frequent Sampling and Adjustments
             How does Scrum Work?
• Product Backlog
   – all the items to be done in order to ship a product
• Product Owner
   – the guy who priorities the Product Backlog
• Sprint (or Iteration)
   – 30 days (or less)
• Sprint Backlog
   – all the items to be completed in a Sprint
   – each item has an estimated amount of work remaining
• Sprint Demo
   – meeting at end of the Sprint, when working software is demoed
• Scrum Team
   – 7 +/- 2 devs and testers
• Daily Scrum Meeting
   – daily 15 minute stand-up meeting of the Scrum team
• Scrum Master
   – the guy who conducts the Scrum Meeting
             Scrum Artifacts

• Product Backlog List

• Release Burn-down Chart
• Sprint Burn-down Chart
            Burn-Down Chart
            900

            800

            700

Estimated   600


  Work      500


Remaining   400


    in      300


  Hours     200

            100

             0
                  1   3   5    7   9   11   13   15   17   19   21   23   25   27   29   31



                              Elapsed Time in Days
Alternative Sprint Chart
Release Burn-down Bar Chart
              Planning a Sprint
• Sprint planning session part 1
  – PBO explains top highest priority items from the
    Product Backlog List
• Sprint planning session part 2
  – Team estimates how many of those items it can
    accomplish in this Sprint
  – Turns those items into work items
     • This becomes the Sprint Backlog
             Daily Scrum Meeting
• 15 minutes max.
• all Team members must attend
• Scrum Master asks each member 3 questions:
  – what have you done since the last Scrum meeting?
  – what will you do before the next Scrum meeting?
  – what impedes you from performing your work as effectively
    as possible?
  – (optional) did you add or remove any items from the Sprint
    Backlog list?
  – (optional) did you learn anything that may be of interest to
    the team?
     Sprint Review: Demo Meeting

• Output from a Sprint is working software
  – demonstrated at the Sprint Review meeting
• All stakeholders give input
  – “Responding to change over following a plan”


• Scalability – “Scrum of Scrums”
             Why Scrum Works
• face-to-face accountability
• team camaraderie
• hyper-productivity
  – functional roles are discouraged
  – collocation is encouraged
• empower the team
        Agile: Guidance/Advice
• Care about project management!
  – If you don’t, someone else will
  – and they may not know what they’re doing

• Read up on this:
  – Agile Development
  – Lean Development
  – eXtreme Programming
          Design Patterns –
       Beyond the Gang of Four
• Gang of Four’s “Design Patterns” book, 1995
  – 23 patterns
• “Pattern Languages of Program Design (1-5)”
  – Authors include:
     • Ward Cunningham
     • James Coplien
     • Robert Martin
  – The first two volumes were terrible
             Some Comments
• http://en.wikipedia.org/wiki/Category:Software_d
  for a list, probably not complete
• Some patterns now dated; we know more now
  – Visitor has a well-known cyclic dependency
  – Singleton causes temporal coupling if mutable


• Survey – what’s your favorite pattern?
              My Favorite Pattern:
                  Null Object
Problem: Pointers/references to polymorphic
  objects may be null, and you’re often checking for
  null

void Snippet(Customer* pCustomer)
{
  Plan * plan;
  if (pCustomer == null) plan = BillingPlan::getBasicPlan();
  else plan = pCustomer->getPlan();
   // …
Null Object – Class Diagram
                Null Object - Example
class IProgressObserver
{
public:
   virtual void OnProgress(unsigned char) = 0; // 0 - 255: 255 == done
   virtual void Finished(HRESULT) = 0;
};
 
class NullProgressObserver : private IProgressObserver
{
    virtual void OnProgress(unsigned char) {}
    virtual void Finished(HRESULT) {}
public:
    static IProgressObserver& GetNullProgressObserver()
    {
          static NullProgressObserver npo;
          return npo;
    }
};
Design Patterns – Guidance/Advice
• Keep learning
  – The industry is always moving forward
     • We’ve learned things since 1995 (GoF)
     • Ditto C++: some STL concepts are now considered anti-
       patterns
  – Read pattern books
  – Take pattern courses
  – Attend conferences
  How to Write Maintainable Code
• “Big Ball of Mud” – www.laputan.org/mud
   – Experience
   – Skill
   – Time
   – Cost
   – Complexity
   – Change
• Peter Principle of Programming
   – “we build the most complex thing that can be made to
     work”
                  Experience
• What does bad (unmaintainable) code look like?
• Bad code is a class that:
  – has hundreds of data members;
  – has hundreds of methods, most public;
  – implements lots of interfaces;
  – is coupled to lots of other classes;
  – uses friend classes.
 What does Good Code Look Like?
• “loose coupling, tight cohesion”
   – But what does this look like?

• One class, one responsibility
• Data members:
   – either minimal (0, 1, 2), or
   – all data changes at the same frequency
• Be faithful to the Mental Model of the architecture
• A class should have few public methods,
• implement few interfaces
• Be minimally coupled, no cyclic coupling
          Complexity: Coupled Code
• A class A depends on (or is coupled to) another class B if
   –   it inherits from B
   –   it contains a B
   –   it has a method that returns a B
   –   it has a method that takes a B
   –   it has a method that uses a B (e.g., an automatic variable)
   –   it does any of the above with a template (or generic) with
       parameter-type B
   Complexity: Temporal Coupling
• Avoid globals, statics and singletons

• “C++ Coding Standards”, Sutter & Alexandrescu:
  – “the correctness of a piece of code that uses shared
    data is conditioned on the history of changes to the
    data”
Change: Test-Driven Development
• This is the most important one!
• TDD generates a large suite of tests that you
  can use as Regression Tests
  – You can refactor safely at any time
  – What to refactor?
• Large list at http://c2.com/cgi/wiki?CodeSmell
    How to Write Maintainable Code:
           Guidance/Advice
•   Use TDD
•   No cyclic-coupled code
•   No temporal coupling
•   Maintain mental fidelity
•   Read
     – Sutter’s & Alexandrescu’s “C++ Coding Standards”
 “Working Effectively with Legacy
             Code”
• Read book of same title by Michael Feathers
  – “Legacy Code” definition:
  – Code that doesn’t have unit tests
• You have a “Big Ball of Mud” - now what?
  – Run Away
  – Swamp Guide
  – Sweep It Under the Rug, a.k.a. Quarantine
  – Reconstruction, a.k.a. Total Rewrite
     Legacy Code, Age & File Size
• MS Research investigated where the bugs were in
  Win2K:
  – Large old files didn’t have many bugs
  – New files with simple classes didn’t have many bugs
  – Medium-sized files had most churn and bugs

• Conclusion:
  – There’s value in old mature code.
  – Don’t throw it away without a good reason.

• Question: How do you avoid creating those pesky
  middle-sized files?
           Keep it Working:
       Refactoring Legacy Code
• Chicken & Egg problem:
  – You need tests to refactor safely
  – it’s not easy to write tests, afterwards.
     • Legacy code is not particularly testable
• You need to refactor legacy code just to make
  it more testable!

• Solution: Apply “Pretty Safe” Refactorings
              “Pretty Safe”
         Refactoring Techniques
• General Concepts:
  – “Preserve Signature”
  – “Lean on the Compiler”


• Specific Techniques:
  – “Extract Interface”
  – “Virtualize Method”
  – “Subclass And Override” a.k.a., “Test Sub-Class”
       How to Mock an Object:
         “Extract Interface”
• Interfaces are critical for Mock Objects
• Interfaces change less than classes
• “Preserve Signature” applies to class and
  interface definitions, too.
       How to Mock an Object:
         Virtualize Method
• Copy-n-paste method signature from class
  into its newly-extracted interface
• Add “virtual” keyword; make pure virtual
• “Lean on the Compiler”

  – Let the compiler find the (virtual) methods that
    belong in the interface
         Subclass and Override
• Problem: a method in the system under test
  contains a call to a “bad” API:

  – Too slow, hoses registry, sends email, etc.
  – You don’t want to call this “bad” API every time
    you run the tests
             Subclass and Override
Before:
class Foo {
public:
   void Bar() { CallToAReallySlowFunction(); }
};
After:
class Foo {
public:
   void Bar() { CallToAReallySlowFunction(); }
   virtual void CallToAReallySlowFunction()
   { ::CallToAReallySlowFunction(); }
};
class TestFoo : public Foo {
   virtual void CallToAReallySlowFunction() { }
};
               Friend Classes
           Accessing Private Data

• “Extract Interface” (no data members)
• For each directly accessed data member:
  – Compile the system under test:
     • “X is not a member of” interface
  – Add public accessor to interface
     • Search-and-replace friend accesses w/ accessor
            Link Dependencies
• Linking many .libs into one .dll
  – Can’t test .lib separately: unresolved externals
• Short-term Solution:
  – Link in .objs, not .libs
  – Write many small (mostly throwing) stubs
• Long-term Solution:
  – Convert all dependencies between .libs to
    Directed Acyclic Graphs (DAGs)
“Working Effectively with Legacy Code”:
               Advice/Guidance
•   Don’t write brand-new legacy code!
•   Refactor unrelengtingly
•   Write a TDD-style test for each new bug
•   Quarantine working code
•   Rewrite worst code
      How to be a Successful Software
     Development Engineer at Microsoft
                                     or
                      Things You Didn’t Learn in School



                        Bill Hanlon
              Software Development Engineer
                       Visual Studio
                        Microsoft
                                                                                   1




1.   Babble about me for a bit:
         1. ChEg/Math, not CS (25 years ago!)
         2. Cyclical industry – “what else do I know how to do?”
         3. Really happy
                  1. coulda been stuck in a chemical plant for 30 years
                  2. Software industry changes constantly – always something new
                       happening
2.   10 years at MS
         • Windows (MMC), Windows Server, Windows Networking, Visual Studio
              Core, VS Setup
         • Previous – lots of graphics work (bit twiddling)




                                                                                       1
                                    Agenda
•    Test-Driven Development
•    Agile Techniques
•    Design Patterns: Beyond the "Gang of Four"
•    How to Write Maintainable Code
•    “Working Effectively with Legacy Code”




                                                                                         2




1.   IMO, Microsoft has been successful (i.e., $$$) by of “extraordinary efforts by
     extraordinary people” – no longer good enough
         1. Why? Code bases are getting really big:
                  • Windows – 64 million lines of code
                  • even Visual Studio – 44 million lines of code
         2. Too big for even the most extraordinary architect to hold in his head
                  1. Even if he could, so many people are working on it, that the
                      architect couldn’t control every aspect

2.   What I plan on talking about during this talk are things I’ve discovered recently
     (i.e., ~5 years or so)
           •    that often aren’t taught (or can’t be taught) at school
           •    that really help
                    • with quality
                    • with shipping on time
                    • with writing good code, now and
                    • writing good code with an eye to the future

•    Caveat: my opinions only, based on my experiences.

4.   Feel free to interrupt, ask questions, heckle, etc.




                                                                                             2
              Test-Driven Development


                                 a.k.a.
                      “Continuous Refactoring”
                      “Test-First Development”
                         “Emergent Design”


                                                                                            3




2.   TDD != testers are in charge
3.   TDD != QA practice
4.   TDD will not put testers out of business

6.   TDD is a development activity, not a test activity, that gives really good results….




                                                                                                3
                    Wild Claims about TDD
•        TDD reduces bug density for new code…
         •   … by 77% (from MS VS Core experience)
         •   … by 50% (from IBM study)
         •   … by 90% (my gut)


•        TDD generates better, cleaner code
•        TDD raises developer confidence
•        TDD is fun!
                                                                                         4




    1.   When I joined VSCore, a couple of years ago, one team (MSBuild) had been doing
         TDD for about 9 months
            • The 77% number is from comparing them against the other teams in VS
                 Core

    2.   Dr. Laurie Williams of North Carolina State University:
              1. They have 109K LOC and 2500 tests =>
              2. 1 test per 43 lines of code
                      • Not nearly enough!
                      • Likely really lousy code coverage numbers
              3. IBMers => just barely doing TDD.
    •    I trust my gut.
              1. I don’t expect anyone else to.
              2. I do expect people to try TDD.
    •    by constant refactoring (will talk more about this later)
             1. Both JP & I said independently of each other: “every time I add a new
                 feature, my code gets smaller”
    •    because the code passes a large suite of developer-written tests
    •    I’ve been coding professionally for a long time (25 years: since before PCs!)
              • I think TDD is the best thing since sliced bread.




                                                                                             4
                            TDD - Step 1
• for the new software intent, before you write
  any source code, write a test
     – how big is an “intent”?
          • definitely not bigger than a method call
          • possibly several intents in each method (depending on
            parameters or state)
          • “baby steps” are good, especially early
     – Example tests for a stack:
          •   create an empty stack, verify size == 0
          •   push a single object, verify size == 1
          •   push and pop a single object, verify size == 0
          •   push and pop an object, verify got same object, etc.
                                                                                    5




being purposefully vague about what an “intent” is, and how big it is:


if you know a lot about the problem at hand, then an intent can be pretty big:
if it’s not clear in your head, take “baby steps”.


Note: some people think they’re doing TDD if they write all the tests first. This
is not the case.




                                                                                        5
                        TDD - Step 2
• Write just enough code for the test to Fail
  – why?
      • Occasionally the test will pass
      • Failing the test tests the test
  – never write a test that succeeds the first time:
      • process of changing from failure to success
        validates a test
  – stub out the call
      • throw an exception;
      • return a bad value or null; or,
      • do nothing
                                                                  6




1. (why?): especially when using someone else’s component.
2.
3. stub): have examples of each mechanism of stubbing out calls
      • throwing exception – works great w/ NUnit




                                                                      6
                          TDD - Step 3
• Write just enough code for the test to pass
     – don’t write too much source yet, because...
         • ... you don’t have tests in place for them yet!
     – you’ll be writing more tests shortly
         • add more source only then
     – don’t design too much future functionality into
       the code - YAGNI



                                                                             7




resist the temptation to write too much code! Note: this temptation goes away
pretty quickly; replaced w/ reluctance to add new code without tests in place.


YAGNI – “You aren’t gonna need it”




                                                                                 7
                          TDD – Step 4
• Refactor all your source
     – not just for large architectural changes
     – removing duplication in code drives refactoring
     – Martin Fowler lists 72 different refactorings
     – poorly factored code is hard to test
            • feedback promotes loose coupling, tight cohesion
            • increasingly better design and code
     – Well-factored code tends to be more flexible and
       maintainable
                                                                                  8




1. Not years: every few seconds!
2. (removing): I find that about 95% of my refactorings are due to finding
duplication in code; the rest is simplifications, better ideas, etc.
3. smells
4. (loose coupling, etc.): there is natural feedback here: you get better tests
and code for free!




                                                                                      8
                                 TDD – Step 5
• goto 1
       – elapsed time < 10 minutes, often <<
       – a large refactoring may take > 10 minutes
            • but you do them in small steps, each of which is << 10
              minutes
            • “rapid unhurriedness” – Fowler
       – an “if” statement implies at least two tests



                                                                                             9




1. Often seconds – speed is critical!
         1. I have a batch file that builds, then runs the tests
                  1. elapsed time == < 3 seconds.

2.   A couple of things to note:
         1. Each step is quite small, but executed quickly: “rapid unhurriedness” – Fowler
                   • You generate lots of tests quickly (ideally)
         2. If your code has an “if” statement,
                   • You’ll need at least two tests
                                 1. One that enters the if clause
                                 2. One that doesn’t




                                                                                                 9
                         TDD Summary
•       write a test
•       write enough source to make the test Fail
•       write enough source to make the test Pass
•       refactor – all tests must Pass
                                       NUnit
    –      “Red, Green, Refactor”:
    –      the TDD rhythm
    –      “isolation testing”

                                                                                 10




1. Aiming for 100% code coverage
        1. Necessary, but not sufficient: isolation testing
        2. Difficult to get >85% without mockObjects
               • Not true, I guess: I got 98.6 on my last project w/o mocks


“isolation testing”


•   We’re trying to get good test coverage of the code at our layer
        1. Necessary, but not sufficient
             • If our layer has poor code coverage, there are guaranteed to
                be bugs in the system.
             • If our layer has great code coverage, there may still be
                integration bugs
                      1. QA does end-to-end testing, integration testing, etc.
        2. Our tests must be blazingly fast
               1. Can be, because of mocks




                                                                                      10
              How & Why TDD Works
• Quickly, easily generate a live, up-to-date
  suite of regression tests
• close to 100% code coverage
• often as much test code as source code
     – tests must drive the code down all code paths
• tests make great sample code!
• rather than wasting time debugging, just
  throw away the last few minutes of coding
  and take a smaller step
                                                                                 11




1. suite of tests is run constantly; all test must pass; can’t get out-of-date
2. ideally (my experience: between 80 and 90%. Anecdotally, >95% => you’re
   making the tool happy)
      • Not any more! I’ve been getting ~95 with little effort.
3. dev tests only! there should be many more Test-style tests!
      • Typically, 125% as much dev-test code as source code.
•   sample code that never gets out-of-date
•   no debugging! (do comment on best practice of stepping through all the
    code)




                                                                                      11
          How & Why TDD Works (2)
• You have control, security and freedom to
  improve your designs at any time
     – how often is your first design your best idea?
• In what kind of component would you rather
  fix an old bug, a component with or without a
  huge suite of unit tests?



                                                                             12




1. “refactoring: improving the design of existing code” – book by Fowler
2. in the Windows org and here in DevDiv, code lives for years. Judiciously fix
   a bug in some code that was written years ago, test as best you can, toss
   over the wall to testers and tell them to bang on it really hard.... => lousy
   way to work


4. Tell story of lousy C++ code here in devdiv – “locally safest change
   possible”




                                                                                   12
                     Psychology of TDD
• Tests must run fast; else they won’t be run at
  all
• In non-TDD programming,
     – tests can only bring bad news;
     – in TDD, tests help with your designs
• having all tests pass after major refactoring
  feels great
     – you just made your code and design better

                                                                                      13




•   heavy-weight test harnesses will not do! Speed is all-important. We need
    results now, not over-night. TDD is not regression testing.
•   Unit tests (written after) are the first thing to get cut, under time pressure.
       • TDD != Testing Activity
       •   TDD == Design Tool
•   psychologically, TDD feels great. I’ve been programming for > 25 years,
    and I’m having more fun now than ever.




                                                                                           13
                  TDD: Guidance/Advice
• Try it!
      – New skill
      – Get good at it


• “Do as much design (up front) as makes you
  feel comfortable, and TDD the rest.”



                                                                                      14




1.   It’s a different way of programming
         1.   But if/when you get it, you’ll never go back to programming the “old”
              way – life changing event
         2.   Not everyone “gets” it.
                  •    Lots of people claim to do TDD, but only do so in a half-hearted
                       way



•    My experience: lots of extremely senior people (architects) don’t get it:
         1.   ?
         2.   Smartest people we have (can argue well) – been very successful at
              writing code the “old” way
                  1.   rather be highly competent the “old” way, than
                  2.   Initially somewhat incompetent at the “new” way.




                                                                                           14
                        Agile Techniques
• http://AgileManifesto.org

• Project Management
      – “Why should I care?”
      – Scrum
             • Certified ScrumMaster




                                                                                       15




1.   What is Agile?
         1.   “Agile Manifesto” – signed by industry luminaries in 2001
                  •    Ward Cunningham (Wiki)
                  •    Martin Fowler
                  •    Robert Martin, etc.
         2.   (slide after next explains in more detail)


•    I wanna talk about Project Management for a bit
         1.   You might say “why would I care? I just wanna write software.”
         2.   It’s going to impact you:
                  •    You may be managed by non-programmers, or at best, ex-
                       programmers
                  •    In particular, most companies, especially large companies
                       haven’t got a clue how to manage a software project
                  •    (more on this)


•    Scrum – Rugby term, having to do with getting the ball back into play via self-
     organization.


•    I attended a talk given by Ken Schwaber (at MS)
         •    intrigued enough to buy his first book
         •    when I read it, “Wow: this is how I’d like to be managed!”
         •    Later I got my certification….




                                                                                            15
                   Problem Statement
• Software Projects Slip a lot
     – Scheduling is hard
     – Cost Estimation is hard
     – Control is hard
     – We can’t tell we’re slipping until late
     – Team gets randomized




                                                                                    16




1. Scheduling and Controlling a Software Project is hard
       1. Scheduling and Costing is hard, because
              1. devs don’t know enough at the beginning of the project
                     • unless you’re building the exact same thing: so, why are you
                       building it again?
              2. it always impresses the heck outa me that testers can cost their
                 work so precisely.
       • Control is hard because
              1. it’s so easy to write a spec to add yet another cool feature,
              2. code is so easy to write, and
              3. bugs are so easy to introduce
       • We can’t tell we’re slipping because we’re not tracking the right things
         (more on this later)
       • according to PSP/TSP and CMM guy (Watts Humphrey – IBM VP), a dev
         writes code for 1 hour a day.
              1. The rest is spent in meetings, reading email, sending status, getting
                 randomized, etc.
              2. anecdote: when the management had an offsite, the dev
                 productivity doubled.




                                                                                         16
                            Agile Manifesto
•    Individuals and interactions over processes and tools
•    Working software over comprehensive documentation
•    Customer collaboration over contract negotiation
•    Responding to change over following a plan

      – “While there is value in the items on the right, we value the
        items on the left more.”
      – http://AgileManifesto.org



                                                                                           17




    1. Ken Schwaber and Mike Beedle, the inventors of Scrum, are signatories of “The
       Agile Manifesto”.
    2. Most of this talk will center on the last bullet point, but Scrum also values the
       other three points extremely highly.
             1. In fact Scrum & TDD go together really well
                      1. tell Agile2005 conference story


    4. Q. How does this help?




                                                                                                17
    How Software Projects are Typically
               Managed
• use MSProject
• Dev Estimates are really bad - why?
     – incomplete knowledge
     – little or no feedback
• Sum of really bad estimates (completion date)
  is also really bad

                                                                                        18




1. In MSProject – I cringe when I see MSProject charts hanging on the walls....
   (Pert/Gannt charts)
       1. ask devs for estimates for each task
       2. add up total, load balance, etc. => get an estimated completion date
              1. unfortunately, dev estimates are typically really bad
                      1. devs don’t have complete knowledge of problem
                      2. get this knowledge as they start implementing
                      3. the very beginning of a project is when you know the least:
                         why try to make all the decisions then?
       3. time passes, some work gets done, know more;
              • perhaps add and/or remove tasks
              • are we slipping? get new estimates
       4. these new estimates are likely just as bad as the first and will give a just as
          bad estimated completion date.


• the psychology of why Dev estimates are bad, read Steve McConnell’s books on
  the topic (“Code Complete”)
       • developers typically give the shortest time period they can defend. -
         correct
       • “short and long estimates average out” – WRONG! Clearly: we always ship
         late.




                                                                                             18
                    What Scrum isn’t
•   a design methodology
•   a programming methodology
•   a quality control methodology
•   a documentation methodology

     – you are free to use whatever you like to address
       these concerns. Scrum is meant to coexist with
       existing practices.

                                                                                19




Scrum is not a development methodology.


3. Scrum is a “management device”
      •   wraps eXtreme Programming, evolutionary/iterative programming, even
          Waterfall, etc.




                                                                                     19
     Defined vs. Empirical Processes
• Defined – think “assembly line”
     – well-known
     – predictable
     – repeatable
• Empirical
     – chaotic
     – unpredictable
     – non-repeatable
• Frequent Sampling and Adjustments
                                                                                     20




(from control theory)


3. defined process
       • when you get the same result every time.
       • like, the next car down the line takes the same amount of time as all the
         rest
       • same inputs yield same outputs == “defined process”
4. empirical process
       1. Here’s a quote from Schwaber’s web site: “
              • We assert that the systems development process is empirical:
              • Applicable first principles are not present;
              • The process is only beginning to be understood;
              • The process is complex;
              • The process is changing.”
       2. Two teams building the exact same software will not get the same results.


5. How do you control empirical processes? Frequent Sample and Adjustments: this
   is the gist of Scrum.




                                                                                          20
                              How does Scrum Work?
    • Product Backlog
         – all the items to be done in order to ship a product
    • Product Owner
         – the guy who priorities the Product Backlog
    • Sprint (or Iteration)
         – 30 days (or less)
    • Sprint Backlog
         – all the items to be completed in a Sprint
         – each item has an estimated amount of work remaining
    • Sprint Demo
         – meeting at end of the Sprint, when working software is demoed
    • Scrum Team
         – 7 +/- 2 devs and testers
    • Daily Scrum Meeting
         – daily 15 minute stand-up meeting of the Scrum team
    • Scrum Master
         – the guy who conducts the Scrum Meeting
                                                                                                                                                        21




(Keep Available!)

3.    anybody can add items to the Product Backlog
             1.    requirements, use cases, features, bugs, requests
             2.    only the Backlog Owner can prioritize the list: someone in authority (i.e., PUM level)
             3.    everybody realizes that not everything on the project backlog list will get done
                            1.    if you think your pet item must make it before shipping, it’s your responsibility to convince the backlog owner of this
                            2.    low priority items will likely be cut
                                         1.     don’t need specs for lowest priority items
4.    the Product Owner represents the Voice of the Customer (Six Sigma terminology).
             •      I like to think of him as the guy who pays the salaries –
             •      He’s ultimately responsible for the project.
5.    Sprint == 30 days (no more, no less: hard date)
             1.    output from each Sprint is potentially shippable binaries
                            1.    as bug-free, fully tested, documented, etc. as possible.
             2.    Shorter iterations are better: 2-3 weeks, 1 week == eXtremeProgramming
•     the Scrum team members pick how many of the top priority items they think they can accomplish in a sprint.
             1.    this becomes the Sprint Backlog
             •     the Team is utterly self-managing
                            1.    nobody can add or remove anything from the Spring Backlog, except the Team
             3.    Management has to leave the Scrum team alone for the duration of the Sprint. (There is an early abort mechanism.)
             4.    it’s each team member’s responsibility to update his estimate of the work remaining on his current work item every day.
            5.      at the end of a Sprint is the “Sprint Demo”. Two purposes:
            6.       a hard drop-dead date that can’t be missed
                           •        something to focus the team (ex.: is this work item necessary for the demo?)
                           •        you can’t be 90% done and give a demo.
            7.       give all the stakeholders are view of what’s being built
                           1.      if the team’s going in the wrong direction, we find out after at most 1 iteration.
7.    the Scrum Team succeeds or fails as a team
            1.      there are skills, not roles
            2.      can switch roles as necessary
                           1.      ex: if a tester needs to write a tool, and it will take him a week to get up-to-speed, but would take me 1 hour to write
                                   it for him, switch jobs or pair program
8.    short daily “stand-up” meeting
            1.      the Scrum Master asks each team member 3 questions:
                           •       “what did you do since the last meeting?”
                           •       “what will you do before the next meeting?”
                           •       “what problems are getting in your way?”
            2.      no one but Scrum Team members can speak at Scrum meetings:
                           1.      “chicken and pigs”
                                           1.    you can tell the VP (i.e., a chicken) to shut up
9.    The Scrum Master’s responsibility is
            1.      to keep the Daily Scrum Meeting on track, and
            2.      to remove impediments




                                                                                                                                                              21
                         Scrum Artifacts

• Product Backlog List

• Release Burn-down Chart
• Sprint Burn-down Chart




                                                                                      22




       Artifact == think of archeology; stuff left over after the project is done.


1. the Product Owner’s responsibility to maintain a prioritized list of work remaining.
       1. priorities may change over time
              • market may change
              • competition may change
              • our thinking may change, likely will change (since we know more)
       2. new items may be added, items may become unimportant
       3. not a static list: “Responding to change over following a plan” – part of
          the Agile Manifesto.
              • it’s painful and stupid to follow a plan that’s wrong.


• Release Burn-down chart and Sprint chart look exactly the same, but at different
  scales:
       • Release chart is for the entire ship cycle (months or years)
       • Sprint burn-down chart covers 30 days.




                                                                                           22
                   Burn-Down Chart
                  900

                  800

                  700

Estimated         600


  Work            500


Remaining         400


    in            300


  Hours           200

                  100

                    0
                        1   3   5    7    9   11   13   15   17   19   21   23   25   27   29   31



                                    Elapsed Time in Days

                                                                                                 23




1. my first Sprint burn-down chart!
      • vertical axis is “Estimated Work Remaining, in hours”.
      • horizontal axis is “Elapsed Time in Days”.
      • week-ends are included in this particular graph, so some short level spots are
            expected

2. if the estimates had been perfect, this would have been a straight line, with slope of 1.0.
        1. this graph shows why bad estimates don’t matter:
                1. it’s the slope, the all-important slope, that matters: the amount of real
                    work done per unit of time.
                         1. if the estimates had all be off by a factor of 2, the slope would
                             have been ½.
                         2. from the slope you can extrapolate a completion date.

•   this, according to Schwaber, is a typical Sprint burn-down chart for a team that’s
    inexperienced with Scrum:
         1. note how the amount of work was bouncing around for the first third of the
             Spring:
                 1. work was getting done, but
                 2. as the team learned more about the problem space, more work items
                     needed to be added.
                         • that’s ok and expected.
                 3. after a little more than a week, the team learned enough so that more
                     items or increasing estimates were not necessary any longer.
         • so, at around day 21, panic started setting in, as it became obvious that all the
             work was not going to get done in time for the demo.
                 • important! Note that rather than just dropping features willy-nilly, we
                     knew exactly how much to drop.
         • now everybody on the team sees these charts and gets feedback on their
             estimates,
                 • and, in theory, gets better at estimating.




                                                                                                      23
              Alternative Sprint Chart




                                                                                24




Problem: on the previous chart, you can’t really tell how much work was
   discovered/added.


• Cumulative Flow Diagram – invented by David Anderson (works here at MS), of
  FDD (Feature-Driven Development)
• Idealized graph:
       • Started line is way too smooth, as is Completed line
5. One Big Advantage over Sprint Burn-down Chart:
       • You can tell when and where work was added (or removed)




                                                                                     24
       Release Burn-down Bar Chart




                                                                                     25




1.Invented by Mike Cohn, of “User Stories Applied” and “Agile Estimating and Planning
2.Again shows new work added (or removed) by PMs, stakeholders, etc.
3.Assumes there is a cut-off line, below which we won’t do any of the requests
       •That cut-off line moves up or down, representing work that we won’t do for
       this product cycle
4.To Extrapolate End Date:
       •Draw a horizontal line from the bottom of the last bar, extending to the right,
       and
       •Fit a line to the tops of the bar charts, representing the teams velocity
       •Where they intersect is your extrapolated end date.


•“Evaporates == you’re done” - Tony




                                                                                          25
                            Planning a Sprint
• Sprint planning session part 1
      – PBO explains top highest priority items from the
        Product Backlog List
• Sprint planning session part 2
      – Team estimates how many of those items it can
        accomplish in this Sprint
      – Turns those items into work items
              • This becomes the Sprint Backlog


                                                                                                        26




1. Part 1:
         •     it’s ultimately the Product Owner’s responsibility to determine what constitutes
               the next Sprint
         •     but, it’s the Team’s responsibility to determine how much it will attempt to do
               during the next Sprint
         •     Part 1 is about selecting what to do for the next Sprint
                    • the Product Owner explains the top few highest priority items, in
                         enough detail so that...
                    • ...the team can select those it feels it can turn into working software in
                         one month.

•   Part 2:
         1.    the Product Owner must be available to answer questions that the team might have.
                    • no one other than team members is allowed to do anything but observe or answer
                        questions.
         2.    the output from the second part of the Sprint Planning Session is the Sprint Backlog
                    1. tasks
                    2. task estimates
                    3. initial task assignments
                    4. the list might not be complete, but complete enough so that the team feels
                        confident that it can produce demo-able software in 30 days.
                             • “potentially shippable”!!!
         3.    How to do this:
                    • Put User-Stories (or Use Cases, or Requirements) on Product Backlog List
                    • You must provide some business value to the user during each iteration!
                    • Splitting user-stories into iteration-sized pieces is a skill: read “Agile Estimating
                        and Planning”, by Mike Cohn
                    • Then at the Sprint Planning Session, the team takes the highest priority user-
                        stories (that it feels it can commit to), and breaks them down into tasks.
                             1. Getting the Product Backlog right is the hard part; generating the Spring
                                  Backlog should be pretty easy.
         4.    Sprint
                    1. the Team works undisturbed for 30 days




                                                                                                              26
                         Daily Scrum Meeting
• 15 minutes max.
• all Team members must attend
• Scrum Master asks each member 3 questions:
       – what have you done since the last Scrum meeting?
       – what will you do before the next Scrum meeting?
       – what impedes you from performing your work as effectively
         as possible?
       – (optional) did you add or remove any items from the Sprint
         Backlog list?
       – (optional) did you learn anything that may be of interest to
         the team?
                                                                                                           27




1.   Agile Manifesto says “Individuals and interactions over processes and tools”
         1.   the process must be as light-weight as possible
         2.   the Scrum process has two areas of overhead
                  •    the Daily Scrum Meeting – stand-up meeting!
                           •    the meeting must be kept under 15 minutes:
                           •    the Scrum Master is responsible for this.
                  2.   and updating the estimate of work remaining
                           1.   I’ve been using a Excel spreadsheet to track the work remaining and
                                generate the Sprint charts
                           2.   it takes just a minute to update your item
2.   3+2 questions: what have you done, what will you do, and what’s getting in your way?
         •    the meeting is for the benefit of the team members. Not a status meeting!
         •    when a new guy shows up, he invariably talks directly to me (the Scrum Master) when giving
              his answers.
                  1.   per the book, I purposely avoid eye-contact while someone’s answering the three
                       questions.
                           1.   that forces them to talk to each other, rather than to me.
                           2.   it reinforces the idea that they’re in charge and responsible for the Sprint.
         3.   should only answer the 3 (or 5) questions
                  •    shouldn’t digress into issues, designs, discussion of problems
                  •    if someone needs help or if something needs to be discussed, anyone can set up a
                       meeting for all interested parties immediately after the Daily Scrum Meeting.
         4.   I’ve informally asked some team members if they remember what someone said the
              previous day:
                  •    they said they did, if their part or component was dependent on another’s work.
         5.   Optional questions from Craig Larman, “Agile & Iterative Development”
         6.   ScrumMaster’s job – remove impediments, immediately if possible
                  1.   Do whatever it takes to keep distractions from the team, keep them working at full
                       capacity




                                                                                                                27
            Sprint Review: Demo Meeting

• Output from a Sprint is working software
      – demonstrated at the Sprint Review meeting
• All stakeholders give input
      – “Responding to change over following a plan”


• Scalability – “Scrum of Scrums”



                                                                                               28




1. you can’t demo software that 90% done. It should be
        1. complete in the sense that it’s:
                1. potentially shippable
                2. tested
                3. debugged and as bug-free as possible
                4. documented, etc.: whatever your development process requires.
2. the purpose of the Sprint Review meeting is for the Team to present to the Product Owner
   and stakeholders what was done during the Sprint.
        1. attendance is mandatory!
        2. starts with a review of the Product Backlog committed to, and what Product Backlog
           was completed.
        3. team members present their demos.
        4. at the end of the presentation, stakeholders are polled, to get
                1. their impressions,
                2. any desired changes and their priorities
                3. stakeholders are free to voice any concerns, observations, or criticisms.
                4. stakeholders can identify and add new functionality to the Product Backlog.
                5. re “Responding to change over following a plan” – Agile Manifesto
                        1. whenever I hear the phrase “Plan of Record” I get annoyed.
3. the Product Owner reprioritizes the Product Backlog, and schedules another Sprint Planning
   Session (usually for the next day).
        •   the Cycle repeats.
4. I’ve done weekly “SoS” meetings twice – questions are a little different:
        • “Is the project on tract”? – show Sprint burndown chart
        •   “any impediments?”
        •   “anything added to Sprint Backlog (or Product Backlog)?”
        •   “anything we should know about?”




                                                                                                    28
                             Why Scrum Works
• face-to-face accountability
• team camaraderie
• hyper-productivity
       – functional roles are discouraged
       – collocation is encouraged
• empower the team



                                                                                                             29




1.   every day, you stand in front of your teammates and explain what you did (or didn’t do)
         •    if a team member isn’t pulling his weight, it becomes evident immediately.
         •    you don’t want to let your teammates down
         •    can be used to track low performing team members
2.   the team succeeds or fails as a unit
         •    the team makes all the decisions and is responsible, as a unit
         •    engenders camaraderie
3.   During Scrum meetings, a pig might say “I’ve been working on XYZ for two days, and I can’t get it to
     work right” in the presence of another dev who’s done this three time already and is willing to help
     out. This is where hyper-productivity comes from.
         1.   note that these don’t work:
                  •    sending status by email
                  •    having one guy go around collecting status
                  •    having a weekly status meeting
         2.   Schwaber says having everyone in one big room works well
                  1.   with workstations on wheels, the team members reorganize daily, or as needed
                  2.   with cubicles, he’d see heads popping up like prairie dogs, synergistically talking
                       with one another.
                            1.   Ex. “should this parameter be a const ptr?”
         3.   “Radical collocation can double the productivity of software development teams”, by Judith
              Olsen
•    trust the team: they’re closest to the problem and know best (Lean Development idea).
         1.   the team is authorized to do whatever it takes to meet the Spring Goal:
                  •    tools, training
                  •    bring in an expert, consultant
                  •    have a morale event
         2.   in my experience, it’s really hard for leads and mgrs to give up control
                  1.   I’ve seen the dev mgr
                            1.   argue with a team member over his estimates;
                            2.   assigning work, rather than letting the team divvy it up.




                                                                                                                  29
        Agile: Guidance/Advice
• Care about project management!
  – If you don’t, someone else will
  – and they may not know what they’re doing

• Read up on this:
  – Agile Development
  – Lean Development
  – eXtreme Programming

                                               30




                                                    30
                 Design Patterns –
              Beyond the Gang of Four
• Gang of Four’s “Design Patterns” book, 1995
      – 23 patterns
• “Pattern Languages of Program Design (1-5)”
      – Authors include:
             • Ward Cunningham
             • James Coplien
             • Robert Martin
      – The first two volumes were terrible

                                                                                        31




I’m assuming Design Patterns are taught, but I’m also assuming that it’s the original
     GoF patterns.


3.   Seminal work
         1.   Provides good solutions to common problems
         2.   Provides a common language to talk about common constructs
         3.   contains 23 patterns, such as
                    •   Class factory
                    •   Singleton
                    •   Proxy
         4.   For a long time, I thought this was the only book, then I discovered….


•    PLoPD series
         •    Lots more patterns
         •    There are conferences!
         •    The 5th one is edited by Dragos Manolescu (who works at MS)




                                                                                             31
                      Some Comments
• http://en.wikipedia.org/wiki/Category:Software_design
  for a list, probably not complete
• Some patterns now dated; we know more now
     – Visitor has a well-known cyclic dependency
     – Singleton causes temporal coupling if mutable


• Survey – what’s your favorite pattern?


                                                                     32




Let’s do a survey:
2.what’s your favorite design pattern? Or
3.what design pattern do you use the most often?


(Expectation: ~10-20 percent of people mention a non-GoF pattern.)




                                                                          32
                    My Favorite Pattern:
                        Null Object
Problem: Pointers/references to polymorphic
  objects may be null, and you’re often checking for
  null

void Snippet(Customer* pCustomer)
{
  Plan * plan;
  if (pCustomer == null) plan = BillingPlan::getBasicPlan();
  else plan = pCustomer->getPlan();
   // …                                                                               33

}
 My favorite pattern: Null Object


 3.Error handling is often a large percentage of the code-base.
 4.Anecdotes
         1.    I’ve heard people say that after a while error checking/handling takes up
               most of the code
         2.    Using the Null Object pattern is one way to keep error handling to a
               minimum, and
                   1.   So can shrink the code-base significantly.




                                                                                           33
             Null Object – Class Diagram




                                                                                       34




“Introduce Null Object” – I first saw this in Fowler’s “Refactoring” book.


3.You create a do-nothing object, and never need to check for null-ness again.
4.This requires that you have an interface
         •    If you don’t, extract one. “extract interface” – also from Refactoring
              book
         •    Interfaces decrease coupling (a little), so this isn’t a bad idea.




                                                                                            34
                      Null Object - Example
  class IProgressObserver
  {
  public:
     virtual void OnProgress(unsigned char) = 0; // 0 - 255: 255 == done
     virtual void Finished(HRESULT) = 0;
  };
   
  class NullProgressObserver : private IProgressObserver
  {
      virtual void OnProgress(unsigned char) {}
      virtual void Finished(HRESULT) {}
  public:
      static IProgressObserver& GetNullProgressObserver()
      {
            static NullProgressObserver npo;
            return npo;
      }
  };
                                                                                      35




Real code from a project I’m working on right now.
Perfect example: an observer, as sometimes you don’t have one, want one, need one
(we install in “silent” mode)



5.Performance: people always ask about performance
        1.   We’re trading clarity for performance
                 1.   or is this even true?
                          1.   Remove jnz & call, replace with call via vtable
        2.   Probably worth it, unless you have profiler data telling you otherwise
                 1.   “Premature optimization is the root of all evil” - Knuth




                                                                                           35
 Design Patterns – Guidance/Advice
• Keep learning
     – The industry is always moving forward
          • We’ve learned things since 1995 (GoF)
          • Ditto C++: some STL concepts are now considered anti-
            patterns
     – Read pattern books
     – Take pattern courses
     – Attend conferences


                                                                         36




“Read” – coincidentally, I am reading “xUnit Test Patterns” right now.


From Dragos’s talk:
       “Take pattern courses
       Get involved
              Talk to pattern experts
              Provide known uses
              Participate in/organize pattern mining workshops
       Join the expert community
              Write patterns
              Organize pattern events”




                                                                              36
     How to Write Maintainable Code
• “Big Ball of Mud” – www.laputan.org/mud
   – Experience
   – Skill
   – Time
   – Cost
   – Complexity
   – Change
• Peter Principle of Programming
   – “we build the most complex thing that can be made to
     work”
                                                                                                                                  37




“Big Ball of Mud” == Spaghetti, Shantytown
2.    Why do we write "Big Ball of Mud" code all the time?
              1.    Experience - really scary - don't know what good code looks like
                           •      "fresh out of school inexperience“
                           •      "domain inexperience“
              2.    Skill
                           •      Fred Brooks (I think) said (in "Mythical Man-Month") that the difference in productivity between
                                  best and worst coder was 20x.
                           •      For architectures, (I think) it's much much higher: almost /no-one/ knows how to write
                                  maintainable code
              3.    Time pressure
                           •      "getting the architecture right frequently takes a back seat to more mundane concerns like cost,
                                  time-to-market, etc.“
              4.    Cost - doing architecture right is expensive, especially when you don't know the domain yet
                           •      "getting the system right seems like a pointless luxury once the system is limping well enough to
                                  ship“
                           •      "an investment in architecture usually does not pay off immediately“
                           •      "design debt" - managers want more features, not fixing up existing architecture, until it gets so
                                  bad that it's impossible to work with (more on this below)
              5.    Complexity
                           •      we write really big program nowadays - Windows - 64 mloc, VS 44 mloc.
                           •      "Conway's Law" - "4 teams working on a compiler, => you'll get a 4-pass compiler“
              6.    Change
                           1.     the original architecture will likely not be able to handle arbitrary new feature requests;
                           2.     either re-architect the whole thing, or
                           3.     contort the code to fit. (See above: we do the latter).
                           4.     if no regression tests, can't respond to change (i.e., refactor) safely.

3.    “the Peter Principle of Programming”: PP == “rising to our level of incompetence”. Why?
            •      Because of the above (lack of experience, skill; time/cost pressure; etc.) we make the locally safest change
                   possible
                           •      i.e., a band-aid
            2.     After time, there are so many band-aids (i.e., it’s so complex) that you can’t see the architecture any more
                           •      “mental model” – more on this later
            3.     Eventually, it gets so bad, that making a change or bugfix over here causes 3 bugs over there
                           •      And fixing them causes another cascade of bugs
            4.     i.e., our rate of forward progress has slowed dramatically
            5.     And we stay like that forever, from that point on.
            6.     v1 is easy, but
                           1.     We often get the “dead core” within 3-5 years (Ken Schwaber’s term)
                           2.     and it’s often the most important code!




                                                                                                                                       37
                                Experience
• What does bad (unmaintainable) code look like?
• Bad code is a class that:
     – has hundreds of data members;
     – has hundreds of methods, most public;
     – implements lots of interfaces;
     – is coupled to lots of other classes;
     – uses friend classes.



                                                                                           38




1.   anecdote:
         1.   I spent a lot of my career at Microsoft in the Windows org.
                  1.   Very old code, written a long time ago, by kernel / driver guys, in
                       C
                  2.   When my boss and I moved to the Developer Division, I thought
                       to myself:
                           1.   “oh good, we’ll be right next to the smart guys, the C++
                                compiler team, and all the code will be nice and clean”
                           2.   Didn’t happen
         2.   In Visual Studio Core, my job was to bring TDD (and Scrum) to the guys
              there
                  1.   Their code was just as bad as Windows’. Example of bad code
                       above. Breaks every one of my rules.
                  2.   I’d go to each team, take a hunk of their code, get it in my test
                       harness and write some tests.
                           1.   I’d almost always find a few bugs.


•    What really worried me, though, was that the new kids probably thought that this
     was the right way to write code.
         •    We’re Microsoft, right? I’ll just add another friend, another piece of
              data, and another method to this (already huge) class


•    Recognizing bad code is a start. Will Rogers: “Good judgment comes from
     experience, and a lot of that comes from bad judgment”




                                                                                                38
     What does Good Code Look Like?
 • “loose coupling, tight cohesion”
    – But what does this look like?

 • One class, one responsibility
 • Data members:
    – either minimal (0, 1, 2), or
    – all data changes at the same frequency
 • Be faithful to the Mental Model of the architecture
 • A class should have few public methods,
 • implement few interfaces
 • Be minimally coupled, no cyclic coupling
                                                                                                                                   39




Ok, so every book I’ve ever read on OO, says “loose coupling, tight cohesion”. I thought I was doing that (but I wasn’t)
       Need some metrics

4.    One class, one responsibility – (everybody says this, too).
            1.     I’m really hard-core about this.
                            •     Ask “what does this class do?” and look for the word “and”, e.g., ( this is the metric)
                            •     This class does this and that – wrong!
                            •     It’s better to have lots of small, hierarchical classes than one big one.
5.    Data members: metric
            1.     Either minimal (i.e., just a couple), or
            2.     They all change at the same frequency. Why?
                            •     It keeps your class from being able to do too much:
                            •     If your class does this and that, it probably needs this data and that data.
                            •     Sometimes there’s data that stays together as a group. This is ok, if
                                          1.     They all change together, or
                                          2.     The class has no methods other than ones that delegate to its data members’
                                                 methods.
                                                        1.     I call this a “containing” class – it only contains its delegates
                                                        2.     the other kind I call a “functional” class – it actually does something
6.    Metaphor a.k.a. “Mental Fidelity” a.k.a. The Model (From Ron Jeffries:)
            •      Develop a common vision of how the system works – XP calls this the “metaphor”.
            •      It’s a simple evocative description of how the program works
            •      Sometimes its not obvious, but in any case,
            •      Use a common system of names so that
            •      Everyone understands where to look to find the functionality they’re looking for
            •      I can’t stress enough how important this is!
                            1.    Relate anecdote about HotIron’s MetaData/XML classes and how everyone wants to throw their
                                  functionality in there.

7.    Few public methods:
           1.     If a class is responsible for doing only one thing, does it really need lots of public methods?
                          •       Probably not. (STL’s basic_string from 1995 had about 100 public methods, and is not considered
                                  a mistake)
           2.     Ditto few interfaces

9.    On to “coupling”, next slide:




                                                                                                                                         39
                   Complexity: Coupled Code
• A class A depends on (or is coupled to) another class B if
      –   it inherits from B
      –   it contains a B
      –   it has a method that returns a B
      –   it has a method that takes a B
      –   it has a method that uses a B (e.g., an automatic variable)
      –   it does any of the above with a template (or generic) with
          parameter-type B




                                                                                                                                         40




 Here’s the definition of coupling I use.

 •“Uncoupled code” == A knows nothing about B, B knows nothing about A. Great – unrelated classes.
 •“Decoupled code” == one-way coupling only: A depends on B, B knows nothing about A.
 •“Coupled code”, or “Cyclic coupling”
             •     A depends on B, and B depends on A, in a cycle, or
             •     A depends on B, depends on C, depends on D, which depends back on A.

 6.There are other measures of coupling (HK measure, afferent/efferent, etc.)
              •     They try to tell you how coupled you are, but
              •     That’s like being “a little bit pregnant”: with my definition, your code is either cyclically coupled or it’s not.

 7.Claim: “cyclic coupling” is really bad. Why?
              1.     "Anyone wanting to understand any class in the cycle effectively has to understand every class in the cycle"
              2.     "Anyone wanting to test any class, effectively has to test every class in the cycle"
              3.     "Anyone wanting to lift a class for reuse in another system, ends up having to lift every class in the cycle"
              4.     That doesn’t sound too bad if the cycles are relatively short, but how bad is it in the wild?
                            1.      Anecdote:
                                           1.   VSCore’s MSBuild team wrote an assembly in C#, using TDD.
                                           2.   I wrote a tool to use reflection and count up the number of cycles in their assembly:
                                                20.
                                           3.   Now, another team .Net for CE (Mobile Edition) wrote the same assembly, w/o
                                                using TDD: 200.
                                                       •      Guess where the bugs were.
                                           •    Conclusion – TDD helps you write decoupled code (for free!), and this makes sense,
                                                       1.     since TDDers will immediately test their classes, and
                                                       2.     they don’t want to instantiate a whole bunch of classes just to test one.

 9.More on TDD coming up, but first, try this:
             1.   take a usual-sized existing class from your code base, and try to instantiate it in a test harness.
             2.   You'll have lots & lots of "unresolved externals“ (that usually live in other .libs)
             3.   Adding more .libs just makes matters worse (because of coupling).

              5.     This is instructive – it gives you an idea of how coupled your code is
                             1.     When I approach a legacy code-base, it usually takes me between 4 hours and 2 days just to get
                                    the class under test into my test harness (i.e., the simplest test: construct the object and see that
                                    it doesn’t throw an exception)
                             2.     4 hours – typically bad
                             3.     2 days – my worst experience
                             4.     3 days – friend of mine on the Windows kernel team (Object Manager)




                                                                                                                                              40
      Complexity: Temporal Coupling
• Avoid globals, statics and singletons

• “C++ Coding Standards”, Sutter & Alexandrescu:
     – “the correctness of a piece of code that uses shared
       data is conditioned on the history of changes to the
       data”




                                                                                      41




1.   Quote:
         "These [globals and singletons] complicate program logic and cause tighter
             coupling between different (and, worse, distant) parts of the program.
             Shared data weakens unit testing because the correctness of a piece of
             code that uses shared data is conditioned on the history of changes to
             the data, and further conditions the functioning of acres of yet-unknown
             code that subsequently uses the data further.“


3.   That pretty much says it all. I never use globals, statics or singletons.


5.   Ok, careful readers will have seen that I used one in the Null Object pattern. But.
         •    It’s a do-nothing object with no state: no data. So it’s ok.
         •    Also immutable objects are ok, too.


•    So what I really mean is: Avoid mutable globals, statics and singletons.




                                                                                           41
 Change: Test-Driven Development
• This is the most important one!
• TDD generates a large suite of tests that you
  can use as Regression Tests
      – You can refactor safely at any time
      – What to refactor?
• Large list at http://c2.com/cgi/wiki?CodeSmell



                                                                                                     42




2.If you do TDD, you’ll generate a large suite of tests
          •      Use as regression tests when you refactor


3.Why do I say it’s the most important?
          •      Because your code will deteriorate, and refactoring reduces “design debt”


4.You refactor
          1.     when you see a problem or,
          2.     when you realize you’ve been going in the wrong direction, or
          3.     when you get a better idea
          4.     when your requirements change and your current architecture can’t handle it
          5.     when you “smell” a problem
                        1.   “code smells” – XP term
                                 1.   I love this term – you can tell something’s rotten in there,
                                      but you can’t quite see it


•some Code Smells (from Fowler’s Refactoring book):
          •      Duplicated code
          •      Long method
          •      Large class
          •      Long parameter list
          •      etc.
          •      Comments! Don’t use comments as a deodorant!




                                                                                                          42
    How to Write Maintainable Code:
           Guidance/Advice
•   Use TDD
•   No cyclic-coupled code
•   No temporal coupling
•   Maintain mental fidelity
•   Read
     – Sutter’s & Alexandrescu’s “C++ Coding Standards”



                                                                                  43




MSR talk on “Is Linux Maintainable?” – Stephen Schach of Vanderbilt University (you
can watch it @ http://www.researchchannel.org/prog/displayevent.aspx?rID=4520).
Results:
       Code size of Linux is growing linearly with version number, while
       Coupling is increasing exponentially.




                                                                                       43
     “Working Effectively with Legacy
                 Code”
• Read book of same title by Michael Feathers
      – “Legacy Code” definition:
      – Code that doesn’t have unit tests
• You have a “Big Ball of Mud” - now what?
      – Run Away
      – Swamp Guide
      – Sweep It Under the Rug, a.k.a. Quarantine
      – Reconstruction, a.k.a. Total Rewrite

                                                                                              44




1.   WELC is highly recommended.
2.   Legacy Code == Code w/o UTs

4.   Say it’s too late: the code is already a mess. Now what? Here are the easy ways:
          1. Run away:
                       • some people (consultants, devs at very large companies, architects)
                          just move from project to project, and
                       • just leave their messes behind them. Selfish bastards.
          2. Swamp Guide: this is the complete opposite of “run away”:
                       1. Become the expert – easiest if you wrote it in the first place.
                               1. You’re the only guy who can understand it, no one else even
                                     comes close
                               2. Job security!
          3. Quarantine: say the code mostly works, but you’re afraid to make any changes
                to it.
                       1. isolate it – (think Chernobyl: big concrete bunker)
                       2. if you need to add more functionality,
                               • start new classes, in a new .dll, (<- you need this kind of
                                     isolation!!)
                               • that possibly makes calls into the old quarantined code.
                               • At least the new code will be clean and refactorable.
                       3. To do this, find areas of code that seem less tightly coupled and draw
                          architectural boundaries there.
                               1. Enforce communication with quarantined code via well-
                                     defined communication
          4. Reconstruction: it’s hopeless, you have to rewrite, no matter what the cost
                       1. find the worst code (maybe all of it) and rewrite it.
                       2. Do TDD




                                                                                                   44
          Legacy Code, Age & File Size
• MS Research investigated where the bugs were in
  Win2K:
      – Large old files didn’t have many bugs
      – New files with simple classes didn’t have many bugs
      – Medium-sized files had most churn and bugs

• Conclusion:
      – There’s value in old mature code.
      – Don’t throw it away without a good reason.

• Question: How do you avoid creating those pesky
  middle-sized files?                                                                45




1. Old mature files had already had most of the bugs fixed in them.
2. New small files were so simple they didn’t have many bugs either.


4. Life cycle of a file:
        • Starts off clean and simple
        • Features and changes are added, usually without much thought about re-
          architecting
        • Bugs are fixed, usually in the simplest, locally safest way possible
        • Eventually most or all bugs have been removed, but the code is complex
          and brittle.


Question: How do you avoid going from small simple files to those bad middle-sized
   files?




                                                                                          45
                  Keep it Working:
              Refactoring Legacy Code
• Chicken & Egg problem:
     – You need tests to refactor safely
     – it’s not easy to write tests, afterwards.
          • Legacy code is not particularly testable
• You need to refactor legacy code just to make
  it more testable!

• Solution: Apply “Pretty Safe” Refactorings
                                                                                       46




1. Chicken & Egg Problem:
       • Writing tests afterwards is hard, since the code is not particularly
         factorable/testable
       • If you could refactor the code to make it more testable, then you could
         more easily write tests
2.
3. “Pretty Safe” refactorings (will define on next slide), just to bring the legacy code
   (“system under test”) into your test harness
       1. Note: it often takes about 4 hours to get my first test to link:
               1. Dependency problems – often must write many (throwing) stubs
               2. Coupling problems
               3. Globals/Singletons
               4. Unresolved externals from many .libs.
               5. (will talk about these later, too)




                                                                                            46
                      “Pretty Safe”
                 Refactoring Techniques
• General Concepts:
     – “Preserve Signature”
     – “Lean on the Compiler”


• Specific Techniques:
     – “Extract Interface”
     – “Virtualize Method”
     – “Subclass And Override” a.k.a., “Test Sub-Class”

                                                                                    47




Taken from Michael Feathers’s “Working Effectively with Legacy Code”

• “Pretty Safe” == refactorings that you’re unlikely to screw up:
     1. Ones where you can use “cut-n-paste”,
     2. Where you can “preserve the signature”,
     3. Where the compiler tells you if you screw up.

• Note: if there is a good, safe, high-quality refactoring tool, using it speeds things
  up immensely.
      1. What is a safe refactoring tool?
             1. Example: if you “extract method”, does the tool check that the new
                 method’s name does not collide with a base class’s method?
                      1. If not, you may inadvertently override a method, and break
                         code.

5. “Extract Interface” – necessary for creating mock objects
6. “Virtualize Method” – adding pure virtual methods to newly extracted interface
7. “Subclass and Modify” – to extract bad, slow, or inconvenient functionality from a
   method.

9. NOTE: some of these refactorings don’t make the code simpler or cleaner:
     • But they do get the code into your test harness.
     • After writing more tests, and refactoring to make the code more testable,
         you may no longer need the initial refactorings, and can revert them.




                                                                                          47
              How to Mock an Object:
                “Extract Interface”
• Interfaces are critical for Mock Objects
• Interfaces change less than classes
• “Preserve Signature” applies to class and
  interface definitions, too.




                                                                                       48




“Extract Interface” – from Martin Fowler’s book “Refactoring – Improving the Design
   of Existing Code”

Say you need to mock up an object so you can pass it to your System-under-Test. How
   do you proceed?

• Mocking an object requires the use of interfaces (mostly: another option is to use
  “link-seams” or “link-time polymorphism”. Doesn’t scale.)
      • legacy code often uses lots of COM, still.

7. Extracting an interface == converts a class into the class plus an abstract base class
       • Often used in combination w/ “Virtualize Method”
       • One use is to add a layer of abstraction to multiply-inherited interfaces
           (see example, next slide)

9. Is it ok to extract interfaces all over the place?
        1. Yes: interfaces change more slowly than do classes and are a good design
            technique, as it decreases coupling.
        2. Performance: some or many methods become virtual. Unless you have
            data showing otherwise, this is probably ok.
                • Knuth: “Premature optimization is the root of all evil.”

10. “Pretty Safe” refactoring:
       1. Can cut-n-paste new interface definition from existing class definition.




                                                                                            48
             How to Mock an Object:
               Virtualize Method
• Copy-n-paste method signature from class
  into its newly-extracted interface
• Add “virtual” keyword; make pure virtual
• “Lean on the Compiler”

     – Let the compiler find the (virtual) methods that
       belong in the interface

                                                                                      49




1. “Virtualize Method” – to help create mockable interfaces.


3. Say you have a non-virtual method on a class that you need to mock.
4. After extracting an interface, cut-n-paste the method’s signature from the class
   into its newly-created interface.
5. Then add the virtual keyword in front (make it pure virtual).


6. The actual class doesn’t change.
       • Typos will be caught by the compiler.
       • “Cannot instantiate abstract class”
       • Let the compiler find all the (virtual) methods that belong in the interface
              1. Say the system-under-test has a method that used to take a class
                 ptr as parameter, now takes an interface ptr
              2. If the new interface is missing members, you again get compiler
                 errors, and need move more methods into the interface and
                 virtualize them.


7. Can work incrementally.




                                                                                           49
                Subclass and Override
• Problem: a method in the system under test
  contains a call to a “bad” API:

     – Too slow, hoses registry, sends email, etc.
     – You don’t want to call this “bad” API every time
       you run the tests




                                                                    50




Technique is called “Subclass and Override”


(In TDD, the tests must run very quickly, or you won’t run them.)
       CallToAReallySlowFunction or
              DepositMoney, or
              SendEMail, or
              FormatDrive, or
              MuckWithRegistrySoBadItllReallyHosesYourMachine, or
              LaunchNukes, etc.


Solution on next slide.…




                                                                         50
                 Subclass and Override
Before:
class Foo {
public:
   void Bar() { CallToAReallySlowFunction(); }
};
After:
class Foo {
public:
   void Bar() { CallToAReallySlowFunction(); }
   virtual void CallToAReallySlowFunction()
   { ::CallToAReallySlowFunction(); }
};
class TestFoo : public Foo {
   virtual void CallToAReallySlowFunction() { }
};
                                                                                         51




Solution:


• Step 1: make slow API a virtual method on the class,
       • With exactly the same signature (“PreserveSignature”)
       • Implement virtual method by making call to API


• Step 2: derive TestFoo from Foo
       1. Implement virtual method in safe, fast, better way (i.e., empty)


• Step 3: test TestFoo class in harness rather than Foo.
       1. Has exactly the same code as Foo, except for bad API
       2. Can also do fault-injection, if API returns HRESULT or bool, or something.


This refactoring doesn’t make the code clearer or better: it allows you to write tests
    for the Bar method.




                                                                                              51
                       Friend Classes
                   Accessing Private Data

• “Extract Interface” (no data members)
• For each directly accessed data member:
    – Compile the system under test:
         • “X is not a member of” interface
    – Add public accessor to interface
         • Search-and-replace friend accesses w/ accessor



                                                                                     52




Friend classes are the tightest form of coupling


3. Friend class should be used as a last resort and actively discouraged.
4. in one code base I had to write 44 accessors instead of direct friend accesses.
       1. How to fix:
               1. Extract interface (but don’t move data members!);
               2. Compile:
                        • “Lean on the compiler”
                        • compiler complains that data member doesn’t exist;
               • add accessor;
               • Search-and-replace direct accesses with accessor




                                                                                          52
                     Link Dependencies
• Linking many .libs into one .dll
     – Can’t test .lib separately: unresolved externals
• Short-term Solution:
     – Link in .objs, not .libs
     – Write many small (mostly throwing) stubs
• Long-term Solution:
     – Convert all dependencies between .libs to
       Directed Acyclic Graphs (DAGs)

                                                                                   53




Many/most teams had huge link dependency issues.

3. .lib dependency problems don’t show up because we link up a monolithic .dll and
   everything is resolved.

5. But you can’t test each .lib separately;
      • You can’t even link in a .lib (without linking in all of them)
      • In one instance, I had to write 125 throwing stubs.

6. .Libs are inherently non-hierarchical (Cycles are just too easy to create).
       • Dlls can be hierarchical:
             1. Should be ordered in direction of greater stability:
                     1. Most stable dlls are lowest level leaf nodes
                     2. Highest level nodes churn the most
                     3. “A library should depend only on libraries that are more
                         stable that it is” – Robert Martin, “The C++ Report”
       2. Header files (w/o forward references) can also be hierarchical
             • A la STL: my personal preference, but
             • putting all code into headers may cause slow build times
       3. TDDing .libs will help:
             1. Natural feedback to put code in correct lib
             2. Won’t be able to put code into wrong .lib if you need to write a test
                for this code
                     • (you’d have to start writing a bunch of stubs: indicator that
                         the code is in the wrong place)




                                                                                        53
“Working Effectively with Legacy Code”:
               Advice/Guidance
•   Don’t write brand-new legacy code!
•   Refactor unrelengtingly
•   Write a TDD-style test for each new bug
•   Quarantine working code
•   Rewrite worst code




                                              54




                                                   54

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:70
posted:8/11/2011
language:English
pages:108
Description: Scrum Burn Down Template document sample