Agile Software Development for an Agile Force

Document Sample
Agile Software Development for an Agile Force Powered By Docstoc
					              Software Engineering Technology


                 Agile Software Development for an Agile Force
                                                                                                                                  John S. Willison
                                                                                                         U.S. Army CECOM Software Engineering Center
               I remember in a meeting I attended, an Army general arguing the need for the software community to fall in-line, saying that
               “Software itself has never killed anyone.” Maybe, maybe not, but I have seen software kill many Army programs and careers.
               As the Army transforms itself into a more agile force, the Army software community continues to struggle with the challenge
               of effectively providing software to support that force. This article identifies some components of an effective approach to soft-
               ware development and provides an example that is leading the way.

I  t is necessary to provide a characteriza-
   tion of the current U.S. Army business
environment to set the context for the rec-
                                                             platforms, competition to provide soft-
                                                             ware is expanding and the barrier to enter
                                                             the competition is low. Users have access
                                                                                                                 interactions over processes and
                                                                                                                 tools, working software over com-
                                                                                                                 prehensive documentation, cus-
ommended components for good busi-                           to a wide range of sources, and more                tomer collaboration over contract
ness. Historically, the Army acquisition and                 importantly, a wide range of sources have           negotiation, and responding to
development processes have been driven                       access to users. Increasingly, initial and          change over following a plan.
by the attempt to institutionalize success                   incremental capabilities can be provided
and avoid failure. The Army management                       to users as software-only releases, and in      Historically, the Department of Defense
and acquisition processes are based pri-                     some cases simply can be downloaded             and the Army have emphasized process.
marily on hardware models that, in turn,                     over the network.                               To produce a more agile force, the Army
are based on the value-added discipline of                       Finally, there is this question: how        needs a software community that has
risk management.                                             much alike or different should the Army         process discipline, but is more agile as
     With hardware, it is critical to mitigate               software community be from the commer-          well.
risk and get it right the first time, particu-               cial software industry? Clearly there are
larly prior to entering any stage that                       some differences. Most notably, the Army        Get as Close to the User as Possible
involves significant expense such as pro-                    software community’s priority is capability     The only one who truly knows what the
duction. The Army has evolved into using                     and readiness whereas the commercial            user wants or needs is the user himself.
a rigid approach where requirements are                      software industry’s priority is profit. Those   The closer you get to the user, the closer
defined and then used as the basis for                       different priorities have historically been     you will get to developing software that he
development, testing, and determining suc-                   used to rationalize the need for unique and     or she will accept and adopt; it is never too
cess. Further, as development and hard-                      rigid approaches to software.                   early to do this.
ware sustainment are different activities,
the Army has defined different processes                     Components for Good Business                    Show Them, Ask Them, and
and funding strategies for these distinct                    The Army develops, integrates, and              Repeat Often
activities.                                                  employs as wide a range of software-based       The Army is very good at generating
     With software, the processes and                        capabilities as any other organization.         requirements, and generating endless
investment strategies are different than                     While no single method for improving the        cycles of life-cycle events aimed at meet-
hardware; the risks are also different, and                  Army’s approach to software development         ing those requirements. Instead of asking
yet we attempt to manage them the same                       would suffice, there are some common            users what they need and then getting back
way. Software sustainment to a large degree                  components for improvement.                     to them only after developing the solution,
is simply doing more development; howev-                                                                     the Army should be prepared to show
er, development and sustainment are often                    Balance Between Plan-Driven and                 users what they could get up-front. If
managed by different organizations and                       Agile Development                               nothing else, this builds the users’ confi-
funded differently. The risks associated                     The October 2002 edition of                     dence that the Army is able to deliver
with software are different as well; and yet,                CrossTalk [1] did an excellent job of           something. The focus should be on early
we attempt to manage them the same as we                     contrasting the plan-driven [2, 3] and agile    and continuous software delivery.
work through a sequential series of mile-                    development approaches to software, and
stones. The real risks with software are in                  the spectrum between these two perceived        Architecture, Architecture,
taking too long before giving the user                       extremes. There is much to be gained            Architecture
something that knowingly will evolve over                    from both approaches.                           Architect Frank Lloyd Wright is believed
time, and in measuring success as meeting                        The Software Engineering Institute’s        to have said that no matter what you are
predefined requirements as opposed to                        Capability Maturity Model®, for example,        building, always remember:
getting the user something he or she wants                   has done much to address software as an
and likes.                                                   engineering discipline and the need for a           It will take longer than you plan, it
     There are other factors that influence                  plan-driven approach. Agile development,            will cost more than you figured,
the way the Army acquires and develops                       as characterized by the Agile Alliance [4],         and it will be messier than you
software. For the increasing percentage of                   finds:                                              could have ever have anticipated.
Army capabilities that are hosted on com-                                                                        But remember the most important
mercial off-the-shelf (COTS) hardware                           … more value in individuals and                  thing is not what is visible. What’s

16 CROSSTALK The Journal of   Defense Software Engineering                                                                                          April 2004
                                                                                                                                            Agile Software Development for an Agile Force

    most important is the foundation.
                                                      Product Development
    Bad software products do not neces-
sarily have bad software architectures.
However, good software products are like-                                                                    Release - 3 Months
ly to have good software architectures.                                                                                    Installation Configuartion
                                                                                                                         Management Version of Code
While the Army, and everyone else for that            Mission                      Adapt and
                                                                                                              Deliver
                                                      Analysis                       Clarify                                                      Iteration - 4 Weeks
matter, has increased the amount of atten-                                                                        PdM, TSM, and
                                                                                                                    Developer
tion and discussion surrounding software                                                                             Decision
                                                                                                                                                                        Development
architectures, the understanding and prac-                                                                                                         User Stories        Team Decision

tices associated with software architectures                                                                  Release          Release
are still not sufficient.                                                 Mission Goals                       Planning        Objectives         -Use Flow
                                                                                                                                                  Diagrams
                                                                                                                                                                 Demo
                                                                                                        -Req'ts Analysis




                                                          Mission Goals
    Senior Army leaders echoed the need                                                                 -Schedule and                                   Design
                                                                                                                                                   -Code Structure
for the Army’s development approach to                                                                   Resource Review
                                                                                                                                                   -User Interface
be more agile and responsive at a recent




                                                                                                 Adapt and
                                                                                                   Clarify




                                                                                                                                                                         Adapt and Clarify
Association of the United States Army
Symposium [6]. Lt. Gen. John Caldwell,                   Architecture
                                                    -Requirements Review                                                            -White Board Diagrams
                                                                                                                                    -Documents1
military deputy to the assistant secretary of       -Arch. Development and
                                                     Assesment                                                                      -User Interface
the Army (ASA) for Acquisition, Logistics,                                                                   Tech Eval                               Construct
                                                                                                        -Research
                                                                                                                                                -Code
and Technology (ALT) said, “If you wait                                                                 -Evaluation
                                                                                                        -Prototyping                            -Product Integration
to put a perfect capability in the field, you                                                                                                   -Unit Test




                                                                                                                                                                         Identify Bugs and Fix
will never put anything in the field,” and,                                                    Architecture

“People are the key to our business.”                                                                                                      -Built Code
    Col. Bruce Jette, director of the recent-                                                                                              -List of Changes

                                                                                                                                                       System
ly established Rapid Equipping Force, said                                                                                                           Integration
that the goal is to go from “idea to equip-                                                                                                     -Build
                                                                                                                                                -System Test
ping” in two to three months and that,
“field commanders are tremendously
accepting” of this approach.
    Col. Nick Justice, director Future          Note: 1Documents are created for critical, high-risk, unclear, complicated tasks, and external interfaces.
Force ASA ALT said, “The best way to            Figure 1: The Defined Process
find out how to engineer solutions is to get
out with the guy who uses them.” These          gained widespread acceptance within the                                            the defined process.
principles are captured in the components       Army command-and-control user com-                                                      One of several key aspects of the
for good business.                              munity. Representative of the success of                                           process is the notion of iterations and
                                                the product are comments made by Lt.                                               releases. The MCS Light project has adopt-
Maneuver Control System                         Gen. John Vines, previously the comman-                                            ed a four-week iteration and a three-month
Light Program                                   der of the 82nd Coalition Task Force                                               release cycle. Within an iteration, a team
The feasibility and benefits of applying the    (CTF82) in Afghanistan and currently the                                           may cycle through the design, construct,
components for good business can best be        commander of the 18th Airborne Corps,                                              integrate, and demo steps several times.
illustrated by way of example. The Army’s       who wrote:                                                                         This can be done within a team and also
Maneuver Control System (MCS) program                                                                                              across teams within the project. It is also
is part of the Army Battle Command                  MCS Light is the best tool available                                           important to note the level at which the
System and provides the commander with              today … recommend the Army                                                     definition of what is in a release and what
the capability to plan and monitor the bat-         adopt CTF82’s employment of                                                    is in an iteration is managed. At the release
tle. The MCS program is managed by the              MCS-Light as its strategy to rapidly                                           level, agreement is reached with the PdM.
product manager (PdM) MCS, under the                deploy a standard, interoperable,                                              Definition and modification at the itera-
program manager Ground Combat                       digital command-and-control sys-                                               tion level is managed at the project-leader
Command and Control and Program                     tem Army-wide. [7]                                                             level. Again, this provides for the flexibili-
Executive Officer Command, Control,                                                                                                ty needed to effectively manage in a very
Communications – Tactical. Development              MCS Light has become the planning                                              dynamic environment
is led by the Air Mobility Command              tool of choice for nine out of 10 active
Communications-Electronics Command              Army divisions. Much can be learned from                                           Balance Between Plan-Driven and
(CECOM) Software Engineering Center             examining this success.                                                            Agile Development
and supported by Shonborn-Becker                    The MCS Light development process is                                           The MCS Light software process has
Systems Inc., L3 Ilex, Lockheed Martin,         the result of several years of direct experi-                                      struck a balance between agile develop-
CECOM Research Development and                  ence developing software in a very dynam-                                          ment and plan-driven development, or
Engineering Center, and others.                 ic environment. The process is well                                                planned agility in the following ways.
     MCS Light was born out of opportu-         defined and has been, in fact, in use for
nity and necessity. The MCS Light product       several years. And the process has resulted                                        Individuals and Interactions Over
implements command-and-control func-            in a high-quality product that has been                                            Processes and Tools
tionality on a PC/Notebook/Windows              widely accepted by the user community.                                             The MCS Light project and its broader
platform. The MCS Light product has             Figure 1 is a graphical representation of                                          organization have consistently placed sig-

April 2004                                                                                                                                                                                       www.stsc.hill.af.mil   17
Software Engineering Technology

nificant emphasis on individuals and have                    a change is necessary – is overly burden-     establishing a Beta Site concept.
backed up this emphasis with investment.                     some to any development effort looking        Leveraging industry practices, some oper-
Roughly half of the development team                         to rapidly respond to a customer’s needs.     ational units were identified as official
are government civilian employees, the                       The project has adopted the equivalent of     Beta Sites. As a Beta Site, the units were
other half are contractors working on-site                   a level-of-effort agreement with the PdM.     provided with developmental releases of
as an integral part of the team. Software                    Within this approach, it can measure          software. The premise was simple: the
developers represent more than 85 per-                       progress at the standard milestones and       team would provide incremental releases
cent of the project staff, and all civilian                  measure earned value.                         of software, the user would provide feed-
engineers have either completed or are                                                                     back, and the team would respond rapidly
pursuing advanced degrees in software                        Responding to Change Over Following           where possible with another incremental
engineering.                                                 a Plan                                        release.
    Consistent with agile development                        The Army as an institution is well versed          Instead of having to wait years for a
approaches, the overall development team                     in the development of plans. Fortunately,     new version of software that would likely
is comprised of smaller teams. These                         the Army also recognizes that no plan,        not satisfy their needs, users were rapidly
teams typically consist of three to 10 indi-                 even the best plan, survives long in a        and frequently given developmental
viduals who are co-located within the                        dynamic environment before needing to         releases of software that, incrementally,
same office. Interaction is informal, con-                   be revised. Planning for software develop-    met more and more of their needs.
stant, and essential to the approach.                        ment is not significantly different than      Confidence and trust between the devel-
                                                             planning for a battle. The MCS Light          opers and the users were formed. With
Working Software Over                                        effort has consistently placed an emphasis    trust comes the need for less bureaucracy,
Comprehensive Documentation                                  on responding to change. This emphasis        thereby enabling the streamlining of the
The MCS Light project has placed consid-                     gives the team the flexibility to respond     approach even more. Since its inception,
erable emphasis on the software product                      effectively to the constant evolving and      every active Army division has come on-
and has considered extensive documenta-                      changing user needs.                          line and requested to become an MCS
tion as a significant distraction from devel-                                                              Light Beta Site.
oping the end product (more than 800,000                                                                        The benefits of this approach cannot
source lines of code). Therefore, it con-
cludes that developing such documenta-
                                                                “The MCS Light effort                      be overstated. Through this approach, it
                                                                                                           is worth noting that Army units have
tion represents an even greater risk than it                   has consistently placed                     demonstrated a willingness to accept
is intended to avert.                                                                                      good enough software much sooner over
     The architecture is extensively docu-                         an emphasis on                          the promise of better quality software
mented and that documentation is main-                                                                     much later. If they do not like the prod-
tained. In addition, an “MCS Light For                         responding to change.”                      uct delivered, or the product delivered
Dummies Guide” has been developed as a                                                                     does not work, the user has no problem
training guide for users. Additional train-                                                                saying so.
ing documentation has been and will be                       Get as Close to the User as Possible               The best case is that the team rapidly
developed to an even-greater degree as                       On the MCS Light project, the team has        responded to the user’s need and got valu-
fielding of the MCS Light product pro-                       been accused in the past of listening too     able feedback as to what else was needed.
gresses. There is also documentation that                    much to the user and the surrogate user,      The worst case is that the team learned
traces planned and delivered functionality                   Training and Doctrine Command System          what the user did not want or need, and
back to the system Operational Require-                      Manager (TSM), as opposed to strictly         only lost the time invested since the previ-
ments Document.                                              adhering to requirements definitions and      ous release. In that respect, the Army is
     The product itself, as opposed to                       programmatic structures. Doing so has         no different than commercial industry –
extensive documentation, has served as                       served the project well. As stated earlier,   time to market or time to field is a priori-
the basis for interactions between the user                  as a developer the closer you are to the      ty, and only an agile approach will do.
and the development team. Relatively                         user, the more likely you will develop
speaking, little documentation has been                      something useful. Simply put, that means      Architecture, Architecture,
developed on the MCS Light project, and                      having software developers and end users      Architecture
no one has missed what has not been                          working side by side.                         It would not have been enough to simply
developed, including those paying the                            On MCS Light, the project leader, all     be close to the user and provide early and
bills.                                                       team leaders, and a significant number of     frequent development releases. The prod-
                                                             project engineers have spent a significant    uct also needed to be sound and evolv-
Customer Collaboration Over                                  amount of time in the field with users.       able. From the onset of the project, archi-
Contract Negotiation                                         MCS Light software engineers have             tecture definition and evolution has been
A heavy emphasis has always been placed                      worked side by side with users in garrison,   a cornerstone of the development effort.
on collaborating with the user. For the                      at war-fighting exercises, and have even      Software architecture was defined almost
MCS Light development team, there is                         deployed with units to Afghanistan and        from day one, and a well-defined architec-
also another customer: the PdM MCS.                          Iraq. Being that close is harder than not,    ture has been kept up to date and have
Interactions with the PdM are frequent                       but it is the only way to develop a useful    served as the basis for all development
and less formal than the requirements-                       product.                                      efforts.
based contracting approach so often                                                                            Also key to the success of the project
implemented within the Army. The over-                       Show Them, Ask Them, and                      has been a well-structured architecture. In
head associated with detailed contract                       Repeat Often                                  the case of MCS Light, a three-tier archi-
negotiation – and renegotiation every time                   Key to the MCS Light success has been         tecture was defined and adopted. This

18 CROSSTALK The Journal of   Defense Software Engineering                                                                                     April 2004
                                                                                              Agile Software Development for an Agile Force

architecture has served the project well in       Talk 15.10 (Oct. 2002): 15-18              Operations.” Association of the
allowing developers to leverage COTS              <www.stsc.hill.af.mil/crosstalk/2002/      United States Army Acquisition
products and tools across the different           10/paulk.html>.                            Symposium, Falls Church, Va., 8 Sept.
tiers as well as in providing a powerful       3. Boehm, Barry. “Get Ready for Agile         2003.
approach to managing data. While every-           Methods, With Care.” IEEE               6. Vines, M.G. John. “Commander
one talks about how important architec-           Computer Jan. 2002.                        CTF82, Memorandum Thru Com-
tures are, MCS Light as a project has actu-    4. Agile Alliance. “Agile Software            mander CTF180 and Commander U.S.
ally implemented an architecture-based            Development Manifesto.” 13 Feb.            Central Command For U.S. Army
approach to development, and the contin-          2001 <www.agilealliance.org>.              Deputy Chief of Staff for Plans and
ued evolution of the product is the best       5. U.S. Army. “Transforming Current           Operations.” 15 Jan. 2003.
testimony to that case.
                                                                           About the Author
Summary
Insanity has been defined as doing the                          John S. Willison is       the Army’s Distinguished Service
same thing over and over again and                              director of Advanced      Award, the Secretary of the Army
expecting different results. If the Army                        Battlespace Solutions     Award for Outstanding Achievement,
software community is to truly, that is                         for the U.S. Army Com-    the Federal Technology Leadership
truly, achieve gains in effectiveness and                       munications Electronics   Award, and the Federal 100 Award.
efficiencies, it must be willing to abandon                     Command (CECOM)           Willison has a Bachelor of Science in
those practices that have not served it
                                                Software Engineering Center, Fort         electrical engineering from Lafayette
well. The Army must be willing to adopt
practices that strike a balance between dis-    Monmouth, N.J. CECOM is responsi-         College and a Master of Science in
cipline and agility.◆                           ble for developing software architec-     software engineering from Monmouth
                                                tures and products for Communi-           University.
References                                      cations, Command, Control, Com-
1. U.S. Air Force Software Technology           puter, Intelligence, Electronic Warfare       CECOM Software
   Support Center. “Agile Software              and Sensors systems. Willison is expe-        Engineering Center
   Development.” CrossTalk 15.10                rienced in the application of software        ATTN: AMSEL-SE-AT
   (Oct. 2002) <www.stsc.hill.af.mil/           technology, software architecture, pro-       Fort Monmouth, NJ 07703
   crosstalk/2002/10/index.html>.               totyping, and management. He has              Phone: (732) 532-2342
2. Paulk, Mark. “Agile Methodologies            received numerous awards, including           E-mail: john.willison@us.army.mil
   and Process Discipline.” Cross-




April 2004                                                                                                            www.stsc.hill.af.mil   19