Document Sample
hi5 Powered By Docstoc
					        ~C:IS~A    Systems II1fOfmalion
                   Certified Auditor'
                                          Chapter 3-lnformation Systems A cquisition,
                                          D evelopment and Implementation                                                        Section Two : Content
                   ........ ~-

        Consistent with standard systems development meth odologies,                  • Pursuit of competitive adva nta ge-Most organizat ions have,
        stringent change cont rol procedures should be fo llowed since the               fo r many years, automated their baSic, high-vo lume acti vities.
        basic assumpt ions and fo rmulas may need to be changed as morc                 Significam organi zationwide IT investment stich as ERP
        expert ise is gained. As with other systems, access should be on a              systcms is now common place. Many companies have OJ' arc
        necd · to~kn ow    basis.                                                       now investing in Internet teclmo logy as a means of di stributing
                                                                                        product/service and supply chain integration. Howeve r)
        The IS auditor should be knowledgeable abo ut the va rious AI and               utiliza tion of IT to maintai n and extend a firm 's know ledge
        expe rt system applica tions lIsed within the orga ni zat ion.                  ca pital represents a new opportunity to use techno logy to ga in
                                                                                        an advant age over competitors.
        The IS auditor needs to be concerned with the cont ro ls relevant to          • Legalrcquiremellts- Legislatioll sllc h as the US Sarbanes-
        these systems when lIsed as an integra l parI cran orga nization's              Ox ley Ac t an d the US Patriot Act exist to enforce the need for
        business process or mission critical functions, :lnd the level of               co mpanies to have an understanding of the "whole of bus iness."
        experience or inte lligence lIsed as a basis for developing the                 Financial instifutions IllllSt now be able to report on al l accoUIitS/
        so ftware. ~hi s is cri tical si nce errors produced by AI systems may          instru ments that their customers have and all transaetions
        have a more severe impact than those produced by traditional                    aga ins t those accounts/instruments, including any suspicious.
        systems. This is true especially ofi ntclligent systems that                    transaction pattern s.
        facilitate hea lth care professionals in the di agnosis and treatment
        o f i'~lI ries and ill nesses. Error loops/routines shoul d be designed       To deliver effective 01, organizati OlfS need to design and
        into these systems.                                                           implement (progressive ly, in IllOst cases) a data architecture. A
        Specifically. the IS auditor should:                                          complete daHl archi tec ture consists of two components:
        • Understand th e pu rpose .and tl.lllct ionality of the system               • T he enterprise da ta now arc hitecture (EDFA)
        • Assess the sys tem's signiricance to the organ iza tion and re lnted        • A logicn l data architecture
          businesses processes as we ll as the assac iated pote ntia l risks
        • Rev iew the ad herence of til!! system ta co rporat e policies and          An optimi zed enterprise dhta now architecture is depic ted ill
 .•,      procedures'
        • Review the decision logic bui lt into the system to ensure that
                                                                                      exh ibil 3.1 6. Explanations of the va ri ous layt:rs/components of
                                                                                      Ih is data now architccture follow:
          the expert know ledge or intelligence in the system is sound t1nd           • Prese nta ti on/des ktop access layer- Th is is wherc end users
          accurate. The IS aud itor should ensure tl1 m the proper level of                                                                         nn
                                                                                         di rectly deal wi th information. Thi£ layer inc ludes H il iar
          ex pertise was lIsed in developing the basic assumptions and                   desktop tools such as Microsoft Access and Excel, direc t
,::t      formu las.                                                                     q'ueryi ng tools, repon ing and analysis suites offered by vendors
        • Review procedures fo r updating info rmati on ~ n the knowledge                such as Cognos and Business Objec ts, and possibly purpose-
          base                                                                           built applications such as balanced scoreca rds and,di gital
        • Review security access over the system, specificall y the                      dashboa rds. Power users wil l have the abil ity to build the ir qWI1 '
          knowledge base                                                                 queri es and reports while ather' use rs wi ll interact with the data
        • Review procedures to ensure that qua li fied resources are                     in predefined ways. Increasingly, users are bei ng provided ,vilh
          available for maintenancc and upgradjng                                      · more than static reporting ca pabi li ties by embell ishing reports
                                                                                         with parameterization and drill-down capabilities and presenting
        3.6.21 BUSINESS INTElLIGENCE                                                     da ta in visual formats as a supplement or replacement of
         Business intell igence (8 1 is a broad field of IT that encompasses
                                    )                                                    texllmlltabular data prese ntation.
         the collecti on and di sscm ination of info rmati0 11 to assist decision     • Data so urce layer- Enlerprise informat ion derives from a
        .making and assess orga ni zational performance.                                 Ilu mbe r of sources:
                                                                                        - Operational data- Data ca ptured and ma intained by an
         Inve.stments in BI technology can be app lied to enhance                           organiza tion's,existing systems, and' usually held in system-
         understanding ofa wide range Q business qilestions. Some
                                                f                                           specific databases or possi bl y nat files                    .
         typical'areas in whic h BI is app,1 ied for measure ment and analysis          - E;< D.ata provided to an organization by
       . purposes incl ude:                                            .                    extern al.sources. This could inc lude data-such as customer ,         ".
         • Process cost,'efficiency and qua1ity                                             delllographics and Ill:lrket share information.
         • Customer sati sfhction with product and sei'vice olTcrlngs                   ~ NOllopcratlonal data- Informa tion needed by end use rs that is
         • Custonier profitab ility, including d~terll1in ati o n of whi ch                 not current ly maintained in ,~ co mputer-access ible fa rma t .
         · ~lttribules i1 i'e useful predictors 'of tustolllcr profitabili ty         • Core dat a warehouse-This is· where all the data (or a't leas t
         • Staff and uni t achievement of key pe rforma nce                   the majority) of interest ta nn organiza tion is captured and
           indicators                      .                                            organ ized to assist'feporting and ana lysis. OWs are normally
         • Risk managemen t- e.g., by identi fyi ng ullusua l transaction               instituted as large relationa l databases. While there is not
           patterns and accumulat ion of incident and loss statistics                   unani mous agreement, many pu ndi ts suggest the wa rehouse

                                                                                        should hold fully normalized data to give it the nexibili ty to
        The interest in Bl as a distinct fie ld of IT ac tivity is being splll' red     deal wi th complex ancl changing busi ness structu res. A properly
       ·by a number or factors:                                                         ct;)J1slillltecl OW should support three basics forms of inqui ry:
        • T he increasillg size and C IllIJ)cxit y of 1lI 0dCI'II
                                         O                                                 Drilling up and dri lling dow n- Using dimcnsions of interest
          orga ni zations- The result is tha t eve n flllldamcntal business                to the busi ness. it should be possible. to Clggregate data (e.g. ,

          q uest ion~. canJ1ot be properly answered wit ho ut establishing ,
          se rious BI capability.
        CISA Review M anual 2011
                                                                                           sum store sales to get region sales and uhimatc ly nati onal
                                                                                           sales) as well as dri ll down (e.g .. break store: sales Gown ta

Section Two: Content
                                                          Chapter 3-lnformation Systems Acquisition,
                                                                     Development and Implementation                     ~C:IS~A    Cerlirled Information
                                                                                                                                   Systems Auditor'
                                                                                                                                   .. -..c-......

                                                  Exhibit 3,1 &-Data Flow Architecture Sample

                                                             Presentation/Desktop Access Layer

                                                      -         A~

                                                                                                                 l>       0
                                                                                                                 0.       ~.
                                              ~                 Data Mart Layer                                  :I:
                                                                                                                 0        ~
                                              •                                                                 "         0
                                                                                                                0         c:
                                                                ,~                                              c:        ~
                                                                                                                 ~.       m'

                                              :            Data Feed/Data Mining
                         :;;                  ·                Indexing Layer
               "2-       iil
               .,;'               S                             A•
               ~         '"
                         c:       ~
               0'        en       0.
               s         s       1ir
               en         '"
                                  ro                             Data Warehouse Layer
                                 '0           ~

                          ro      0
               >0        ro
                                  ~                                                                         ~
                         r        r
               '0        '"
                ,                                   Data Staging and Quality Layer
                                              •                       Data Access Layer

                                                                             Data Source Layer
                                              •           Nonoperatiooal Data, E         ata,
                                                                                xternal D Operational Data

                                                  Nonoperational              External Data             Operational Data
                                                     .Data                     Providers                  Providers

  counter sales). Attrib'utes a~ailable at the more granul ar levels          analysis would be to report monthly store sales and then
  of the warehouse can also be used to refine the ana lysis (e.g.,            repeat the analys is usi ng only customers who we re preex isting
  ana lyze sales by product).                                                 at the start of the yea r in order to separate the effect of new
- Drill across- Use conunO attri butes to access a cross·section
                                ll                                            customers from the ability to generate repeat business wi th
  ofinfofmation in the warehouse such as SlI lll sales across all
  prod uc t lines by customer and groups of customers accorcii!1g
                                                                              existing customers.
                                                                          • Data mart layer- Data marts represent subsets of infonU       alion            •
  to length of association with the company (and/or other
  att ribute of interest)
                                                                            from the core OW selec ted and organized to meet the needs
                                                                            of a particular business unii or business line. DaHl mart? Illay
- Historical analysis- The warehouse should stlpport th is by
  holdi ng hist9ri cal, tiAlc-variant data. A n example of historical
                                                                          . be relational databases or some fonn of online analytical
                                                                            processing (OLAP) data structure (also known as a data cube).
196                                                                                                       CISA Review Manl!al 7011
          ~C:IS~A   Sys!ems InlormaHoll
                    Certified Auditor'
                                          Chapter 3-lnformation Systems Acquisition,

                    . ,.....
                    ,                     Development and Implementation                                                          Section Two: Content

               OLAP tecimologies and some variants (e,g., relational OLAP               determination by business domain is that di flerent paris of large
               [ROLAPj), allow users to "slice and dice" data presented in tCllns       business organiza ti ons will often deal with differen t transaction
               of standardized measures (i.e., numerical facts) and dimens ions         sets. customers and products,
               (i.e., business hierarchies). Data marts have a si mplified stntcture
               compared to the normalized DW Ifusing a relational database,             Ultimately, the data architecnlre needs to be structured to
               a popular structure is the star schema which involves a fact             accommodate the needs of the organization in the most efficient
               table connected to denormalized dimension tables. A simplified           mztJUler, Factors to consider include the types of tmnsactions in
               structure assists the'business user's understanding or the data          which the organization engages, the enti tics that participate in

    ,          and relationships. Jr data are he ld in relational tables, less joins    or form part of these transactions (e,g" customers, products,
               (connections between tables) are necessary when quclying, and            staff and communication channels), and the dimensions
               the need to become fam iliar with a large I1lUnber of code va lues is    (hierarchies) that are important to the business (e,g., product
               avoided if ful l text descriptions are used instead.                     and organization hierarchies).
            • Data staging and quality layer- This layer is responsible
               for data copying, transformation into OW format and quality              With modern DWs. storage capacity is not really an iss ue.
               control. It is particularly important that only reliable data get       The refore the goal shou ld be to obtain the most granular or
               loaded to the core DW. This layer needs to be able to deal with          atomic data possible. The lowest leve l data are most likely to have
               problems periodically thrown up by operati onal systems such             attributes that can be used for analys is purposes Ihat \vould be Iosr-
......         as changes to account number formats·and reuse of old account            if summarized data are loaded.
               and customer numbers (when the,OW still holds information on
               the original enti ty).                                                   Va rious anal ysis models uscd by dma architects/ana lysts follow:
            • Data access layer- This layer operates to,connect the data storage        • Context diagrams- Outline the major processes of an
               and quality layer with data stores in the data source layer and, in        organ i7.ation and the ex ternal parties with which the business
               the process, avoiding the need to know exactly how these data               interacts.
               stores are organized. Technology now pemi.its SQL nccess to data         • Activit y or swim-lane dia gra T11s- Deconstruct business
              even if it is not stored in a relational database,                          processes.
            • Data prepanttion layer- This layer is concerned with the                 • Entity relationship dia grams- Depict data entit ies and how
               assembly and preparation of data for loading into data marts.              they relale. These data analys is .methods obviollsly play an
              The usual practi~e is to precalculate the va lues that are loaded           importan t part in deve loping. an e.nterprise data model. However,
               into OLAP data 'repositories to increase access speed, Specialist          it is also crucial that know ledgeable business operatives are
              data mining also no rma lly requires preparation of dat a. Data             involved in the process. This way proper understanding can be
               mini ng is concerned with exploring large vo lumes of data to              obtai lied of the business purpose and context of the data. This
              determine patterns and trends of information . Data m i nil~g oftcn         also mitigates the risk of tile replication of suboptimal data
               identifies patterns that are counterintuitive due to the number            con figurations from existing systems and databases into the DW.
              and complexity of da ta relationships. Data qua lity needs to be
               very high to not corru pt the results.                                  Business Intelligence Governance
            • Metadata repository laye r- Metadata are data about data.                To ma ximizc the value an organization obtai ns from its BI
              The information held in the metadata layer needs to extend               initiatives, an effec tive 81governance process needs to be in
              beyond data strucillre names and formats to provide deta il on           place,
              business purpose and context. The metadata layer should be
              comprehensive in scope, cove ring data as they flow between              An important part of the governance process involves
              .the various layers, including documenting transformation and            determining which BI initialives to fund, what priority to assign
               validation rules. Jdeally, information in the metadata layer c;l.n      to initiatives and how to measure their ROJ. This ,is particularly
              be di rectly sourced by software opcl?ting in the other layers,          impo rtant since the inyestinent needed to 'build 81infrastruchu'e,
              as required,                                                             stlch as a OW, is considerable, Additionally, the scope and                .
          :. \V~rehou se mana ge m ent, l aye r~Th~ function of th is layer js         co'mplexity orall organizatiomvide 'D\V 1l1eans tbai, realistically,
              the scheduling of the tasks necessary to bUIld and maintain the          it fn ust be bi!iit III stages:                . .        .,
               OW and populate data marts. This layer is also invo lved in the
              administration of security.                                            . A recommended practice in the' area ofBI funding gove rnance
          . ' Application messaging layer'-This layer is concerned with                is to establish a business/IT advisory tea nl that al.l ows differcllt .
               transporting information between the various layers. In addition        fli n c l iOJ~all? e rspectives to be rcpresentecL reconllnends ·inve.s.unent
         . to business data, thi s layer encompasses gcnel:ation. stol:age and priorities and establislles cross-organizational benefit measures.
              targeted communication of control messages.                              Final funding dec isions should rest wi th· a technology steering
            • lnterllet/intranet la ye r- This layer is concerned wit h basic          committee that comprises senior managemcnt.

              data communication. Included here are browser-based uscr
~l.            interfaces and TCP/IP nctworking.                                       A further important part of overall 8 1 governance is data
                                                                                       governance, Aspects to be 'considered here incltlde establishing
            The construction of the logical 'data architectln'e for an enterprise      ~tand ard definitions for data, business rules and metrics,
            is a major undertaking that would normally be undertaken                   identifying approved data sou rces. and establ ishing standards for

 •          in stages, Que reason for sep~rating lo~ical data model                    data reconciliation and balnncing.

           elSA Review Manual 2011                                                                                                                         197
  Section Two: Content
                                                            Chc,pter 3-lnfortnation Systems Acquisition,
                                                                        Development and Implementation                         ~C~IS~A   Cenifled Auditor'
                                                                                                                                         Systems IfIformaliOll

 3.6.22 DECISION SUPPORT SYSTEM                                              The decision-structure dimensio n is also broke n into three parts:
 A decision support system (OSS) is an interactive system that               I. Structured
 provides the user wit h easy access to decision models and data from        2. Semistructured
 a wide range of sources in order to support semistructured dccision-        3. Unstructured
 making tusks typically for business purposes. It is an in tonnationai
 application that is designed to assist an organizalion in maJcing           The degree to which a problem or decision is structured
 decisions through data provi ded by business intelligence tools (in         co rresponds roughly to the ex tent to wh ich it can be automated or
 contmst to an operational application which collects the dat.a in the       programmed.
 course ofnorlllnl business operations). Typical information that a
 decision support application might gather and presen t would be:            Another DSS framework is the Sprague-Carson framework
 • Comparative sales riglll'es between one week and the next                 that is initi nted with an eftbrt to create family trees- whi ch
 • ProJected revenue figures based 0 11 new product sales                    is a generalization of the structure of a DSS. This framework
   Clssumptions                                                              suggests th at every DSS has data, a model and n dialog generator
 • The consequences of different decision alternatives given past            subsystem. This framework emphasizes thc importance of data
   expericnce in the desc ri bed con text                                    management in DSS worl{. This framework also stresses the
                                                                             impo rtance of interactive user interfaces in a DSS. The genern lion
 A DSS may present informati on graphicall y and may include                 and management of these interfaces require appropri ate software
    expert system or art ificial intelligence. Further: it may be
                                                                             and hardwa re- the dia log management system. In general, <l
 aimcd al business exec utives or SOllle other grou p of knowledge                              er
                                                                             system must alT more than one interfnce and might need to
 workers.                                                                    provide a tai lored interface for each user.

 Characteristics of a DSS arC:                                               Design and Development
 • Aims at solvi ng less-structured, unclerspccificd problems thaI           Prototy ping is the most popular approach to OSS design and
   senior managers face                                                      dcv~l opment. Prototyping lIslil.ll ly bypasses the usual requiremen t
 • Combines til e usc of models or analytic techn iqlles \vith                                                                -
                                                                             defin ition. System requircments evolve throu!!h the user's learnillIJ
                                                                             process. The bcnefits of prototyping include the following:

   traditi onal data ncccss and rctrieval functions
 • Emphasizes flexibility and adaptability to accommodate changes            • Learni ng is explicitly incorporated in to the desion process
                                                                                                   .                               0

 · in the environment and the decisiolHllaking npproach of the users           been use of the iterativc nature of the system design.
                                                                             • Feedback from design iterations is rapid to maintain an effective
 Efficiency vs. Effectiveness                                                  learning process for the user.
 A pri nciple of DSS design is to conccnt rnte less on effi ciency           • The user 's expertise in the problem area helps the lIser suggest
 (i.c., performi ng tasks quickly and red·tlei ng cos ts)·and morc on          system improvements.                                          ...
 eftective ness (i.e., perforn;ing the right task). Therefore DSSs are       • The ini ti al prototype mllst be inexpensive to create.
 often developed usi ng 4GL 100ls that arc less efficien t. bu t allow
 for flexible and easi ly modified systems.                                 Implementation and Use
                                                                            It is diffi cult to implement a DSS because of its di sc retionary
  Decision Focus                                                            nanlrc. Using a DSS to solve a problem represents a change
  A DSS is often developed with a spccific decision or we ll-               in behavior 0 11 the part of the lIser. Imp lemcnting a DSS is
  defined class of decisions to solve; therefore, some cOlllmcrci<l1        an exercise in changing an organization's behavior. The main
  software packages that claim to be DSS arc nothing ll~ore than a          challenge is to get the users to accept the lise of software. The
  DSS generator (tqols with which to con st ru ~1 a DSS).                   following are the steps invo lvc.d in changing behavior:
                                                                            • Unfrcezing- Tbis step alters the forces act ing 011 individuals
  DSS Frameworks                                                               such that the individuals are cli.slracted sufficicn tly to change.
. FramcwOI:ks arc generalizations about aJie ld th at help put many·         · Unfreezing is accomplished either through increasing tIle
  specific cases nnd idcns into pcrsp·ccrive., The G. Go rry-M.S.           · pressure for change or by redu cing SO)llC of the tlucats of or
  Morton fiTllllcwork is the most complete know led.gc;- and system-           resistance to change.
  contro l-related IS model, fmd it is based a ll problem Classification    • !Vloving- This step presents a ·~irection ~fch angc and the
  into structured and ~lI1strucnl rcd lYRes as wc.l l as the time hQri?on     actua l process of learning new attinlcies .
  of the dccisions. This framewo rk characteri zes DSS activities         . • Rcfn czin g- This st~p 1     11tegrates tl~c changed attitudes into the
  illong two dimensions:                                                       indi vidua l's personality.                 .
   I. The degree of Slfllclure in lh~ dccis·ioll process bei ng support~d
  2. The ma nagement level at which decision making takes place             Risk Factors
                                                                            Developers should be prepared for eight ·implementation ri sk
  This rramework also portrays alliS eff0l1 s as add ressing distinct       facto rs:
  types ofproblcms, dC pc~ldin g on thc above two factors.                  I. Nonexisten t or unwilling users
                                                                            2. Multiple users or implcmcntcl:s                                                   •
  The manageme nt-level dimension is broken in to three parts:
   I. Operational cont ro l                              .
                                                                            3. Disappcnrin g users, implcmenters or ma inta iners
                                                                            4. Inabili ty to specify purpose or usage patterns in advnnce
  2. Management·control
  3. Strategic planning
                                                                            5. Inabil ity 10 predict and cllshion·impact on al l partics
                                                                            6. Lac k or loss of sllpport
 198                                                                                                           CISA Review Manual 2011
     ~C~IS~A    Certirled AucliIO(
                                      Chapter 3-lnformation Systems Acquisition,
                                      Development and Implem entation                                                    Section Two: Content
                .. 0I>t>'(":"_


     7. Lack of experience with simi lar systems                                 • Excellent graphic presentations
     8. Technical problems and cost-effectiveness issues                         • Dynam ic graphic, data editing
                                                                                 • Simulation
     Implementation Strategies
     To plan for the risks and prevent them from occurring:                      ass Trends
     • Divide the project into manageable pieces.                                DSS trellds include:
     • Keep the solution simple.                                                 • Gradual improvement and sharpening of ski lls in the
     • Develop a satisfactory support basco                                        development and implcmentation ofa traditional DSS
     o Meet uscr needs and institut ionali ze the system.                        • Advances in database and g r~l phics capabilities for
     Assessment and Evaluation                                                   • Exploratory wo rk in such fields as expert systems. a DSS to
     The true lest ora DSS lies in whether it improves a manager's                 support group d ~cision maki ng and visual interactive modeling
     decision maki ng, which is something not easily measured.
     A DSS also rarely results in cost displacements such as a                   3.S.23 CUSTOMER RELATIONSHIP MANAGEMENT
     reduction in slaff. or other expenses. In addition, because a DSS is        The emcrging cus tol11 er~d ri vc n business trend is to be focllsed
     evolutionary in nature, it lacks neatly defined completion d~lles.          on the wants and needs of the customers. With the customer's.
                                                                                 expectations constantly increasing, these objectives are becoming
     Usi ng an incremental approach 10 DSS development red uces                  more difficult to achieve. This emphasizes the il nportnnce of
     the Ilecd for evaluation. By deve lopi ng one step at a time and            focusing on information relating to tml1saction datH, prcferences,
     achicving tangible rcsults at the end cfeach step, the user docs            purchase patterns, status, contact histOlY, demographic information
     not need to make extensive cOlllmitmen ts of time and money at              and serv~ce trends of customers mthcr than all products.
     the beginning of the development process.
                                                                                  All these factors lead to customer relations hip manage mcnt
     The DSS designer and USC I' shou ld use broad.evaluati on cri leria.        .(CRM), whi c-h is an opti mum combination or stra tegy, tac ti cs,
     These criteria should include:                                               proccsses, skill sets ftnd techno logy. CRM has hecome a strategic
     • Tracli tional cost~bene rit analysis                                       slIccess factor for all types ofbusincss, and ·its proficiency has a
     • Procedural changes) more alterna ti ves cxaminecL less time                significant impact 0 11 profitability.
       consu med in making th e dec ision
     • Evidencc of improvement in decis ion mak ing '                             The customer expectations arc increasing tremendously which,
     • Changes in the dccision process                                            in turn, -raises the ex pectation of service levels. Therefore,
                                                                                  the customer-centered appl ications foclls o'n CRM processes
     Some COlllmon trends in DSS usage include:                                   emphasizing the customer, rather than marketing, sales or any
     • The need for more a~curate information by manage rs is an                . othcr function. The new busincss model will have an integration
       important motivator for DSS development.                                   of telephony, web and database techn ologics, and illterenterprise
     • Few DSS evaluations use the traditional cos l~be n efjt analysis.          integration capabilities. Also, this model spreads to the ol her
     • End users are usually the motivators In devcloping a bss.                  business partners who can share information. communi cate and
     • Development staff members are drawn largely from functional                collaborate wi th the organization with the seamless in tegration
       area staff or the plann ing depa rtment, not from the IS                   ofweb~e nabled applicat ions and without changing thei r local
       department.                                                                nctwork and other confi gurations.
     • Users perceive ncx ibility as th e most important factor
       innuencing the systcm'S success.                                          It is possible to distinguish between operati onal and analytical
     • Few DSS projects arc being developed today for thi rd-                    CRM. Opertitional CRM is concerncd with maximi zi ng the
       generation sb flwa ~e· faci li ties. User~oriented, fourth-generation    ,uti lity of the customer's service experiel~cc while also capturil1g
       languages and planning languages predominate.                             useful data abollt the clistomer interaction. Anal ytical CRM
     • Planning, evaluation and training.for DSS projects traditionally          seeks to ana~yze in fonnation captured by the organization about
       have bech performed quite poorly.                                         its eustoITlCrs and the ir interactions with the orgadit alioll 'into
                                                                                 infonnation that allows grea ter .value to be obtaill~d from the .
     DSS Common Charact eristics                                                 customer base. Among uses of analytical CRM arc increasing
     Some of the common characteristics of ~SSs include:                         cust~m~f prqduct ho ldings or "share of customer \yallet," moving
     • Orie'nted toward decis ion maklllg                                        c'listomcrs into highei-margi n products, 1110ving customers to
     • Usually based on4G L                                                      lower~cost'se~v i ce channels, increasing l1laJ'keting success rates.
     • Slilfable                                                                 and mak ing pricing decisions.
     • Linkable
     • Drill-down
     • Semaphores (signals to automatica lly alert when a decis ion
                                                                                 3.6.24 SUPPLY CHAIN MANAGEMENT
       needs to be made)                                                         Supply chain management (SC M) is about link ing the bllsine~s
     • Ti me series lliialysis                                                   proccsses between the r.elated entities such as the buyer and the

     • What if( l:cfers to scen'ari a modeling: i.e., determining '..vIHlt is    seller. The link is prov ided to all the connec ted areas slleh as .
!      the end result ofa'changing variable or variables)                        managing logistics and the cxc hange of information, services and

•    • Sensitivity analysis
     • Goal-seeki ng
     CISA Review MafluarZ011
                                                                                 goods among su pplier. conSllmer. wa rehotlse, wholesale/retail
                                                                                 distributors and the manufacturer of goods.

 Section Two: Content
                                                          Chapter 3-lnformation Systems Acqu!sition,
                                                                     Development and Implementation
                                                                                                                             e elSA
                                                                                                                                        Cerlirled InlO(maliOll
                                                                                                                                        Syslerns, A
                                                                                                                                                   e..~ _

  SCM has become a focal point and is seen as a new area in strategic       associated risks found in contemporary software development
  managemen t because of the shift in the business scenario at the          projects. Sources of complexity incl ude:
  advent of global competition, proliferat ion of the Intemet, the          - Range and scope of systems now being developed-
  instantaneous transmission of information and web presence in all           e.g., e-business, customer relat ionsh ip management (CRJ\II),
  spheres of business act ivities. SCM is all about managing the now          supply-chain integration, online transaction processing and
  of goods, services and infonnati6n among suppliers, manu facturers,         onl ine analytical processing
. wholesalers, distri butors, stores, consumers and end users.              - Rate ofbllsiness change, which il!creases requirements
                                                                              instabi lity
SCM shifts the focus si nce all the entities in the supply chain can        - Diverse architectural decisions that need to be made- e.g.)
work collaborati vely and in a real-time mode, thus reducing, to              whcther the system should have a graphical user interface
a great extent, the requircd ava ilable inventory. The just- in-time          (GUJ) or browser interface; \vhich fi"rewall to use; which web
(JIT) concept becomes possible, and the cycle ti me is reduced                server to use; whether to use an application server or other
with an objecti ve towa rd reducing the unneeded inven tol)'. .               integration/control middleware; which DBMS to use; which
Scasona((e.g., bot h availabili ty and demand) and regional (e.g.,            communication and data defini tion protocols to use; whether
preferences in size, shape, quantity, etc.) factors arc addressed.            to organi ze the system in terms of subsystems and mod ul es,
                                                                              classes and objects, or empl oy a component architecture; and
Stock levels of. nonmoving items are significantly reduced, and there         where should business logic reside?
is an automated now of supply and demand. Also, the intrinsic costs         - A fj'equent need to integra te new systems with legacy systems
and errors associated with the manual means such as fax, input of
data, delay and inaccurate orders, can bc avoided.                        \Vit hin the overall category of an iterati ve life cycle there are a
                                                                          number of variants. TIlesc include:
EDI, which is ex tensivcly llsed in SCM . is no\ a new technology         • Evolutionary developm ent- Protolyping is used to build 1\
but the cost associate.d wit h it was not affordable to lllallY llI\til      \vorki ng model that is us~d to clicit/verify requi rements il l)d
recent years. Now, <lvai lnbility of inexpellsi\'e, we b-based SCM           explore design issues. Eventually the prototype is hardened
solutions has opener! this area for eve rybody. However, there is a          from a security standpoilit so it can be implemented into
need for significant change in the business model.                           proci uciioll, or perhaps the system is recoded based on lea rn ing
                                                                             fro m the prototype.
                                                                          • Spiral de"elopment- A series of prototypes is lIsed to develop
3.7 ALTERNATIVE FORMS OF SOFTWARE                                           a sol ut ion to Ihe point of detai led design, build and test. The
                                                                            sol ution spirals out from the initial limited prototype to become
PROJECT ORGANIZATION                                                         progressively more expansive and detailed. Formal ri sk analysis
                                                                            p~'e ce de s each proto. ype and the various prototypes (dependin g
The following section describes diAe rent approaches to                     on the iteration reached) form the basis for producing
organizing a software project, depending on imperatives such as             software development products, incl uding the requ irement's
time tQdeli very, scale of the ~ys tem, clarity of requirements and
       .                                                                    spcci fi cat ion, system des ign and test plans.
maturi ty of the technology being used.                                   o Agile development- The projcct is broken down into relati ve ly
                                                                            short, timc-boxed iterations. From the earliest iterations the
An IS audi tor must understand that the basic steps outli ned in the        cmphas is is to produce actual worki ng functionality, although
tmditional approach exist to some extent on nearly all soft\vare            software re lease may not always coincide with completi on of
development projects. However, the sequencing of steps, the number          an improvement. In early steps, a tracer-bu lle ~ approach may
of times they will be repeated, their durat ion and the methods             be used in which a piece of functionality is developed in a
employed vary considerably from project .to project.. It is also true       manner consistent with the intcnded dcyelopment approach and
that the thinking on how best to organize a software.project has            iuchi teL:ture. The functionality extends from the user interface,
evolved over time as the complexity of systems has incre·ased.              tlnollgh intenn~di ate layers, to stored data and back, Dnce
                                                                          · confidence is obtail1~d in the intended architecture, the foclls
Giv~n lhe~c factor£, other approaches ml IS auditor n1~y                    in late:r iterations shi fts to del ive ring. the functional ity with tflC
cncounter 'i nclude:                                                        greatest return on investment.
o Incremental or progressive devclopmcnt- The system
  is \)ui.!t· in stages or re leascs rather than being delivered in its
 entirety in Olle implcl11entatiop. The separate re leases are often
                                                                          3.7.1 AGILE DEVELOPMENT
  still large undertakings that operate as disc rete sl1bproject~. The    The term "agile.development" refers to a fam ily of similar
  usual practi ce is to deliver the basic system architecture in the      development processes that espollse a nontraditional way of
  first releasc. Sllbsequent releases expa nd the system in terms of      developing comp lex systems. One of (he first agile processcs,
  functionality, range of users or usage locations.                       Serum (a rugby analogy): emerged in the early 1990s.
oUeratjvc development- This involves bui ldi ng the system
  in iterati ons, with feedback occurring after each iteration 10         Serum aims to move planning and directi ng tasks fi·om the project
  fac ili tate any necessary adjustment of project plans ancl software    manager to the team, leaving the project manager to work on
 development products. Iterative development is now commonly
  regarded as best practice. Iterative development is considered
                                                                          removing the obstacles to the team. achieving theirnbjectives.
                                                                          SCI1IJ1l is a project management approacb that f.ilS we ll with
  the most appropriate way of dea li ng 'with the' complexities and       other agi lc techniques. Olher agile processes have si nce emerged
200                                                                                                          elSA Review Manual 2011
     ~C~IS~A   Ceflifledlnformalion
               Systems, Auditor'
                                      Chapter 3-lnformation Systems Acquisition,
                                      Development and Implementation                                                               Section Two: Content
               ..    .........__

     snch as Extreme Progranuning (XP), Crystal, Adaptive Software                        Agile development does not ignore the concerns' of tradi tional
     Development, Feature Driven Development, and Dynamic Systems                         software developme nt but approaches them from a different
     Development Method. These processes are tenned "agile" because                       perspective:
     they are designed to nexibly hand le changes to the system being                     • Agile deve lopment on ly plans for the nex t iteration of
     developed or the project that is perfonning the development.                           development in detail , rather than piarmi ng subsequellt
                                                                                            development phases far alit in time.
     Agi le development processes have a number of common                                 • Agile development's adaptive approach to requirements does not
     characteristics:                                                                       emphasize managing a requirements oaseline.
     • The use of small, t i mc~boxed subprojects or iterations, as shown                 • Agi le development's foclis is to quickly prove an architecture
        in exhibit 3.1 7. In th is instance, each iteration forms the basis                 by building actll al ftlllct ionality vs. formally de fining, early on,
        for pla nning the next ite rati on.                                                 software and data architecture in increasingly more dela iled
     • Replanning the project at the end of each iteration (referred to                     models and descriptions.
        as a "sprint" in Scrum), i.nclud ing reprioritizi ng requirements,                • Agile development assumes limits to defect testing but attempts
      · iden tifying any newTequi remcnts and determ ining within wh ich                    to va lidate functions through a frequent-bu ild test cycle, and
        re lease delivered functio nality should be implemented                             correct problems in, the next subproject before too much time
     • Relat ively greater reliance. compared to traditional methods, on                    and cost are incurred.
        tacit knowledge- the kn owledge in people's heads- as opposed                     • Agile development does not emphasize defined and repeatable
        to external know ledge that is captured in project documentation                    processes but instead performs and adapts its development
     • A heavy in fluence·on mec han isms to effectively disseminate                        based on fi'equent inspections.
        taci t knowledge and promote teamwork. Therefore, teams
        are kept small in size, comprise both business and techn ical        3.7.2 PROTOTYPING
        represe ntatives. and are located physically toget her. Team         Prototyping, also known as heUl;slic or evolutionary development,
        mcetings to verbally discuss progress and issues occur daily, but is the process of creating a system through control led trial and eITor
        with strict time lim its.                                           'procedures to red uce the 'Ievel of nsks in developing the system.
     • At least some oflhe agile Ill.ethods stipula te pair-wise            That is, it enables the developer and customer to understand and
        programming (two perso ns code the same part of the systcm) as react to risks at each evolutionary level (usi ng prototyping as a
        a means of sharing knowledge and as a qualify check.                 risk reduction mechanism). It combines the best features of classic
     • A changc in the ro le or the project manager, from one primarily      SDLe by maintaining the systematic stepwise approach but
        concerned with plan ning the projc.l:cl, allocating tas ks and       incorporates it into an iterntivc framework that more realistically
        moni toring progress to that of a faci litator and advocate.         reflects the real world.
        Responsibility for planning and contro l is delegated to to thc
        team members.

                                                               Exhibil 3.17-0ala Flow Architecture Sample

                                                          Agile Development Framework

                                                                    Daily Meeting
                                                                    • Done since last meeting
                                                                    • Plan for today
.~                                                                  • Obstacles?
                                        Planning M  eeting
                                        • Review ~roduct ~acklog
                                       .• Estimate backtog .
                                       '. Commit to 30 days
                                                                       BacRlog Tasks
                                                                           expanded    .'.:j
                                                                                                .  ·
                                                                                                 'r.·:\    (   ~o da.
                                                                                   Oda_ ,
                                                                            ~>'iJ J_ ~s __ _/
                                        • Goal

           Product Backlog
           Prioritized features                                     Backlog
          desired by customer                                       Features assigned
                                                                    Estimated by team

     Source: Cutter Consortium, USA. © 2009. All rights reserved. Used by permission.

     CISA Review Manual 2011                                                                                                                               201
......... '-~-- '--' ..-.--- --....~ ........

        S e ction Two: Conten t
                                                                                         • '.M; ..,. .........

                                                                   Chapte r 3- lnformation Systems A cquisition;
                                                                                     D evelopment and Implem entation
                                                                                                                                  •• ,,'
                                                                                                                 -~. !'-...:.~..;;.        ,.,;-::::..~

                                                                                                                                                          , .. , .'    "   .... ,

                                                                                                                                                                               .Certifled Inl«malion
                                                                                                                                                                      elSA S"'~ ""'~.
                                                                                                                                                                                ..   """,   ...   ~   ...

       Moreover, prototyping reduces the lime to dcploy systems pl;lnarily            Althouoh the IS aud itor should be awa re of the ri sks associated
       by Llsi ng faster development tools, such as fourth-generati on     .                 "
                                                                                      with protOlyping, the IS auditor should also be aware that this
       tec hni ques, that allow a user to see a high-level view orthe work lllgs      mcthod of system deve lopment can provide the organ iza tion with
       of the proposed system within a short period of time. A typ ical .             significa nt time and cost savings.
       software deve lopment ellvironmcnt that supports fourth-gene ration
       tec hniques may incl ude some or all of the following tools: .                 3.7.3 RAPID APPLICATION DEVELOPMENT
       nOl1procedura l languages for database quelY, report generation. < !nta
                                                                                      RAD is a methodology th at enables organi za tions to develop
       manipulat ion, screen interaction and definition, high-level graplucs
                                                                                      strategically important systems qui ckly while reducing
       cn pability, and configuration management.
                                                                                      dcvelopment costs and mai ntai ning quali ty. This is achi eved by
                                                                                      using a seri cs of proven application development tech ni ques
       The initial emphasis during the deve lopment of the prototype is
                                                                                      wi thin a well-defi ned methodology. These techniques include the
       usuall y placed on thc reports and screens, which are the system
                                                                                      lise of:
       aspects 1110S1 used by the. encl Llsers. This all?\~s the end l~ ser to see    • 'Small, well-trained development teams
       a working model oftbe proposed systcm wlthll1 a short tllne.
                                                                                      • Evolutionary prototypes
       There arc two basic methods or approaches to prototyping:
                                                                                      • Integrated power tools that supporlmodeling, proto typing and
       I. l3uild·the mode l to create the design (Le., the mechanism for
                                                                                         componen t reusabi li ty
          defining requi rements). Then, bascd on that model, deve'lop
                                                                                      • A central rcpository
          th e system design with all the performance, qua lity and
                                                                                      • Intcrac tive requirements and design workshops
          main tenance fentu res needed.
                                                                                      • Rigid limits 0 11 dcvelopment time fra mes
       2. Graduallv build the actua l system that will operate in
          prod l1cti~n llsing a 4GL that has been determined to be
                                                                               RA D supports tht.: analysis, design, develop ment ancl
          appropriatc' (or fhe system peing built.
                                                                               implementmion of individual application sys tems. However.
                                                                               RAD docs l10t support the plannilig or analys is required to
        The problem wi th the first approach is tha t there can be
                                                                              define the informat ion needs Of the enterpri se as a whole or of
        considerable pressure to implement <I n early prototype. Often.
                                                                              a major business area of tile en terprise. RAO provides a meallS
         use rs observing a working tnodel cannOI understand why the cn rly
                                                                               for deve loping sys tems faster whil e redu cing cost and increasing
        prototype has to be rcfine(l further. The fact that the prototype
                                                                              qua lity. This is done by automating large portions of the SOLe,
        has to be expa nded to handle tra nsaction volumes, cliem-se rver
                                                                               imposi ng ri gid li mits on developmcnt time frames and reus ing
        network cOllnectivity, and backup and recovery proced ures,
                                                                            . ex isting component s. The RAO methodology has four maj or
        and provide for security, auditability and cont ro l is not often
        und ers t oo~.
                                                                               I. The concept definition stage defines the business functions and
                                                                                  dala subject areas tl~at Ihe systein will support and determines
        The second approach typically \varks with 'S mall app licatiops
                                                                                  the system scope.
        using 4GL tools. However, for larger efforts, it is necessary to
                                                                              2. The·functional design stage uses w~rkshop s to Il,lodel the
        develop a design strategy for the system even ifaAGL is used.
                                                                                  system"s data and processes and build' a work ing prototype of
        The use of 4GL techniques alone will cause the same difficulties
                                                                                  critical system components.
        (e.g., poor quality, poor maintainabilit y and low user acceptance)
                                                                              3. The development stage completes the construction of the
        cncountered when developing business appl ications using
                                                                                  physical database and application system, builds the l:ol1version
        convent ional approaches.
                                                                                  system, and develops user aids and deploy ment work plans.
                                                                              4. The deploy ment slage includes final-uscr testing and training,
        Another ove rall di sadvantage of prowlyping is Ihat it often
                                                                                  data conversion and the implementati.oll of the application
     . leads to fUllctions or exfras bei ng added to the system that arc
        110 t includG:d in the initial requ irements document. All major
        enhancements beyond the inllial requircmen ts docu ment shoul d.
                                                                              RAD -uses pn?tOlyp ~n g as its core development tool no matter
        bc ensure they me'c t the. str(\tegic nee~s of the    .
                                                                             whi en underlying tec hnology 1S lIsed: ln co ntrast, '
        orga nizatioll and are.cast-effective. OtherWise, the fi~a l system
                                                                              s·~flware developmc nt (OOSD) and 'data-ori ented system
        lilay be func ti onally rich but inefficien t.
                                                                              development (OOSD) use cominuOllsly -de\leIOI)ing mode ls but
                                                                              have a foclIs 011 con tcnt solution space (e.g. ) how to best addres~
        A pot~ntial ri'~k with protOlyped systems is that the finished
                                                                              the problcm to inake the code reusable anCllllft intc nable) 'and can
      . sys tem will have poor controls:·By focusing .mainly 0 11 what
                                                                              be ftpplied using a tradi tional waterfall approac h. It should ~Iso
        the user wan ts and whal the use r sees, system developers may
                                                                              be noted that blisiness process rcengineeri ng (BPR) attempts to
        miss some oflhe controls tha t come out of the tradi tional sys tem
                                                                              convert an existing business process rather th an make dynamic
        development approach sllch as backup recovery, security and
        audit trai ls.

       Change cOlllroi often beco mes much more complicated wi th
       pro toty ped systems. Changes in desiglls and requirements Ilappell           3.8 ALTERNATIVE DEVELOPMENT METHODS                                                                                    •
       so quick ly that they are seldom documented or approved. and the
       system can escalate to a poi nt of not being maintainable.                    In the fnee of increasing system complexi ty an d th e need to
                                                                                     implcment new systems more quick ly to ac hieve ben:fits before
       202                                                                                                                elSA Revie w M anual 2011
     ~C~IS~A    Cefli1ied IniormatiOfl
                Systems Auditor'
                                         Chapter :'-Information Systems Acquisition,
                                         Development and Implementation                                                       Section Two: Content

      the business changes, software development practitioners have                 particular programming technique, does not imply or require use
      adopted new ways of organizing software projec ts that vary, or               of a particular software developme nt methodology.
      in some cases radically depart from, the traditional waterfall
      model previously described. III addi ti on, there has been continued          Objects usually are created from a general template called a
      evolution in the thinking about bow best to analyze, design and               class. The template contains the characteristics of the class
      construct software systems and inlhe information leclmoiogies                 without co ntaining the specific data that need to be inserted inlo
      avai lable to per~orll1 these activities.                                     the template to form the object. Classes are the basis f9r most
                                                                                    design work in objects. Classes arc either superclasses (i.e., root
      This section describes different techniques of understanding,                 or parcill classes) with a set of basic attributes or methods, or
      designing and construct ing a software system. The choice of                  subclasses wh ich inherit the characteristics of the parent class
      a panicular method will be driven by considerations such as                   and may add (or remove) functionality as required. In addition to
      organizational policy, developer knowledge and preference. and                inheri tance, classes may interact through sharing data, refe rred
      the teciUlology being used.                                                   to as aggregate or component grouping, or sharing objects.
                                                                                    Aggregate classes interact through messages, which are requests
      Please note that the selection of one of the met hods described in            for services from one class (called a client) to another class
      this section is generally independent of the selec tion of a project          (called a server). The ability of two or more objects to interpret a
      organization model. An object ~oriented approach design and                   messag'c differently at execution, depending Oil the superclass of
      codi ng could be utili zed on a projeci organ ized inlo distinct              the enlling object, is termed polymorph ism. in the wa terfall model of software development just
      as it could be with an agile project where each short iteration               The first objected~orie nted language, Silllula67, was released in
      del ivers working software.                                                   1967. Smalltal k emerged during the 1970s as the first commercial
                                                                                    O ~i cc l - ori e nt ed language. Then followed a series of languages

      3.8.1 DATA·ORIENTED SYSTEM DEVElOPMENT                                        that were either object-oriented from inception (e.g., Eiffel) or
                                                                                    had been modified to include object-or iented capabil ities (e.g.,
      'DOSD is a method for representing softwa n:: requiremen ts by
                                                                                    C++, Object Pascal, and Ada95). The emergence of Java dnring
       focusing on data nntl their strl1cture. There nrc insti tut ions such as
                                                                                    the late I990s provided a signi ficant boost to the acceptance of
•      stock exchanges and service providers such as ~I irlill es. telephone
                                                                                    object technology.
      .companies, etc .. tllat ge nerate lime-depende nt da ta. These data
       arc provided to some of their subscri bers omine- e.g., stock
                                                                                    To reali ze the full benefits Of liSillg object-oriented programming,
       exchange to stock brokers to subbrokers; airline to their travel
                                                                                    it is necessary to employ object-oriented analysis and design
       agencies, etc. These data arc in preknown or prespecified fonnats,
                                                                                    approaches. Deali ng with objects should permit analYSIS,
       which may come in CD- ROt-.'1or be downl oadab le through File
                                                                                    develop.ers and programmers to consider larger logical chunks of
       Tra nsfer Protocol (FTP) in comma separated va lue (CSV) or
                                                                                    a system and clarify-t he programming' process.
       ASC II or other specified formats. The lIser organization develops'
     . its own application (platform variant, development tool variant)
       and can use these data directly in their applications to issue tickets
                                                                                    The major advantages ofOOSD arc as follows:
m      or initiate tl\e purchase or sale of stocks to thei r clien ts. The major
                                                                                    • The abi lity to manage an unrestricted va riety of data types
                                                                                    • Provision of a means to model complex relationships
       adva ntage of this data-oriented system development approach
                                                                                    • The capacity to meet the demands ofa changing environment
       is that it eliminates data transformation errors sllch as character
       transposition, wrong transcription, wrong porting, or conversion,
                                                                                    A significant deve lopment in OOSO has been the decision by
       transcription and transposition. DOSD is genera lly combined
                                                                                    some of the major players in object-ori en ted development to
       with another development techn ique that considcrs processing
                                                                                    join fo rces and me rge indi vidual approaches il.lto a unified
       issues 10 develop a su itable business solution.
                                                                                    approach usi ng the Unified Modeling Language (UML). UML
                                                                                    is ri genera l-purpose notational languagc: for specifying an.d
      3.8,2 ·OBJECT·ORIENTED SYSTEM DEVELOPMENT                                     visuali zing complex sof1ware for large object-oriented projects.
       OOSO is the 11rocess of solution·specification. and modeling                 this signa ls a ll1atll~ation of the object-oriented development
       where data and proceaures'can be grouped into an ent,ity known               approach. Whi le .objeCho"rienlation is not yet pcrvasive; it can
       as an object. An object's data arc referred to as its attributes and         accurately be said to have entered the computine. mainstream.
       its functiona li ty is referred to.a.s its J1.ielhods. This contrasts with   App l~cations lilat lIse obJect-oripnted technology' arc:
     . the tradi tiOlial (structure,1SDLC) approach which considered data           • Web applications                     . .
       sep'ar'atcly fromlhe· procedures that act on them (e.g., program ·           • E-bllsiness applications
       and database specifications). Proponents ofOOSO claim th'e                   • CASE for software development
       combination of data and fun ctiona lity is aligned with how                  • Office automation fore-ma il and work orders
       humans conceptualize everyday objects.                                       • Artifi cial intelligence
                                                                                    • Computer-aided manufacturing ( AM) for production and
      OOSD is a programming tec hnique, not a software development                    process control
      methodology. One eaIl do 0050 whi le follow ing any of the
      widely diverse set of software methodologics: waterfall,                      3.8.3 COMPONENT·BASED DEVElOPMENT
      iterati ve, software enginecring. ag ile and even pure hacking (and
                                                                                    Component-based development can be regarded as an oui g 7'owth
     ·prototyping). A particular programming language, or use of a
                                                                                    of object-oriented developm9llt. Componenl-based development

      CISA Review Manual 2011                                                                                                                         203
 Section Two: Content
                                                       ChB.Pter 3 - lnformation Systems Acquisition,
                                                                    Development and Implementation '
                                                                                                                     e  elSA
                                                                                                                               • cerl"""n formaon
                                                                                                                                s",.", '00".'

means assembling applications from cooperating packages of              Components playa significant role in web-based applicat ions.
exec utable software that make their services available through         Applels are required to extend static HTML, Acti ve X controls
defined interfaces (i.e., enabling pieces of programs called           or Java. Both technologies are compatible with component
objects, to communicate with onc another regardless of which           deve lopment. Component-based dcvelopment:
programming language they were written in or what operating             • Reduces development time. If au application system can be
system they afC running). The basic types of components are:              assembled from prewritlen components and only code for
• In-process client components- These components nlust run                unique parIs of the system needs to be developed, then this
  from withi n a container of some kind such [IS a web browser~           should prove faster than wri ting the enti re system from scratch.
  they cannot run on th eir own.                                       • Improves quality. Using prewritten components means a
• Stand-alone client com ponents- App lications that expose               signifi cant percentage of the system code has becn tested already.
  serv ices to olher sofiwarc can be used as components. Well-         • Allows developers to focus more strongly on business
  known examples are Microsoft's Exccl and Word: .                        functionality. An outcome of component-based development
• Stand-alone serve r co mponents- Proccsses running on                   and jts e.nabling tech.nologies is to further increase abstraction
  servers tha t provide services in standardized ways can be              already achieved wi th high-level languages, databases and user
  components..Thcse are initiated by remote procedure calls or            interfaces. Developers are shielded from low-level programming
  some other kind Ofllctwori< call. Techno logies supporting th is     · details.
  include Microsoft's Distributcd Componen t 'Object Model             • Promotes Illodulati ty. By encou raging or forcing impassable
  (DCOM), 'Object Management Group's C0l111110n Object                    interfaces between discrete units offunctiollality, it encourages
  Req ueS! Brokcr Architecture (C'ORBA) and Sun's Java Ihrough            modularity.
  Remote Method Invocat ion (RMI).                                     • Simplifies reuse. It avoids the need to be conve rsant with
• In-pro(:ess ser ver components- These components mn                     procedural or class libraries, allowing cross-Ianguagc
  0 11 servers within containers. Exmnples include Microsoft's
                                                                          combination and allow ing rellsable code to be distributed in
  Transaction Server (MTS) and Sun's Ell tcrplisc Java Beans (E I8).      an executable format- i.e., no source is required. (To date,
                                                                          however, la rge-scale reuse of business logic has not occurred.)
                                                                       • Reduces development cost. Less eflort needs to be expended Oil
 .Note: The e lSA candidate will not be tested on vendor-
                                                                          design and build. Instead, the cost of softwa re components can
 specific products or services in the elSA exam ..
                                                                          be spread across Illultiple users.                         _
A number of di fferent component models have emerged.                  • Supports mUltiple deve lopment environments. Components
                                                                          written in one language call interact with components written in
Microsoft has its Componenl Object Model (COM). MTS when
                                                                         other languages or rUllning on other machines.
combined wi th COM allows developers to create components that
can be distributed in the Windows enviroliIllc nt. COM is the basis    • Allows a satisfactory compromise betwcen build and buy
                                                                         options; Instead of buying a complete solution. which perhaps
for ActiveX technologi~s, \v.ith Ac ti veX Cont rols being 'among
                                                                         does 110t entirely fitrcquircments. it coul d be possible to
the m OS I widely lIsed components. Alternative component models
                                                                         pmchase only needed components and incorporate these into a
include the COREA Component Model and Sun's EJB.
                                                                         customized system.
Please note that COM/DCOM, COREA (itselfa standard, not a
                                                                       To realize these advantages, attention to software integrati on
specific product) and RMI are sometimes referred (0 as distributed
                                                                       should be provided early and continllously during the
object technologies. As the name suggests, thcy allow objects
                                                                       development process. No mattcr how efficient component-based
on distribu ted platforms to interact. These techno logies, among
                                                                       development is, if system requirements are poorly defined or the
others, are also termed midd lewarc. Middleware is a broad term,
                                                                       system fails to adequately address business needs, the project will
but .n bflsic definition is soflwa,re that provides
                                                                       not be sllccessful.
rUh-t ime services whereby programslobjectslcomp'onents 'can
interact with one another.
                                                                       3,8,4 WEB-BASED APPLICATION DEVELOPMENT
Tool developers 'are supporting onc' or another ofthesc standards      Web-based -application deve l opln~1lI is an important emerging
\"ith powerful, visual tools now available fordesigniilg and           software 'deve-Iopment approach designed to achieve easier and
testing component-based applications. Industry "heavy weights"         more effective integration of code mod ules within and between
sllch as Microson and 'I8M arc supporting component-based              entcrprises. Histor!cally, 'software written in one language on a
development. Additional1y~ a growing'number of cOlhmercial~y ..        part"icular platfonn has used a dedicated application prograllliliing
availabJe appl ica'tioil'servers now support MtS or Ern. There is a    interface (API). The use ofspecializ~d APls has caused
growing nlarket for third-party components. A primary benefit of       difficulties in integrating software modules across platforins.
component-based development is the ab ility to buy proven, tested      Technologies such as CORBA and COM tha t use remote
software from commercial developers. The range of components           procedure calls (RPCs) have been developed to allow real·timc
avai lable has increased. The first components we re simp le in        integration of code across platforms. However, using these RPC
concept- e.g .. buttons and list boxes. Componenls now provide
much more dive rse functionality. Databases arc now avai lable on
                                                                       app(oaches for different APls stili remains complex. Web-based
                                                                       app lication development ,lnci associated Extensible Markup
the web {Q sean:h for commercial components.                           Language (XML) tec hnologies are a recent deve lopment designed
                                                                       to further faci litatc and standardize code modu le and program


204                                                                                                    CISA Review Manual 2011
        ~C~IS~A   Ceftire:llrtlorma(ion
                  Systems, A~tor'
                                          Chapter 3-lnformation Systems Acquisition,
                                          Development and Implementation                                                  Section Two: ·Content
                  .. ...crew .....

        The other problem that web-based application deve lopment                which acts as an electronic directory accessible via a corporate
        seeks to address is to avoid the need to perform redundant               intranet or across the Internet, and allows interested parties 10
        computing tasks wit h the inherent need for redundant code.              learn of the existence of available web services.
        One obvious example of this is a change of add ress notification
        from a customer. Instead of having to update detai ls separately         Standards fo r SOA p, WSDL and UDD I hove been accepted by
        ill l11ultipie databascs-e.g., contact management, accou nts             the World Wide Web Consortium. A numbe r of curren t sofhvare
        receivable and credit control-it is preferable that a coml11on           products and development environments, including Microsoft's
        update process upda tes the nlul tipre places req uired. Web se rvices   .Net fam ily ofprociucts, support we b services. However, some
        arc intended to make this relatively easy to ach ieve.                   imponant standards Stich as those address ing security and
                                                                                 transaction management are yet 10 be defined. Other issues such
        Web applicati on development is different than traditi ona l th ird-     as cha rgi ng for lise of commercially developed web services also
        or fourth-generation prqgram developments in Illany ways-                need to be addressed.
        from the languages aild progmmming techniques used, to the
        methodo logies (or lock thereof) used to control ·the development        3.8.5 SOFTWARE REENGINEERING
        work, to the way the users test and approve the devel9pmen t
                                                                                 Reengineering is a process of updating an existing system by
        work. But the risks o f application devclopment rcmain the SClll1C.
                                                                                 extracti ng ~nd rellsi ng design and program components. Tllis
        for example, buffer overnows had been·a risk si nce computer
                                                                                 process is used to support major changes in the way an organi~1 tion
        programming was invented (for example, tru ncation issues 'wi th
                                                                                 operates. A Humber of tools are now available to sllppol1 this process.
        first generation computer programs), but they really hit the
        headlines only when they could be exploited by aimost anyone.
        almost anywhere in the worl(~ courtesy of tile Internc!.                 3.8.6 REVERSE ENGINEERING
        As wit h traditional program development: a risk-based                   Reverse engineering is the process or studying and ana lyzing
        approac h should be taken in the assessment of web app lication          an appli cati on. a soflware npp licatiol1 or a product fa see how it
        vu lncrabililies: identify the business goals nnd supporting IT          fUllctions anelto usc Ihat in fo rmation 10 deve lop a similar system.
        goals related to the development. then identify what can go wrong.       This proccss can be carried out iJl several ways:
        One's previous experience C<l!l be used to idemi fy risks rc lated       • Dccoll}piling object or executable code in to source code and
        to inadequate spccifications. poor 'coding techniques, inadequate          using it to ana lyze the program
        docllmentation, inadequate QC and QA (including testi ng                 • Black box testing the app lication to be reverse-engineered 1   0
        inadequacies), lack of proper change control tmd coillrols ovcr            unvei l its functionality
        promotion into produc ti on, and so on, and put these in the context
        of the web application languages, development processes and              The major advantages of reve rse enginee ri ng are:
        de liverable.s (perhaps with the support af bes! practi ce mate rial!    • Faster deve lopment and redu ced SDLe durati on
        literature on web appl ications development).1he focus should be         • The possibility of in troduci ng improvements by overcoming the
        011 application development risks, the associated busi ness risks          reverse-engineered application drawbacks
        and technical vu lnerabjl iti es, and how these could materia lize and
        be controlled/addressed. Some controls will look the Saine for all       The IS aud itor sholiid be awore of the following risks:
        application development activity, but many will need to reflect the      • Software license agreements often c011lai n clauses prohibiting
        way the devclopment acti vity is taking place in the area                  the licensee from rcverse engineeri ng the software so that any
        under review.                                                              trade secrets or programming techniques arc not compromised.
                                                                                 • Decomp ilers are rc lative ly new too ls with fimctions that depend
         With web.-based application development, an XML language known            on specific computers, operating systems and programming
         as Simple Object Access Prolocol (SOAP) is used to define APls.           languages. Any change in one of these co mponen ts may require
         SOAP will work with '~ny operating system and progranuning                developing or purchasing a new decomp1ler.
         language that understands 0JvLL. So,AP is simpler than usi ng
       . the more complex RPCMbased approach, with the adva ntage that
         modules are co.llpled loosely so that a cbange to one·component         3.SINFRASTRUCTURE DEVELOPMENT!
         does not n~mnally require changes to other componen ts'. .
                                                                                 ACQUISITION PRACTICES       .
        The second keY 'component of wcb development is the Web
        Services Description La nguage (WSDL). wh ich is also based              The physical architecture ana lys is', the defi nition ofa new one·
        on XM L. WSDl is used to ide illify the SOAP specification that          and the· necessary road map to move from one to the other
        is to be used for the code module AP I and Ihe format s of the           is a critical task for an IT department. Its impact is IlOt only
        SOAP messages used for input and ou tput to the code module.             economic, but also technological, si nee it decides many other
        The WS DL is also used to identify the particular web serv ice           choices dowilstream sllch as operati onal procedures, trai ning
        accessible via a corporate intranet or across the In ternet by being     needs, installmion issues and tOlal cost of ownership (TCO).
        published to a re levant il1l mnct or Intemet web server.
                                                                                 Conflict ing requirements such as evolving towa rd ~ services M
        The final component of web services is nnother XMLMbascd                 based archi tecture, legacy hardware considerations. secure
        language-U niversal Desc ript ion, Discovery and Integration             data access independent of data location, zero data loss, 2417
        (UDDI). UDD I is used to nioke on entry in a UDD I direc tory,           availability anp many others ensure no single plalfolln satisfies

        CISA Review Manual 2011                                                                                                                  205
 Sec t ion Two: Con tent
                                                        Ch ap ter 3-lnformation System s Acquisition,
                                                                    D evelopment and Implem entat ion
                                                                                                                             e  elSA
                                                                                                                                        Certified "lormatiOn

all these requirements equally. Thus physical architecture analys is         The main objecti ve of the second section (3.9.2) is to plan the
cannot be based solely 0 11 price or isolated features. A fo rmal,           physical implementation orthe required tech ni cal infrastructure
reasoncd choice must be made.                                                to set tip the future cnvironment (nonnally producti on, test and
                                                                             development environlllent). This task wi ll cover the procurement
In section 3.9. I, steps are listed to arri ve at the choice of a good       acti vities sllch as contracti ng partners, setting lip the SLAs, and
phys ical arch itccllJre and the way 10 defi ne a possible road map          developing installation plans and installation lest plans. A well-
for supporting the migration of the Icclmical arc hitecture to a new         designed selection process must be ensured, taking.analytical
olle to reach the following goals:                                           results and intuition into account and guara nteeing alignment
• To sliccessfully analyze the ex isting archilcctmc                         and commitment for implemen tation success. Due to the possible
• To design a lIew architecture that takes into account the                  heterogeneolls nature of the infrastructure fou nd. it is necessary
  existing architecture and a company's particular constraints!              to develop a clea r implementation plan (including deliverables.
  requirements, such as:                                                     delivery times, tcst plans, ete.). It is also necessary to plan the               ,
  - Reduced costs                                                            coexistence of the old and new system, to avert possible mistakes                 !
  - rncreased functionality                                                  during the installation and go,live phase (See ex hibit 3.1 8).
  - Minimu m impact on dai ly work
  - Security and confident iality issues                                     Thus, informat ion and communication technologies (ICT)
  - Progressive migration io the new arch itecture                           Ocj5artments often face these requirements. The suggested
• To write the fllllctional require ments of this new arch itecture          solution must:
• To develop a proof of concept based on these fu ncti onal                  • Ensure alignme nt of the information and cOlll mu nication
  requirements:                                                                technologies (I CT) with corporate stnndards
  - To characterize price. functionaliry and perrormance                     • Provide appropriate levels of security
    To identify additi onal requiremen ts th~t will be lIsed later           • Intcgrate \Vi th cUITelll iT systems
                                                                             • Consider IT industry t'rc nds
The resulting requirements wi ll be doclIlllents and drawings                                                            0
                                                                             • Provide futu;'e operati onal fl exibility 1 support busi ness
descri bing the reference infrastructure thm will be used by all               processes
projects downstream. With these rcq uirelnenls in hmlCL these                • Allow tor projected growth in inrrasrmcnn""C wi thout major llpgrades
projects can begin to start implemcntation.                                  • IIl.elude tec hnical architecnlre cOI}siderati ons for information
                                                                               security, secure storage, etc.
  The requirements are va lidated lIsing a pl:oofofconcept. The              • Ensure cost-effecti ve, day-to-day operational support
  proof of concept is a tcst-bed implementation of the physical              • Foster the usage of standardized hardware and software
  architecture. Ii saves mo ney because any problems are detected            • Maximize ROI) c.ost transparency and operational efficiency'
  and correc ted early, when they arc cheaper 10 correct, and it gives
. confidence to the teams that the requirements correctly instruct
  potential ve ndors on the requirements they have to meet.

                                                     Exhibit 3.1&-Risks of Current System

                                                                  Deficits in
                            Insufficient                                                            . Endangered
                              ProGess                                                                    Future ·

                                                ~. ·                     l
                              Support                                                                 . Reliability

                                                              Existing system

                                                 f             does not fulfill

                                           ---- I
                      . Future                             . current and f.uture
                      Business                                    . needs                                      Increase
                    Requirements                                                                                of Cost
                     Not Fulfil led

                                                  Deficits in
                                                                                  Development                                                                      •
                                                    Supply                        Handicapped
2 06                                                                                                          CISA Rev ie w Manual 2011
         ~C~IS~A   Cer tified Inlormation
                   Systems Al.Idito,.
                                            Chapter 3 - lnformation Systems Acquisition,
                                            Development and Implementation                                                        Section Two: Content
                   .. 1SAo:.. , ..... _

                                                          Exhibit 3.1~Project Phases of Physical Architecture Analysis

                                              2. Analysis and      3. Orall            4. Functional         5. Deline final       6. Proof of
                                                design                functional          requiremenls          lunctional            concept
                                                                      requirements                              requirements

                                                                                                  and Discussion

......   3.9.1 PROJECT PHASES OF PHYSICAL ARCHITECTURE                                   discussed and a list of the requirements that need to be refined or
::-~                                                                                     add~d will be c~l1lposcd.
         Exhibit 3.1 9 shows the project phases to physical architecture                 This is the last checkpoint before the sizi ng and the proof
         anal ysis and, in the background, the lime at whic h the vendor                 of eoneept'(POC) slarts. althongh Ihe plan ning of Ihe POC
         selecti on process may wke place.                                               starts alter the second workshop. With the fin ished n.lI1ctional
                                                                                         requ irements, the proof of concept phase begins.
         Review of Existing Architecture
         To start the "process, the latest dOClIlllcnt s about the ex isting             Proof of Concept
         architecture Illu st be revj!'!wed. Participan ts of tile first workshop        Establish ing a POC is highl y recommended to prove that tile
         will be special ists of tile leT department in all areas direct ly              selected hardware and soflwa re are able to meet all ex pectations,
         impacted by physical archilccllIrc. Examples are server, storage,               including security requirements. The deliverable orthe POC
         security nnd overall IT infrastrllcnlre.                                        should be a running. prototype, including the associa ted document
         Special care must be taken in characteri zing all the operational               and test prolOcols describing the tests and their resu lts.
         constrai nts' that impact physical archi tec ture such as:
         • Ground issues                                                                 To start, the POC should be based on Ihe resufis of the
         • Size limits                                                                   procurement phase described below in this sectioll. Fo r this
         • Weight limits                                                                 purpose, a representative subset of the larget hardware is used.
         • Current power supply                                                          The software to nll1 Ole POC can be .either test versions or
         • Phys ica l security issues                                                    software already supplied by the vendor; thcrefore, add it ional
                                                                                         costs are expected to be mininial.
         The Olit PUt of the first workshop is a list of components of the
         current infrastructure and constraints defining the targe t physical            To keep costs low, most elcments of the framework are
         architecture.                                                                   implemented in a simplified fo rm. They will be extended to their
                                                                                         fi nal fo rm in later phases.
         Analysis and Design
         After reviewing the existing architecture, the analys is and design              The prototype should demonstrate the following features:
         of the <te nlal phys i~a l' arc!litecture has to be undertaken, adhering .       • r~e basie setup of the core security infrastructure
         toiJest practices·and meeting business requirements.                             • Corr.eet fu nctionali ty of auditi ng componentS
                                                                                          • Basic.bui functional implementat ion of security measures
         Draft Functional Requirements                            .                          as defined.                              .
         Wit h the first IJhysical archilect tlre desi gn in hml(~ the fi rsr(draft)    ' . Secured transactions
         offullct ional requirements is composed. Th is material is the input             • Characteri zation in terms of insta llation constraints ancrJimits
         for 'the next step and the vendor selection process.                               .(server size, server current consumption, server weight, server
                                                                                             rOom physical security)
         Vendor and Product Selectipn .                                                   • Perforl'nance
         Wh ile the draft f'llllctional.rcquircmenis arc wri tte n, the vendor            • Resi liency
         selection proces~ proceeds in parallel. This process is described in             • Funding and costing model
         detail later in this chapter.
                                                                                         Re lated implementation projects which prepare the ground for
          Writing Functional Requirements                                                dep loymenl'shou ld also be part oflhe POC since Ihey wi ll be
          After fi nishing the draft functional requi rements ano feeding the            lIsed in the same way as they are used in the producti on phys ical
          second part of this project, Ihe funct ional requirements document             arc hitecture. At the end of this phase a last workshop is held
          is written, which wi ll be introduced at the second architecnlre               where the production sizi ng and layoll t is adnpled to include
         .workshop with staff From all afTected parties. The results will be             POC conclusions.

         CISA Review Manual 2011                                                                                                                          207
                                                                                                                                                          .;   ...-:   .:

Section Two: Content
                                                         Chapter 3-lnformation Systems Acquisition,
                                                                    Development and Implementation                          ~C~IS~A   Certified InfOfmatioo
                                                                                                                                      Systems AuditOf'
                                                                                                                                      .. 'S'V,_ ....

3.9.2 PLANNING IMPLEMENTATION OF                                            the chosen solution and detennine the quantity strucnlre of the
                                                                            delivcrables. The rcquirements statements arc also produced.
To ensure thc quality of the results, it is necessary to use a phased       Additionally, the procurement process begins the service-level
approach to fit the whole puzzle rogethcr. It is also fundamental           managemcnt process. During these activities, the preferred
to set up the communication processes to other projects like those          partners are invitcd to the negotiations process and the
described earlier. Through these different phases the components            de li verabl es, contracts and SLAs are signed (See exhibit 3.2 1).
are fi t together, and a clear understanding of the available and
contactable vendors is established by llsing the selection process
                                                                            Delivery Time
during the procurement phase and beyond. Furthermore, it is
                                                                            During the delivery,time phase, the delivery plan is developed
necessary to select the scope of key business and tecimical
                                                                            (See exhibit 3.22). This phase overlaps in some parts with the
requirements to prepare the next steps, wh ich include the
                                                                            procurement phase.
deve lopment of the de livery, insta llation and test plans. Moreove r,
to ensure a future proven solution, it is crucial to choose the ri ght
                                                                            The del ivery plan should include topics such as priorities, goals
partners wit h the right skills. Ex hibit 3.20 describes a phased
                                                                            and nongoals, key facts, principles, communication strategies,
approach.                                                                   kcy indicators, progress on key tasksrand responsibil ities,
As shown in ex hibit 3.20, the requirements analysis is not part of
                                                                            Installation Plan
this pJocess but constantly feeds reslllts into the process.
                                                                            During Ihe installntion planning phnse, the installation plan is
Ifa Ga ntt chart is produced with these phascs, most likely somc
                                                                            dcveloped in cooperation witll all affected parties
phases likcly overlap; therefore , the different phases Illust bc
considered an iterat ive process.                                           (See exhibit 3.23).

During the ro'ur different phascs it is necessary to fit all the           All add itional step is to review the plan wi th the involvcd parties
components together to get prepared for projects downstream                and of course with th ose responsible for the integration projects.
(e.g., data migration).                                                    This is all iterative process.

Procurement Phase                                                           Installation Test Plan
During the procurement phase the communication processes                    Based on the known dependencies of the installation plan, the test
is established with the analysis project to get an overview of             plan is deve loped (See ex hibit 3.24).

                                 Exhibit 3.2D-Project Phases of Planning the ImplemenlaUon of Infrastructure

                     Requirements AnalysiS .-_ _ _ _ _,
                          1. Procurement            2. Delivery                3. Installation            4. Installation
                             Phase                     Time                       Plan                       Test Plan

                                                       Exhibit 3.21-Procurement Phase

       Requirements Analysis
            1. Procurement            ~ . Delivery                3. Installation          4. Installation
               Phase                      Time                     . Plan                     Test Plan

            1.1 Develop vendor
                                        1.2 Develop
                                            vendor long
                                                                  1.3 Develop
                                                                      vendor short
                                                                                             1.4 Select
                                                                                                                        1.5 Define
                                                                                                                            partnership.                                    •
                criteria.                   list.                     lis!.                      vendor(s).
                   Q                           V                          §]                       []                                                                       •
208                                                                                                          CISA Review Manual 2011
    ~C:IS~A   Cer1ifiedJnformat;OI1
              Systems A  odilOl"
                                      Chapter 3- lnforma tion Systems Acquisition,
                                      Development and Implementation                                                                 Section Two: Content
              .. """..:'..-

                                                                     Exhibit 3.22-Delivery Time

                               Requirements Analysis

                                      1. Procurement        2. Delivery                    3. Installation              4. Installation
                                         Phase                 Ti me                          Plan                         Test Plan

                                                 2.1 Develop'                 2.2 Review
                                                     delivery                     delivery
                                                     plan .                       plan.

                                                                  exhibit 3.23-lnstaliation Plan

                              Requirements Analysis

                                  1. Procurement          2. Delivery                    3. Installation             4. Installation
                                     Phase                   Time                           Plan                        Test Plan

                                                                          3.1 Develop'                     3.2 Review
                                                                              installation                     installation
                                                                              plan.                            plan.

                                                                Exhibit 3.24-lnstallation Test Plan

                               1. Procurement          2. Delivery              . 3. Installation                 4. Installation
                                  Phase                   Time .                     Plan                           . Test Plan

                                                                                .   '.

                                                                                                     4.1 Develop                    4.2 Review
                                                                                                         test plan                      test plan.

c   CISA Review M anual 2011                                                                                                                         209
                                                          Chapter 3- lnformation Systems Acquisition,
                                                                                                                              ~C:IS~A   Certified InIOfma!ion
Section Two: Content                                                       Development and Implementation

The lest plan includes test cases, basic requirements' specificmions,          -Compilers
definition of the processes and. as far as possible, measurement               - Program li bra ry software
infol111ation for the app lications and the infiastruchrrc.                   - Database management software an d programs
                                                                              - Communi cations software
3.9.3 CRITICAL SUCCESS FACTORS                                                - Access control software
                                                                              - Job scheduling software
Critical success factors ofpianning the implementation include:
                                                                            • Support requi rements slIch as:
• To avo id delays, the approprialc skilled staff must attend
                                                                              - Systclllmaintenance (for prevent ive, detecti ve [rault
  workshops and pa rticipate for the entire project duration.
                                                                                 reportin g] or cOlTcctive purposes)
• The documentation needed for carrying out the work needs to
                                                                              - Training (user and technical stan)
  be ready at project initiation.
                                                                              - Backups (daily and disaster backups)
• Decision makers must be involved at all steps to ensure alJ
                                                                            • Adaptabil ity requirements such as:
  necessary decisions can be made quickly.
                                                                              - Hardware and software upgrade capabilities
• Part one of the project (Analys is of Physical Architec ture) must
                                                                              - Compati bi li ty wi th existing hardware and ?oftware platforms
  be completed and the needed infrastructure decisions Illust be
                                                                              - Changeover to other equipment capabilities
                                                                            • Constraints stich as:                                                             --,i
                                                                              - Sta rri ng levels
3.9.4 HARDWARE ACQUISITION                                                    - Existing hard ware c<l pacity
Selection ora computer harc!wnfC and software environment                     - Deli ve ry dates
freq uently requires the preparation ofspecificalions for                   • Conve rsion requi n:mcnls such as:
dislri buliOil to hardwarc/sothvare (HW/SW) vendors and criteria              - Test time for the ha rdware and soft ware
fbr evaluating vendor proposals. The specificat ions are sometimes            - Sysren.l conversion raci lities
prese nted to vendors inlhe form of an invi tation to IC!lcier (lIT),         - Cost/pricing schedule
also known as a request for proposal (RFP).
                                                                            Acquisition Steps
The speci fications must defi ne, as completely as possible, the usage,     When purchasing (acquiring) hardwme and softwnre from a
tasks and requirements for the equipment needc(~ and Illust include         vendor, conside rat ion should be give n to the fo llowing:
a descliplion oCthe cnvif"onmcnt where that equipment will be used.         • Testimon ials or visits with ot her users
                                                                            • Provisions for competitive bidding
When acquiring a system, the speci fi cations should include                • Analysis of bids against req uirements
the fo llowing:                                                             • Compariso n of bids ngai nst each other lIsing predefined
• Organizational desc ripti ons indicating whether the computer             - evaluation criteria
    facilities are centralized or decentral iz~d. dist ribllte~        .    • Analysis of tile vendor's fina ncial conditi on
   ou tsourced. man ned or lights-out.                                      • Analysis ofti1e vendor's capability to provide Illaintcnancc and
· -Information processing requirements such as:                               'support (including tra ining)
   - Major existing appl iCalion systems and future application systems     • Review of del ive ry schedules agni llst requirements
   - Workload and pe rfonna nce requirements                                • Analysis of hardware and soOwa re upgrade capabili ty
   - Processing approaches (e.g_, onlinelbatch, cli e nt ~serve r,          • Analysis of security and control faci lities
       rea l ~time databases, continuous operation)                         • Evaluation of performance agai nst requ irements
• Hardware requirements such as:                                            • Revicw and negotia ti on of price
   - Central processing onit (C PU) speed·                                  • Review of contrac t terms (including right 10 audit clauses)
   - Disk space req uirements                                                                                                        .
                                                                            • Prepa rat ion of a formal written report summarizillg lhe-analys is
   -:- Memory requ irements                                                    for each of the alternatives and justi fy ing the selection based on
   - Number of CPUs requited.                          .                       benefits ano cost
- -:- Periphe.ral devices (e.g" sequential qevices such, as tnpe drives;
       direct access dev ices such as magnet ic' di$k dri ves, printers,                                                           r
                                                                             The criteri a a'nd data \lsed for evaluating vendo- proposals should
       compact disc drives, digital 'video disc drivcs. Uni versal Serial be properly planned and doctlmcnted:Thc.foliowing-are some of
       Bus [USB] peripherals, and sectlre digital multimed ia cards          the criteria that should be considered in the evaluation process:
       [SD/M.MC]) requi red or to be excluded (usua lly for security         • Turn arollnd time- The time thai the help desk 0;' vendOF
       reasons)                                                                takes to fix a problcm rrom the mome nt it is logged in
   - Data preparation/input dc\'ices that accept and converfdata for '. nesponse time- The lime a system takes 10 rcspond to a
       machine processing                                                      spccific query by the user
       Direct entry'devices (e.g .. terminals, point·of-sale tel;ll1inals or
                                                                             • System reaction time- The lime taken for logging into a
       automated teller mac hines)
                                                                               system or getti ng connected to a network
   - Network ing capability (e.g. , Ethernet connecti ons, modems
       and integrated sc rvices digitalnetwo(k [ISDN] connections)
  - Numbc r of termillals or nodes the syslem needs 10 support
                                                                             • Throughput- The quantity of use ful work made by the system
                                                                               pcr un it oftilllc. Throughput can b~ mensured in instructions
• System software appl ications such as:
   - Operating systcm software (cui'renl"\'crsion and any
                                                                               per second or some other unit or performance, When rere rri ng
                                                                               to a dat a transfer operation, throughput meas ures the useful .
       requi red upgrades)·
   - Utilities
                                  .                       .                    data transfer rate and is ex prcsse(l in kil obits per second (Kbps).
                                                                               megabits pe r second (Mbps), und gigabits per second (Gbps).
210                                                                                                          CISA Review Manual 2011
        ~C~IS, ,,,mied,,"""""",,
                 H     Systems Auditor'
                                          Chapter 3-lnformation Systems Acquisition,
                                          Development and Implementation                                                       Section Two: Content
                       "-"- 1~1"""'"

        • Workload- The cap.1city to handle tile required volume o f             • Compatibility wi th ex isting systems
          work, or the vo lullle of work that the vendor's system can handle     • Security
          in a given time frame                                                  • Demands on existing stafT
        • Co mlHttibility- The capabil ity of an existing appli cation 1 run
                                                                        0        • Training and hiring requirements
          successfully on the newer system suppl ied by the vendor               • Future growth needs
        • Ca pacity- The capability of the newer system to handle a              • Impact 011 system and network performance
          number of sim ultaneous requests from th e network for the             • Open source code vs. proprietary code
          al)plication and the volume ordata that it can handle fi'om each
          ofl he users                                                           3.9.6 SYSTEM SOFTWARE IMPLEMENTATION
        • Utilization- The system availability lime vs. the system downtime      System software implementation invol ves identifying features,
                                                                                 co nfiguration options and controls for standard configurations
        When performing an audit of this area, the IS auditor should:            to apply across the organization. Additionally, implementation
        • Determine if the acquisition process began with a business             involves testing the software in a nonproduction environment and
          Ileed and whether the hardware requirements for this need were         ob tai ni ng some foml of certification and acc reditat ion to place
          considered in the specifications.                                      th e approved operating system software into production.
        • Determine if severa l vendors were considered and whether
          the comparison between them was done according to the
                                                                                 3.9.7 SYSTEM SOFTWARE CHANGE CONTROL
          aforementioned criteria.
        3.9.5 SYSTEM SOFTWARE ACQUISITION                                        All test results should be documented, reviewed and approved by
                                                                                 tec hn ical ly qual ified subj ec t l11(1tter experts prior to procluctioll llSC.
        Every time a technological development has allowed for increased
        co mputing speeds or new capabilities, th ese havc been absorbcd
                                                                                 Change control pl:ocedures are designed to ensure that changes
        immedia tely by the dcm.,nds placed on computing resources
                                                                                 nrc iluthorized and do nOI disrupt processing. This requires
        by more ambi ti ous app lications. Consequentl y. improvements
                                                                                 that IS managcment and perso nnel are aware of and invo lved
        have led to dece ntralized, interconnected open systems thrOl.\gh
                                                                                  in the system soflwnre change process. Change co ntrol
        fun cti ons bund led in operating system so lhvarc to meet these
                                                                                  procedures should ensure that changes impacting the production
        needs. For example, network manage ment and cOJlnectivity nrc
                                                                                 systems (particularly in relation to the impact of failure
        leatures now found in most operating syStems.
.,.,                                                                             during instal lation) have been nsscssed appropriately, and that
                                                                                 appropriate recovcry/backollt (rollback) procedures exist so that
        It is IS management's responsibility to be flwa re of HW/SW
                                                                                 the impact of allYfa ilure during instnllatioll C~Ul be minimized.
        cflpabilitics since they may improve business processes
                                                                                 For example, change control could be accomplished by havi ng
        and provide expanded applicalion,services to businesses
                                                                                 a configuration managemen t system in place .for maintaining
        and cuSlOlllers in a more effective way. It is importaillthat
                                                                                 prior OS versjons or prior states whcn applying security patches
        organizalions stay current by applying the Ialest version or rele<lse
                                                                                 reJated to high·nsk security issues (see chapter on protection of
        and updates/patches of system softw<lre to remain protected
                                                                                 information assets). hi many situations, IS management will have
        and competiti ve. If the version or release is not current, the
                                                                                 to apply patches to implement vendor solutions related to security
        organization risks bcing dependent 011 software that may have
                                                                                 issues <llTecting netwo rk- and host-based systems. Chan ge control
        known vuln cmbilities or may become obsolete and no longer
                                                                                 procedures should also ensure that all appropriate membe rs of tile
        supported by the softwarc vendor.
                                                                                 management team who could be affected by the chnngc have been
                                                                                 properly informed and have made a previous assessme nt of the
         If changes and IIp.~ates are not made, busi ness operations could be
                                                                                 impact of the cha nge in c.ach.area.
        se riou ~ lyimpeded or disrupted. Software vendors usually advise
         their customers of the latest system upgrades.or fixes aimed at
~~     . reducing secur!ty-r~lated vuln~rabiliti es as w~lI as other jndependent
,. .
,        entities or companies, S.llCh ~ the SANS 'Institute, the CERT ofthq '. ·3.10 INFORMATION SYSTEMS MAINTENANCE
         Software Eng'ineering lnstitute (SEI) at Camegie.Mellon· Univers'ity,    PRACTICES
         O ~lcJ\fee Threat Cenler. In addition, l1~ncurrellt 'software may
         be incompatible with 'new technological req uiremcnts that may
                                                                                 -System.maintenance practices refer primarily to the proces~ qf
         accompany the introduction ofne\\' applications ..                       managing change to applicatiOli systems wh ile maimaining the
                                                                                  integ rity of both the producti.on ~olll'ce·aJ;(CeXeC lIlabl c code.
         Short- and long-term plans should docUlnent IS management's
         plan for mi gmting to newcr, more efficient and more effecti ve         Once a system is moved into production, it seldom remains static.
         operatin g systems and related systems software.                        Change is expected in all systcms regardless of whether they are
                                                                                ve nelor-supp lied or inte rnally developed. Reasons for change
        When selecting new system softwa re, a number ofbllsincss and           in normal ope rati ons incl ude in ternal lTlbusiness C  ha!lges, new
        technica l issues must be considered including:                         externa l.regulations. changes in classification related to either
        • Business, rui,l ctional and technical needs and specifications        sensitivity or crit icali ty, audits. and adverse incidents sllch as
        • Cost and belle fi.(s)

                                                                                illlrtlsions and viruses.
        ·.Ob~cilescc llce

        CISA Review M anual 2011 .                                                                                                                         211
 Section Two: Content
                                                          Chapter 3-lnformation Systems Acquisition,
                                                                     Development and Implementation                       ~c~'s3A    Certified l"fOfma!ioo
                                                                                                                                      ystems Auditor'
                                                                                                                                     .. l$"Vc ...........

 To control the ongoing maintenance of the system, a standard              This process becomes even more important when the programmer
 process for performing and recording changes is necessary.                who creates the program is also the operator. In this case, it
 This process, which lIsually mirrors the organization's SDLe              is assumed that the IS department is either small or there are
 process, shou ld include steps to ensure that the system changes          few app lications being processed. Special change management
 are appropriate {Q the needs orlhe organization, appropriately            procedures must be closely followed since segregation of duties
 aut horized, documented, thoroughly tested and approved by                cannot be established in this environment and compensating
 management The process typically is establ ished in the design            controls are required. It requires user management to pay more
 phase orllle ap plication when app lication system requirements           attention to changes and upgrades Illade by the programmer, and
 <Irc baselined.                                                           proper authorization must be given to the programmer before
                                                                           pUlling any change into production. In lieu of the manual process
 3.10.1 CHANGE MANAGEMENT PROCESS OVERVIEW                                 of management approving changes before the programmer can
                                                                           submit them into production, nUlIlagcment could have automated                    !
  The change management process begins with authorizing                                                                                                      i·
                                                                           change control software installed to prevent lU1<luthorized                       •
  changes to occur. For this'purpose, a methodo logy should exist
                                                                           program changes. By doing this the programmer is no longcr
  for prioritizi ng and approv ing system change requests. Change
                                                                           responsible for migrating changes into production. The change
  requests are initiated from end users as well as operational
                                                                           control software becomes the operator Ihal migrates programmer
. staff and system development/maintenance staff. In any case,
                                                                           changes into production based 011 approval by manage ment.
  aut horization needs to be obtained from appropriate levels
  of end-liser nnd systems management (e.g., a change control
                                                                           Programmers should not have wri te, modify or delete access
  group, configuration control boards). For acquired systems, for
                                                                           to production data. Depending 0 11 the type of information in
  example, :.l rendor may distribute periodic updates, patches or
                                                                           production, programmers may not even have rea d ~o l1l y aCcess
  new release levels of the software. Use r and systems management
                                                                           (or access to customer credi t card numbers US Soc ial Securitv
  ~ holi id review such changes. Determination shou ld be made as to
                                                                           llumbers/nationallD nu.mbcrs. or other sen~itive inrormation tI1<\1
  W lilL'r the changes are' appropriate for the organization or will
                                                                           Illay require added securi ty).
  negatively affec t the.existing system.
                                                                           Deploying Changes
 Users should cOIwey system change requests 10 the syslem
 management using some type of formal correspondence such a.s              After the end user is satisfi ed wi th the system test results and
a standard change request form, mcmo or e-mail message. At a               the adequacy of the system documentation, approva l shou ld
 minimulll, the user request should include the requestor's name,                                  .
                                                                           be obtained from user In anagcmcnt. User approval could be
datc of req uest, dnte the ch,lI1ge is needed, priority of the req uest,   documented on the origina l change request or in some other
a th orough description of the change request, a description of any        rashion (memo or e-mail); however, evidence that verifies llser
anticipated e:fTccts on other systems 0r programs, and fall back           approval should be retained by t!le system maintenancc staff
procedures in case the changes cause the failure of the system.
The uscr could also provide a reason for the change, a cost                Documentation
justification analysis and ihe expected benefits of the change.            To ensure the effecti ve uti lization and fufure maintenance or a
In addi tion, the request should provide evidence that it has been         system, it is important that all relevant system doclllllentat ion
reviewed and authori zed by user management. A signature on the            be updated. Due to light time constraints and limited resourccs,
request form or memo typ ically provides this evidence.                    thorough updates to documentation are often neglected.
                                                                           Documentation requiring revision may consist of program ancV
Change requests should be in a fOnllat Iluit ensures all changes           or system nowcharts, program narratives, data dictionaries, ent ity
are considered for action and allows the system management staff           relationship m~e l s, data now diagrams CDFDs), operator run
to eaS track the status of the ~€;qllest. This is usually done'by
      ily                                                                  boo~s and en d~u s er procedura l manuals. K   eeping the in ternal
assigning a unique tontrol number to each request and enteling the         coherence ofaH these items is a challenge;. softwa re confi guration
chartge request infonnation into a 'computerized system. This can          management .pac~ages (see .below) . valuable 100 1.
also be performed inanuallY. With deta iled infonriatjo~ regarding
~ach request, managemellt ·can identify those requests that ha·ve been     Procedures should be in place to ensure that documentation
completed and are still in progress' or' hilVC not been addressed yet.     stored ofTsite for disaster recovery purposes is also updated. This'
Management can also lise this in'fonnatioll to help enS that user'
                                                          llre             documentation is often overl ooked and may be ?llt of date.
requests are addressed in a timely manner. Se~ ~x'hibil 3-.25.
                                                                           Testing Changed Programs
All requests for changes and related information should be                 Changed programs should b~ tested and also eventually certified
maintained by the system mainte nance staff as part of the                 wilh the same discipl ine as newly developed systems to cnsure
system's permanent documentation.                                          that the changes perform the intended functions. In addition, if
                                                                           the risk analysis determines it is necessary, additional tcsting

Maintenance records of all program changes should exist either             would be required 10 ensure:
manually or automatically. Several library management software             • Ex isting functionality is not damaged by the change
                                                                           • System performance is nO degraded because of the change

prod ucts provide this type of audit trail. The mai ntenance
information usually consists of the pi'ogrammer 10, time and date of       • ~o..secl!rily ex posures havc been created becal1se of tile cl13nge
change, project or request number associated with the change, and
befo.(Cllnd after images of the lines Qf code that were changed.
212                                                                                                        CISA Review Manual 2011
    ~C~IS~A   Systems Information
              Certified Alldilot'
              "I\OGA" _ _
                                     Chapter 3-lnformation Systems Acquisition,
                                     Development and Implementation                                                                   Section Two: Content

                                                                  Exhiblt3.25-Sample Change Request Form

                                                     Request for Change (RFC) Document
                      1.         Contents
                      ThiS document shall ensure that, at a minimum, every major'change will be applied to the customer's mission or business critical
                      systems in a controlled environment. tt shall support all affected parties in gaining a more reliable and resi lient
                      infrastructure . Thus the usage of this document is mandatory for all involved personnel and-during normal operations-
                      has to be distributed in a timely manner to provide the setup of a Forward Schedule of Change. In the rare condition of
                      emergency changes it must be belatedly completed for documentation purpose. In either case there must always be a
                      formal approval for the RFC covered in thi s.document.

                      2.         Usage Guidance
                      The following figure shall illustrate the uliage guideline for this document.
                                                              The area shaded in red symbolizes the mandatory usage of the formal AFC procedure
                                                              depicted in this document.
                                                              If the proposed change has an impact either on criticallvital systems (criticality is high) or
                                                              on a massive amount of systems (wide penetration) then it is- by definition-a major
                                                              change and the AFC procedure has to be executed.
                                                              In any Diner case (area left blank o·n the figure) it is up to the requestor to choose a
                                                              formal approach or io bypass the AFC procedure.
                                                       To follow best practices a list shall by compiled which itemizes all systems classified as .
                                 Penetration           either critical or vital so the requestor will know what to head for (criticality). Additionally a
                      percentage rate of affected systems shall be published as a threshold so tllat it is clearly defined when to execute the AFC
,                     procedure (penetration).
                      3.         General RFC data


                  aiD Has to be filled out by the requestor
                  _        Has to be filled out by the approver

                      4.         Scope of Change

                      Detailed description (please be as spel'ific as you can)

                      Expected benefit

    CISA Review Manual 2011                                                                                                                                    213
                                                                                                                               .    ''':''

Section Two: Content
                                                 Chapter 3-lnformation Systems Acquisition,
                                                            Development"and Implementation                       ~c~'s3A   Certified Information
                                                                                                                           Systems Auditor'

                                      Exhibit 3.25-Sample Change Request Form (cont)

            Implementation checklisllrelease management

            Task       Description                                                           Responsible Party


            Affected configuration items (Cis) and impact on Cis

                                               Yes/No      Description

            Per10rmance degradation             DO
            Redundancy loss                     DO
            Service disruption                  DO
            Reboot                              DO
            Downtime                            DO
            Fallbacklbackout implemented        DO
            Associated costs (hardware, software, manpower, etc.)

            5.       Approval/Rejection

            6.       Postimplementation Review


                                                                          " Yes/No   Description

           " DoeS the implementation meet your expectations?               DO
            Are there any deviations from the outlined procedure above?    DO
            Is the documentatin/configuration management database          DO
            up to date?

            Are the stakeholders informed of the change?                   DO

214                                                                                                CISA Review Manual 2011
                s3 ..---,....-
           ~C~I A     Certilied Inlormalion
                      Systems, Audi tor'
                                              Chapter 3-lnformation Systems Acquisition,
                                              Development and Implementation                                                     Section Two: Content

           Auditing Program Changes                                                      Distributed systems, such as point-of-sa le systel11s, offer an
           In eva luating whether procedures for progra m changes are                    addit ional challenge in ensu ring that changed programs are rolled
           adequate) the IS auditor should ensure that controls arc in place             out to all nodes. The rollout may be performed over asignificant
           to protect production application programs from unauthorized                  period of time to enable:
           changes. The control objectives arc as fo llows:                              • Controls to be exercised over conversion of data
           • Access to progmm libraries should be restri cted.                           • Tra ining of staff who wi ll be lIsing the changed software
           • Supervisory reviews should be conducted.                                    , Support to be provided to users of the changed system
           • Change requests should be approved and doclImented.                         • Reduction o[the risk associated with changing all nodes at
-.         • Potential impact of changes should be assessed.                               the same time, and hav ing to back thelll all out if something
           • The change request should be documented all a standard fOTm,                  goes wrong
             paying particular attention to the followi ng:
             - The change specifications should be adequately describec\ a               In view of the time it may take to 1'011 out chn ngcs to th e who le
               cost analysis developed and a target date es tablished.                   distributed system, controls must ensure that all nodes are
             - The change form should be signed by the tlser to designate                eventually updated, Lf changes are mnde on a regular basis, it may
               approval.                                                                 be necessary to check regu larly thnt all nodes arc running the
             - The change form should be reviewed and approved by                        same versions of all softwarc.
               programming management.
             - The work should be assigned to an analyst: programmer and                 Change Exposures (Unauthorized Changes)
               programming group leader for supe rvision.                                An unauthorized change to applicat ion system programs can
           • A sample or program changes made during the audit.peri od                   occur ror several reasons:
             should be selected and traced to the maintenance (0 1'111 10                • The programmer has access to prociucliolllibraries contain ing
             determinc whether the changes are aut horized check that the                   programs and datn including object code.
             form hns appropriate <lpprova!s, and compare the date on th e               • The USCI' responsible for the app lication was not aware of
             form with the date of prod uc ti011 up~late for agreement.                     the change (no user signed thc maintcnance chrlllge request
           • If <t n independent group updates the program changes in                       apjJroving the start of the work). .
             production, the IS auditor should determine before the update               • f\ change request fonn and procedures were not fonwilly established.
             whet her procedures exist to ensurc possession of the change                • The appropriate management official did not sign the change
             request forlll . (This is accomplished by watching the groups                  forlll approving the start of th e work,
      "      perform their jobs.)                                                        • The use r did not sign tile chan ge form signi fying acceptance
                                                                                           before the change was updated into production.
           Emergency Changes                                                             • The changed soilrce code was not properly reviewed by the
           There may be times when emergency changes are required to                       appropri ate programmin g persoll ne l.
           resolve system problems and enable critical "productionjob"                   • The appropriate management official did not sign the change
           processing to continuc, Procedures should primm;ly' exist in the                 form approving the program for update to producti oll, '
           application's oper<ltions manual to ensure emergency fixes can be             • The programmer put in exh'a code ror personal benefit (i.e.,
           performed wilhoUl compromising the integrity of tile system. This               cOlllmitted fraud).
k~         typically involves thc usc of speciallogol1 IDs (i.e .. emergency IDs)        • Changes received from the acqui red software vendor we re not
           that grant a programmer/analyst temporary access to the production              tested or the vendor was allowed to load the changes di rectly
           environment during these emergency situations. The use of                       into production/sitc. Th is happens ill cases of distributed
           emergency IDs should be logged and carefully monitored since thei r             processing sites, such as point-of·sale, banking applications,
           use grants someone pO\verful privileges, Elnergency fixes should                ATM networks, etc.
           be completed using after·the-ract, follow-up procedures which
           ensure that all nonnal change manage,ne nt controls are retroactively         3.10.2   CONFIGURATIO~          MANAGEMENT
           app lied, Changes done iT; tliis fasl~ion arc Ilc1d'in a'special.
                                                                                Because of the difficulties asso~ iated with excrcising co ntrol
           emergency library from where they should be m,ovcd through the
                                                                                over programming maintenan ce, activities, some org~nizati oll s
           cha nge management process intoJlonnal.productioll Jibrarics in mf
                                                                                implement configufntion management ~yst ems, in a
           expeditious mUIlI'cr. IS audi tors ~eed to pay particular atteillion tlla't
                                                                                coilfiguratiol1l1lanagell1ent system, mai ntenance req uests must
           emergency changes are handled in an appropliat~ malll~cr.
      .,                                                                        be formally documented and approved by a change ,control
:~                                                                              group (e:g., configuration control boards). in add ition, cHrefu l
           Deploying Changes Back Into Production                             , c~ntrol is exercised ovc"r"ea'ch stage orthe maintenance process
           Once use:r management has approved the chan ge, the modified         via checkpoints, reviews and sign-off procedures. FrQIll an audi t
           programs can be moved into the production environment. A group perspective, effective use of this software provides important
           that is independent of colllputcr programming sho uld perform th e evidence of managemen t's comm itment 10 careful control ovcr

           migration of programs frollltest to production. Groups such as       the maintenance process.
           com puter ope rati ons. QA or a designated change control group
~          should perform this fUllct ion.                                      Configura ti on management ilwolves procedures throughout thc
                                                                                         software life cycle (fromTequirements tina lysis to mainte nance)
           To enSlire that only authorized indi viduals have tbe abi lity to             to ident iry. define nnd baseline software Itcms in the system
           migrate programs illto production, proper access,restrictions Illust          and thus provide a pasis for problem management, change
           be in place, Such restrictions can be implemented through the lise            ll1anagelllcnt' releas~ managcment.
           of operating sy~te m secmit y or an external securit y packa¥e.

           CISA Review Manual 2011                                                                                                                       215
                                                                     -.........,. .         '.

  Section Two: Content
                                                          Chapter 3-lnformation Systems Acquisition,
                                                                           Development and Implementation                         ~C~IS~A    Certmed lnlormalion
                                                                                                                                             Systems AuditQf"
                                                                                                                                             ........ J.-

 Configurcuion manage ment is important because software is                   3.11 SYSTEM DEVELOPMENT TOOLS AND
 subject to ongoing change both duri ng and after developmcnt.
 The process involves identification of the items that afe likely to          PRODUCTIVITY AIDS
 change (called configuration items). These include thi ngs such
 as programs, documcntation and data. Once an item is developed               System development tools and productivity aids include code
 and approved, it is handed over to a configurat ion managemcnt               generators, Computer-Aided Software Engineering (CASE)
 team for safekeeping and ass igned a reference number. Once                  applications and fourth-generation languages (4GL). These too ls
 baselined in this way, an item should only be changed through a              and aids are addressed in the fo llowing sections,
 formal change control process.
                                                                              3.11.1 CODE GENERATORS
 Checking in is the process of moving an item to the cOll trolled            Code generators are tools, often incorporated with CASE
 environment. When a change is required (and supported by a                  products, thm generate program code based on parameters
 chnnge control form), the configuratio" ,manager wi ll check
                                       n                                     defined by asystems an;tlyst or on data/enti ty. now diagrams
 out the item, Once the cha nge is.made, it can be checked using             developed by the design modu le ofa CASE product. These
 a different version number. The process of checking out also ·'             products allow most deve lopers to impiemenl software programs
 prevents or mal1ages simul taneous coqe edits.                              wi th efficiency. The rs auditor should be aware of nontradi tional
                                                                           . ori gi ns of source code.
 For con0guration management to work, management mllst suppon
 the cOllcept of configuration llianagcmenl. The configuration
                                                                             3.11.2 COMPUTER·AIDED SOFTWARE ENGINEERING
 management process is implemented by developing and following
 a confib'llrat ion managem.cnt plan and operating procedures. This          Application development efloJ1s require collecting, organizi ng
 plan should not be limited !o just the software developed, but should       and prcscming a substantial amount of'data at the application,
 also incl ude all system documentation.. test plans and procedures, As      systems and program levels, A substantial amount of the application
 part of the solhvare configuration management task, the maintainer          developmellt eftort involves translating this infonnation into
 performs the following task steps:                                          program logic and code for subsequent testing, modification and
 L Develop the configuration managel'nent-plan,                              implementation, This often is n time consuming process but it is
 2. Baseline the code and associated documents.                              neCeSS<lly 10 develop, use and maintain computer applications.
 3. Analyze and report 011 the results of configuration .control.
 4. Develop the reports that provide configuration status                    Computer-aided software engineering (CASE) is the use of
     information,                                                            automated tools \0 aid in the software development process,
 S, Develop release pr.ocedures.                                             Their use may include the application of soft wa re tools for
 6, Perform configuration control activities such as identification          software requirements capture and analysis, software design,
    ·and recording of the request.                                           code production, testing, doc um~ nt ge neration and other software
 7. Update the configuration stat~ls accounting database.                    development acti vities.

 Illl11any cases, commercial software products will be used to               CASE products are generally di vided into three categories:
 automate the manual processes, Such tools should allow control              I. Up per CASE-Products used to describe and document
 to be maintained for applications software from the outset of                  bus iness and application requirements. This information
 system analysis and design to running live.                                    includes data object definitions and relationships, and process
                                                                                definitions and relationships,
  Configuration management tools wi ll support change                        2. Middle CASE- Products used for deve loping the detailed
  management and release managclilc,!lt by providing automated                  designs. These include se re~n and report. layouts, editing
  support fat the fo llowing:      .                                            criteria, data ·object organi zation and process nows. When
. I . I~enti fication of items affected by a p~oposed change to assist          clements or re lationships change in the design, it is necessary
      wi th impact assessment                 .        .                        1'0 il1ake, only . ninor alterations to t!w. f.llllomated design an.d al l
  2.. Recording cO!lfigurJlion itenlS affected by ~uth o ri zed Gh~nges         other relationships ate au tomatically updated.
  3, lmpleme'ntation' of changes in accordance wit h aut horization '        3. Lower CASE- Products involved \vith the generation of '
      records                                                                   program code and database definitions. These products use
  4', Registering of. configuration item changes. when authorized               detailed design infomlation, programming ru les and database
      changes and releases are implemented '                  .                 syntax rules .lo generate progra m logic. da ta fi le formats or
 ·S. Recording of baselines that are related to releases (with known            ent ire app lications.
      consequences) to which an organ ization would revert if an
      implemented'change fa ils                                             Some CASE products embrace Iwo of these categories or all three
  6. Preparing a release to avoid human errors and resource costs           of them.

 A new version of the system (or builds) should only be built from
 the basel ined items,
                                                                            CASE tools are ava'ilable for mainframe, mi nicompu ter and
                                                                            microcomputer environments, These tools can prov higher
                                                                            qual ity systems lllore quickly, CASE products enforce ,) uniform
                                                                            approac h to system development, facilitate storage and retrieva l
                                                                            of documents. and reduce the manual effort in developing and                           •
 216                                                                                                             CISA Review Manual 2011
           ~c~'s3A    Certirled Information
                      Systems. Auditor'
                                              Chapter 3- lnformation Systems Acquisition,
                                              Development and Implementation                                                  Section Two : Content
                      "''''''''' '''~


,..1                                                                                   data file, then sort the file and finally produce the report. A
           presenti ng system design information. This power of au10m at ion
           changes the nature orthe development process by eli minati ng               typical 4GL treats the report as an object with properties, such
           or combining some sleps and aileri ng the means of verifying                as input fi le name and sort order, and methods such as sort file
           specifi cal ions and applications.                                          and print report.

           The IS audi to,r needs to recognize the changes in the development          Care should be taken when llsing 4GLs. Unlike traditional
           process brought on by CASE. Some CASE systems allow a                        languages, the 4GLs can lack the lower level de tai l conunands
           proj ect tcam to produce a complete system from the DFDs                    necessary to perfor m certain types of data intensive or online
           and data elements without any lradilio nni source code. lnl hese            operations. These operations are usually required when
           sima tions, the DFDs and data elements become the source code.              developi ng major applications. For this reason, the use of 4GLs
           The IS auditor should gajn assurances lhal approvals arc obtained           as development languages should be weighed carefully against
           fo r the appropriate specifications, users continue to be involved          tradi tional languages already discussed.
           i"n the deve lopment process, and invest ments in CASE tools yield        • Environmental independ ence (portability)- Ma ny 4GLs
           benef.its j l) quality and speed. Other key issues the IS auditor           are portable across compu ter architectures, operating systems
           needs to consider with CASE incl ude the followi ng:                        and telecomm unicati ons moni tors. Some 4GLs have been
           • CASE tools help in the app lication design process, but do not            implemented on mai nframe processors and microcomputers.
           ensu re that the design, programs and system are correct or that          • Software facilities- These faci lities include the ab il ity to design
           they fully meet the needs of the organization.                              or paint retrieval screen formats, develop computer-aided tmini ng
           • CASE tools should complement and fit imo the application                  routines or help screens, and produce graphical outputs.
              development' methodology, but there needs to be a project              • Programmer workbench co ncepts- The prog rammer has
              methodology in place for CASE to be elTective. The                       access through rhe terminal to easy fil ing fac ilities, temporary
              methodology shou ld be understood and used elTective ly by the           storage, text edit ing and operating system commands. This type
              organization's software developers.                                      ofa workbc nch approach is closely associated with the CASE
           • The integrity of datu moved between CAS E products or between             :lpplication deve lopment approach. It is often refei'recJ to as 3n
              lnall l1a l and CASE processes needs to be monitored and                 integrated.development environment (IDE).
              controlled.                                                            • Simple langua ge subsets-4GLs generally have simple
           • Changes to 'the application should be rcnected in stored CASE             language subsets that can be lIsed by less-ski lled users in an
              product elata.                                                           information center.
           • Just like a traditional application, appl ication Controls need to be
              designcd.                                                              4GLs are ()fien classified in the following ways:
           • The CASE repository (the database that stores mid organ izes
                                                                                     • Query and report ge nerators- These special ized languages
              the documentation, mode ls and other outputs from the difTerent
                                                                                       can extract and produce reports (audit software). Recently,
              phases) needs to be secured on a need-to-know basis. Strict
                                                                                       more powerful languages have been produced that can access
              version control should be maintained on lhis database.
                                                                                       database records, produce comp lex online outputs and be
                                                                                       develppcd .in an almost natural' language.
           The IS auditor may also become a 'user of CASE toolS as several
                                                                                     • Embedded database 4GLs- These depend on self·co ntained
           feahlfes fac ili tate the audit process. DFDs, wh ich may be the
                                                                                       database management systems. This characteristic often makes
           product of upper and middle CASE tools, may be used as an
                                                                                       them more user-fri end ly but also may lead to applications that
           alternati ve to other flowcharting techniques. IS aud itors whose IS
                                                                                       are not in egrated well with other producti on ap plications.
           departments are moving into CASE are using CASE-gene rated
                                                                                       Examples includc FOCUS, RAM IS II and NOMAD 2.
           doc umentat ion part ofule audit. Some are even expe rimenting
                                                                                     • Relational da tabase 4GLs- These high·levellanguage
           wit h the lise of CASE tools [Q create audit documentation.
           In addition, CASE tools can be used to develop interroga tion               products are .usua lly an optional feature on a vendor's DBMS
           softwa re apd eh)bedded aud it modules (EAMs). Repository                   product I.ine. These alld\v the applications deve loper to make
           reports should 'be used to gai n an u nderSHUl~ iJlg of the system an.d     better use of the DBMS prodlict, btu they often arc not .
           to rt:;view contro ls.over the development 'Process.                        end-user_qriented. Examples include SQL+, MANTIS and
                                                                                       NAtURAL.                  .
                                                                                     • Application ge nerators- Tbese deve lopment tools generate
           3.11.3 FOURTH·GENERATION LANGUAGES                                          lower·level programming. languages (3GLs) such as COBOL
 ,j         Wh ile a sta ndard definition ofa 4G L docs not ex ist, the COIlUllon      and C. The application can be furthertailored and customized .
          . charactcris.tics of4GLs are the following:                                 Data processi ng development personnel, not end users, lise
            • No nprocedur al language- Most 4GLs do not obey the                      applicati on genera tors.
            procedura l pa radigm of continuous statement exec ution and
            subrout ine call and con trol strucnlres. Instead. they are eve nt-
            dri ven and make ex tensive use of object-oriented prog rammi ng
            concepts such as objects,. properties l:llld methods.                    3.12 PROCESS IMPROVEMENT PRACTICES

             For examplc. l:l COBOL programmer who wants to produce a                Business processes .requi re improvements, which arc accomplished
             report sorted in a given sequence must first open and read the          with prnctices and tec hniques addressed in the following sections.

            CISA Review Manual 2011                                                                                                                    217
Section Two: Content
                                                          Chapter 3-lnformation Systems Acquisition,
                                                                     Development and Implementation                          ~C:IS~A   Systems inlormalion
                                                                                                                                       Certified Auditor·

3.12.1 BUSINESS PROCESS REENGINEERING AND                                     BPR is the process of respondi ng to competifive and economic
                                                                              pressures, and customer demands to sur vive in the current
                                                                              business envi ronment. This is usually done by automating system
"Business is in reality the unde rlying business process and                  processes so that there are fewe r Illanual interventions and
sllccessful business implies eO"icient underlying business processes.         manua l contro ls. aPR achieved with the help of implememing an
Evclything else thm constitutes a busines5 is losing its uniqueness           ERP system is often referred to as packHge-enabled reengineering
and becoming a commodity. It is no wonder that the efficient                  (PER). Advanlagcs ofBPR are usually experienced where the
organization of processes is becoming a boardroom agenda the                  reengineeri ng process appropriately suits the business needs. BPR
world over." (Nagaraj, N.S.; Business Process /l,Ilflllagemel1l- An           has increased in popularity as a method for achieving the goal of
Emergillg Trelld, InfosysiSETLabs, India, 200 1)                              cost savings through streamlini ng operations.

A busi ness process can be seen as a sociotcchnjcal system, that              The steps in a sLlc"Cessful EPR are:
is "n set of interrelated work activities characterized by speci fic          • Define the areas tQ be reviewed.
inputs and va lue-added tasks that produce specific clIstomcr-                • Develop a project plan.
foclIsed outputs. Busi ness processes consist of ho rizontal work             • Gain an understanding of the process under review.
!lows that cut across several departments or functi ons." (Seth,              • Redesign and strcCllll~ine the process.
Vikram; William King: Organizational Trans/orlllatioll rTiro//gh              • Implement and monitor the I1e\v process.
B//siness Process Reel/gineerillg. Prentice I·fel ll , USA, 1998)             • Establish a continuous improvement process.

The generic model shown in ex hibit 3.26 is a very basic                      As a reengincering process takes h ol (~ ncw results begin to emerge:
dcscription or a process (some form of information enters the                 • New bus! ness priorities based ·on value and customer requirements
pr0cess, is processed. and the .o utcome is measured against the              • A concentration 0 11 proc;ess as a mea ns of improving product.
goal or obj ecti ve of Ihc process). The level of detail needed (e.g ..         service a l~d profitab ility     .
breakdown of subprocesses to activities) depends hi ghl y all the             • Ncw approaches to organizing aile! moti vating people inside and
complexity of tile process, the knowledge of the alYected stan'                 outside tIle enterprise
and the company's requirements regarding aud it fUllctionality                • New approaches to the use of technologit:s ill developing,
(performance and complia!lce) of the process and shall fit inJo an              produci ng (\nd deliv~ring goods and services
ex isting qualit y management. system (e.g., ISO 900 1:2000).                 • New approaches .to the lise of informmioll as well as powerful
                                                                                and more accessible information technologies
Any output produced by a process must be bound to a busi ness                 • Refined ro les for suppliers including outsourcing,joint
objecti\'c and adhere to defined corporate standards. Monitoring                devcl.opment, quick responsc,jllst-in-time inventory and suppor!
of effectiveness (goal achievement), efficiency (minimum effort)              • Redefjned roles for clients and customers, providing them
and compliance mllst be done 011 a regular basis and shaH be                    wi th more direct and active participation in the enterprise's
included in lll8nagement reports for review under the pJan-do-                  business process
ch"ck-act (PDCA) cyc le:

                                                        Exhibit 3.26-Generic Process Model

                                                          Process             Process

                        Process Control
                                                           Owner        I      Goal
                                                              Process Parameters
                                                              (Quality. KPls. etc.)

                                                          D                    D                                                                             I

                           (Defined) Input
                                                          Subprocesses or Activities
                                                                                                    (Define Output

                                                          D                    D                                                                                  •
                      Process Enabler
                                                    I     Resources            Roles
                                                                                                                                                             I·   •
218                                                                                                          CISA Review Manual 2011
     ~c~'s3A'   Systems,Inlormatioo
                Cer1ilied Auditor"
                                        Chapter 3-lnformation Systems Acquisition,
                                        Development and Implementation                                                    Section. Two: Content
                .. """( .. \ ... ·1..

     A successful BPRlprocess change project requires the project               class "reference" in a globalized world. Reference producl~ services

..   team to perform the following for th e existing processes:
     • Process decomposition to the lowest level requ ired for
       effecti ve ly assessi ng a business process (Iypically referred to flS
                                                                                or processes are syslcl11atically analyzed for one or more of the
                                                                                followi ng purposes:
                                                                                • Comparing and ranking
       an eiemcl1!ary process), which is a unit of work performed with          • Strategic planning, SWOR (strengths, weaknesses, opportunit ies
       a defini tive input and output                                             and risks) analysis
     • Ide ntifica ti on of customers, process-based managers or process        • Investment decisions, company takeovers, mergers
       owners responsible for processes fro m beginn ing t6 end                 • Product 'Of process design or redesign/reengi neering
     • Doclimentation orthe elementary process-related profile                  • BPR
       information includi ng:
      - DUr<lIion .                                                             The steps listed below are followed generally by the
      -   Tri gger (which triggers the process (0 act)                          benchmarking 'team in a benchmarking exercise:
      -   Frequency                                                             1, Plan- In the planning stage, critical processes are identified
      -   Effort                                                                   for the IJencrunark ing exercise. The benchmark ing team should
     ·-   Responsibility (process owncr)                                           identi fy the critical processes and understand how they are
      - Input and          OlltP~lt                                                measured, the kinds of data that are needed and how the data
      -   Ex terna l interfaces                                                    need to be collected.
      -   System interaction                                                    2. H.cscarch- The tea m should collect baseline data aboLlt the
      -   Risk and contro l information                                            processes of its ow n organization before collecting these
      -   Performance measurement informat ion                                     data about other organ izati ons. Thc next step is to identi fy
      -   Identified problematic areas and their root causes                       the reference products or co mpanies through sources such as
                                                                                   business newspapers and magaz ines, qua lity awa rd win ners,
     The ex isting baseline processes must be documented---:.prcfcrably in .      ·trade journals, co nsul tants. etc. Depending 011 the tea m's own
     the form of nowchmts and related profile documents- so the baseli ne          preferences and rcso ui·ces. and all the marketplace. several
     processes can be compared to the processes after              scenarios may res ult :
                                                                                  - Benchma rks that satisfy the orga nization'S interest already
     The m:wly designed busi ness processes inevitably inv:olve                     exist at no charge from professional associations,journals
     chan ges in the way(s) of doing business, and coul d impact the                or analysis nn ns (e.g., Auerbach, Compllfen ro r/d,
     fi nances. phi losophy and perso nnel of the orga nization, its                FeedbackMet rics, FOI'rester Research Center, Gart ner, IDC,
     business partners and customers.                                               JOG, US Census Bureau).
                                                                                 - The organization may join or promote a sur vey lau nched
     Throughout th e change process, the BPR teamlllust be sensitive                by a single. or I11lll ti·i ndustry specialized web porta l (e.g., a
     to organization culture, structun:;, direction and the components              bookmark portal).
     of cha nge. Managemen t must also be able to predict andlor                 - The organization may'cond uct or subcontract busi ness
     ant icipa te issues and problems, and offer appropriate reso lutions           in tell igence ..
     that wi ll accelerate tlie change process.                                  - The organization may enter il1lo an agree ment with olle or
                                                                                    marc "bcnclullark partners" who agree to share info rmation.
     BPR temllS can be used to faci litate and assist the staH'in
     transitioning into the reengineered busi ness processes. BPR                 Depending on the aims of the exercise and the resulting scenario
     professionals are valuable in moni toring progress IowaI'd the             · above, the nex t steps wi ll be ski pped or adapted.
     achievemen t of the strategic plan of the organization.
                                                                                3. Obscrve-The nex t step is to collect data and visit the
     A majoi' c011cern in BPR is that key.cont ro ls may be reengineered           benchmarking parlner. There should be an agreement W           'it!llhe
     out of a business process. The IS auditor's task is to identify.              partner organization, a data collection plan and a method to
     the existi ng key.contr:ols and eval uate the im pact ofremoving              fac ilitate proper observation:
     these controls: If tbe controls ~re key pteve ntive.contro ls, the IS      4. An'alyze-This.step involves stllllli1arizing an d·interp reting tile'
     ,llid itor Ilwst cnsure that nianag'ement is aw<)re or tile renlova l         data collected, al1(1analyzing the gaps between an organi7..atiOli's
     orlhe co nt rol and management is willing to accept the P9tential             process and its partner's process. Converting key findings into
     m a teri~l risk of not having that prevent ive control.                       new operntional.goals will be the goal of this stage.
                                                                                5. Adopt- Adopt ing the results ofbcllchmarking C~lJ1 be the
     BPR Methods and r~chniques                                                    IllOSt difficult step. In this step, the teail1 needs lO translate
     App lying B'Plfme thods a;ld techniq ues to a process creates an              the findings into a few core principles and work down from
     imme'dia te environment for change and provides consistency of                principles to strategies to aClion plans.
     results.                                                                   6. Improve- Conti nuous improvement is the key focus in a
                                                                                   benchmarking exercise. Benchmarki ng links each process in an
     BENCHM ARKING PROCESS                                                         organization. wit h an improvement strategy and organ iza tional
     Bellchlnarkillg is about improving business processes. It is defined         F~ L                                                         .
     as a COlltillllO systematic process for evaluating the products,
     services or work pr.ocesses of organizations recognized as a w01:ld-

     CISA Review Manual 2011                                                                                                                        219
 Section Two: Content
                                                           Chapter 3-lnformation Systems Acquisition,
                                                                             Development and Implementation                    ~c~'s3A' "'m...,.....
                                                                                                                                          Systems AudiIOf'

BPR Audit and Evaluatioll                                                         learning environment where succcssfu lly defined and appl ied
When reviewing an organization 's business process change                        processes can be repeated sllccessfully on other projects of
(recngincering) efforts, .lS audi tors Illust determine whether:                 simi lar size and scope.
• The organizalion's chan ge efforts arc consistent with the overall          3. Defined- Lessons learned from the prior phase provide the
  culture and strategic plan of the organi zation                                impetus to develop a standard software process ac ross the
• The reenginecring team is making un effort to minimize any                     organiza tion. This includes both management and software
  negative impact the change might have on the organization's sta H-             engincering ac tivities, doc llmcnte~ standardized and integrated
• The BPR team has documented lessons to be learned after the                    into an institutionalized standard software process applicable to
  completion of the BPRJprocess change project                                   all software development projccts.
                                                                              4. Managed- Once processes are we ll-defined and applied,
The IS auditor would also provide a statement of assurance or                    the organi za ti on has reached a point where it can develop
concl usion with respect to the objec ti ves of the audit.                       and apply quantitative managed cont rol over its software
                                                                                 deve lopment processes. This provides a grea ter dcgl:ce'of
3.12.2 ISO 9126                                                                  precision and control over software projects for i!nprovi ng
                                                                                 software productivity and reaching zero-defect goals,
ISO 9 1 is an international standard to assess the quality of
                                                                              5, Optimizing- When an organizati on has attaincd the ability
software products. It provides the defi ni tion of the cilaracledstics
                                                                                 to quantitati ve ly ancISUccessfu lly control its softwa re I)rojccts,
and associated quality evaluation process 10 be used when
                                                                                 it is in a posi ti on to use continuous process improvement
specifyi ng the rcquircmcJHs for. and evaluating the quality o[
                                                                                 strategies in app lying innovative solutions ~nd 's tate-of-the-art
soltware pl'Oducts throughout their life cycle. Attributes eval uated
                                                                                 techn ologies to its sofhvare processes.
• Flillctiona lity- The set of att ribu tes that bea rs 0 11 the existence
                                                                              Improvemcnt act ivities pe rform ed by an organizatioll
  or a set of functions and their speci fi ed propert ies. The
                                                                              maturity levels' two through. five. The starting 'point is one level
  fun ctions are those that satisfy statcd or im plied needs.
                                                                              above the orga ni zation's curren t maturi ty level.
• Reliability- The set of attributes that bears on the capabil it y
  of sol\warc to maintain its level of perfonnallce unde r stated
  conditions for a S lated period of time.                                    3.12.4 CAPABILITY MATURITY MODEL INTEGRATION
• UsabiJity- The set of attri butes that bears on the etTor! needed          Following the release and successful adoption of CMM for
  for lise and on the individual assess ment ofsllch usc by a st.ated       Software, other models were developed for disciplines such as
 .or implied set or users.                                                  systems engineering. integrated product deve lopment, etc. The
• Efficiency- The set of att ri butes that bears all the relationship       Capability Maturity Model Integration (CMMI ) was conceived as
  betwee n the level ofperformancc of tile software and the                 a means of combining the variolls models into a set of integrated
  amount of resources used under stated conditions.                         models. CM MI also describes five levels of maturity. a lthough
• Main lain ab ilit'Y- The set of attributes that bears on the elTort       the descriptions ofwllat consti tutes each leve l differ frqm those
  needed to make specified mod ifications.                                  used in the origi nal CMM. Supporters ofCMMI it as less
• Portability-,-The set ofattributcs thm bears Oil the abil ity of          di rectly aligned with the trad itional watc rfall approach toward
  software to be transferred from Olle environment to another.              deve lopment and better aligned with contemporary software
                                                                            development practices including:
 3.12.3 SOFTWARE CAPABILITY MATURITY MODEL                                  • Iterative development
                                                                            • Early dcfinitioll of architecture
 The Ca pabil ity Matulity Model (CMM) for Softwarc. deve loped
                                                                            • Model-based design notation
 by Carnegie Mellon's Software Engineering Institute and va riolls
                                                                            • CQmponent-based deve lopment
 indu ~·try and govemment affi liates in the early 1990s, is a process
                                                                          . ~ Denionstration-bascd assessment of intermediat,c deveJopment
 maturity model or framework thal.helps organizations il1~prove the ir
                                                                               products         .
 softwa!'c lifc cycle processes. The model is particularly adept at
                                                                            •.Use or syalablc,.co nfigurable processes
 enabling organiz.1tiolls to prevent excessive project schedule delays.
                                                                        .            '.      .                   .
'and cost overruns by providing the apprQ lidtc'infi'astrut ture and
                                                                          . Mamrity models, such .as·CMtyt l, are use hI I to evali;ate
 SUPPOt1 necessary to help·projects avoid the~ problems:
                                                                            ma;lagemcnt of a computer center, the deve lopment fUllction
                                                                            management proces$, and implement and measure the IT change
 8J1Secl all quality process ·managcment princ;iplcs of five maturity
                                                                            lilanagelnent process. ISOIIEC 15504'(SPI CE)- see nex t
 l ev~ l s, the t~iM was designed to gu ide software organ izations
                                                                           .section- intet:nationally stalldardizcs matlllily models.
 in selecting process improvement stra tegies by determ ining
 cu rrent process maturity and ide nt ifying the few issucs most
                                                                            See exhibit 3.27 ror characteristics ofthcm at urity levels.
 criti cal to software qual ity and process improvement. This enables
 orgailizations to focus on a limited set of ac tivi ties to steadily
 improve sofiware process capability. The five maturi ty levels             3.12.5 ISO 15504
 atta inable by software organizat ions include:
  I. In.itial- Charactcrized as ad hoc, where success depends
                                                                            ISO/IEC TR 15504. aJso Imown as Software Proccss
                                                                            Improvemen t and Capability Detertllinmion (S PICE), is based
     solely Oil individual etTon
 2. Rcpcatable- Discipliilcc! management processe~ are
                                                                            on CMM and Bootstrap and was first published in 1998 (ISOI
                                                                            IEC TR 15504: 1998). Based on the feedback of more lhan 4.000
     established to plan and track cost, schedule and functionality,        assessments. it h<ls been republished during 2003-2005 as a full
     l.1l1d provide oversight over a software project, and create a         internationa l standard.
220                                                                                                            CISA Review Manual 2011
              ~C~IS~A   Systems,kl lormation
                        Ceflirle(! AOOitof'
                                               Chapter 3-lnformation Systems Acquisition,
                                               Development and Implementation                                                                Section Two: Contem

                                                             Exhibit 3.27-Characteristics of the Maturity Levels (CMMI)

                                                                                                         Level S             Focus is on process
                                                                                                        Optimizing           improvement.

                                                                                      Level 4
                                                                                    Quantitatively         Process is measured
- ~~
                                                                                                           and controlled.
   .:.:                                                                              Managed

                                                                          Level 3           Process characterized. for the organization and is
                                                                          Defined           proactive. (Projects tailor their process from the
                                                                                            organization's standard.)

                                                            Level 2
                                                           Managed '          ProcessJs characterized for projects ana is often' reactive.

                                                Level 1
                                                 Initial       Processes are unpredictable. poorly controlled and reactive.

               Source: Adapted from NASA presentation on: Using CMMllor Improvement at G5FC, Systems Engineering Lecture 5eiies, USA, 1 June 2004,
      1_ Godlreyppl

              SP ICE, like e M!'",I!. serves as a baseline ror process improvement              input, processing and Oll~ put fUllct ions, They include methods for
              and process benchmarking, and leverages the adopt ion of best '                   ensuring that:
              practices, Reference models ror. SPICE incl ude:                                  • Only comp lete, accurate and valid data are entered nnd updated
              • Soft ware life c)'c1 e processes- Through ISO/ IEC 12207                          in a computer system
                AMD I/2                                                                         • Processing accomplishes the correct task
              , Sys tem life cycle processes- Through ISOIIEC 15288                             • Processing results meet ex pectations
              • Hum a n-ce ntered life cycle processes- Through ISO 18529                       • Data are maintain'ed
              • Component-based development processes- Tlu'ough the ,
                OOSP ICE project
                                                                                               , Application controls may consist of edi t tests, totals, reconcil iat ions,
                                                                                                 and identification and rep0l1illg of incolTcct, missing or exception
              · IT se rvice man agement processes- Through a SPICE user
                                                                                                 data. Automated controls should be coupled with manual procedures
                group initiat ive
                                                                                                 to ensure proper investigation of exceptions.
              • Qua lit y ma na ge ment system p" ocesses- Through SPICE for
                9000 (S9K)
                                                                                        These controls hclp ensure da ta accuracy, completeness, va lidity,
              • Autom otive embedded softwar e- Th rough Au tomoti ve
  ::"           SPICE (a n initi at ive of The Procurement Forum and The SPICE
                                                                                        verifiability and consistency, thus ach ieving data integrity and
                                                                                        data re liability. Implementati on of these controls hclps ensure
                User GrollP \vi th the major European car manufacturers)             , ,system integrity, that appl icnble system functions operate
              • iVl edical device software- Through ihe Medi SPICE i~it i at i ve       as intended, an~ that informritiOir contained by the system is
                                                                                        re l e\~ant,, secure and rivai l a~~e when nceded.
              The process ~a pabi lily 'dimensions ar~ sl!ghtly ~ i fTcrel) 1 from
              those ltsed ·in·CMM J'(see exliibH 3.28).                                 The (S aud itor's t as~s include rhe followi ng:
                                                                                        • Jdentifying the significant application componcnts and the
              A good schen1atic descri ption 'of the 5 levels is presented in .           now oftrahsaclions thi'ough the system, and gaini ng a detailed
              ex hibit 3.29 .                                                         , 1I1lderstanciilig of the appl ication by reviewi!lg the avai lable
    <                                                                                     documentation and interviewing appropriate personne l
    i.        3,13 ' APPLICATION CONTROLS                                               • Identifying the application control strengths, and eva luating the
                                                                                          impact of the control weaknesses
                                                                                        • Developi ng a testing strategy
     •        Application controls refer to the transacti ons nnc! data relating
     iO       to each computer-based application system: therefore, they
                                                                                        • Testing the controls to ensure their funct ionality and
                                                                                          effectiveness by applying appropriate audit procedures

          •   are specific to each application, The obj-ectives of application
              controls, wh ich may be manual or prog raJllme(~ are to ensure
                                                                                        • Evalutlling the control environ ment by analyzi ng th e test res ults
                                                                                          nnd other audit evidence to detenni nc that control obj ec ti ves

          •   the com plete ness and accuracy of the records and the va lidity of
              the entries made t he re il~ , Application contro ls nrc controls over
                                                                                          we re achieved

          •    elSA Review Manual 2011

                                                                    Chapter 3-lnformation Systems Acquisition,
                                                                                                                                                   ~C:IS~A      Certified Information
                                                                                                                                                                ........ ,
Section Two: Content                                                           Development and Implementation                                                   Syslems,AuditOf'

                                                                    Exhibit 3.28-150 15504 (SPICE)
                   The process is continuously improved to meet
                                                                                                     level 5 Optimizing
                                                                                                        PA.S.l       Process Innovation

                   relevant current and projected business goals.
                                                                                                        PA.S.2       Process Optimization

         -         Predictable                                                        level 4 Predictable
                  The process is enacted                                                  PA.4.1    Process Measurement
                  consistently within defined limits.
                                                                           ~              PA.4.2    Process Control

                                                                       level 3 Established
                                                                          PA.3.1      Process Oefinition

                  A defined process is used based on
                                                                          PA.3.2      Process Deployment
                  a standa rd process.                                                      . '

                                                        level 2 Managed                                     Managed
                                                           PA.2.1     Performance Management                The process is managed and
                                             ~.            PA.2.2     Work Product Management               work products are established,
                                                                                                            controlled and maintained.

                                          level 1 Performed                                 Performed
                                            PA.l .l     Process Performance                 The process is implemented and

                                                                                            achieves its process purpose.

                          level 0 Incomplete                                 Incomplete
                                                                             The process is not implemented
                                                                             or fails to achieve its purpose.

Source: Interspice ltd. © 2006. AI1 rights reserved. Used by permission.

                                                           Exhibit 3.29--15015504 (SPICE) Clarification

                          ·Level 5

                          Level 4
                                          0--           LL-y--.-"

                          Level 3
                                          0--           ,"-..i-L--"

                          L 2
                                          - --          '---_--'

                         . Levell                                                                                                        OUT

Source: M.C. Paulk, C.V. Weber, B. Curtis, and M.B. Chrissis, The Capability Maturity Model: Guidelines for Improving tile SoftWare Process, Addison-Wesley, Boston, 1995

                                                                                                                                 CISA Review Manual 2011
 8          Certified Inlormation
   CI A Systems, Auditor'
            w ___   t.l._
                                    Chapter 3-lnforma tion Systems Acquisition,
                                    Development and InJplementation                                                      Section Two: Content

  • Consideri ng the operational aspects o rthe app lication to ell,sure        Ideally, sou rce documents should be preprinted forms to provide
    its efficiency and effectiveness by comparing the system with              consistency. accuracy and legibility. Source documents should
    efficient system design standards, analyzing procedures lIsed               include standard headings, ti tles, notes and instmctiolls. Source
    and comparing them to management 's objectives for the system              document layouts should:
                                                                               • Emphasize ease of use and readab ility
 3.13.1INPUTj ORIGINATION CONTROLS                                             • Group simi lar fields together to faci litate input
 Input control procedures must ensure tha t every transaction to               • Provide prcdetennineq input codes to reduce errors
 be processed is entered, processed and recorded acc Ul~te ly and              • Contai n appropriatc cross·refe rcllcc numbers or a comparable
 completely. These controls should ensure that only v<'I I,leI and                idcnti fier to faci litate research and tracing
 authorized information is input and that these transactions are               • Usc boxes to prevent fie ld size errors
 only processed once. In all integrated systems environment,                   • Include an appropriate area for management to document
 output generated by onc system is the input for anot her sy~tem.                authori zation

 Therefore} the system receiving the Olll put of anoth ~r s~stem as            AI/ sou rce documenls should be approprialely conlrol/ed .
 inpuUoriginatioll Illust in turn apply edit checks: validati ons and          Procedures should be established to ensure that all source
 access controls to those data.                          .                     documents have been inpm and taken into account. Prenurnberillg
                                                                               source documents facifiTIifcs th is control.
 Input Authorization                            .
 In put authorization verifies that all transactions have beell                Batch Controls and Balancing
 aut hori zed and approved by management.                                      Batch controls group input transactio ns to provide contro l totals.
                                                                               The batch contro l can be based on tota l mOI1~tary amoun t, total
Authoriz<.l tion of input helps ensure Ihal only aut horized data .            items, lotal docu ments or hash totals.
arc entered fo r processing b applications. Au th orization CUll be
perronned on line at the tillle when the data arc entered into. t!lC          Batch header forms are a data prcparation cont rol. All input
system. A computer-generated repon listing Ihe. it~ms req uJrlllg             fonns should be clearly identified with tile applicatioll llaille and
man ual authorizat iolllllay also be generated. It IS Importa nt thaI         transaction codes. Where possible, preprinted and prenlllllbcred
controls exist throughout processing to ensu re that the uut hori·zed         fo rms, wit h transaction id~nti ficatio n codes and' other constant
data remain unchan ged. This can be accomp lished t h rou~h          .        data items, are recommended. This wou ld help ensure that all
various acc.uracy an d completeness checks incorporated II1to an              pertinent data have been recorded on the input forills and can
application's design.                                                         redtlce data recording/entry errors,

 Types of autho rization include:                                    .         Types of balch controls includc:
 • Signatures 011 batc h form s or so urcc doc um ents- Provide                • Total moneta ry amount- Verificat ion that the total monetary
   evidence of proper <l uthorizati on.                                           value of items processed equals the total monetary vahie of the
 • Online access co htrols'-Ensure that only aut horized                         batch documen ts. For example, the to tal monetary va lue of the
    individuals may access data or perroI'm sensi tive fU llcti ons.             sales invoices in the batch agrees with the total monetary va lue
                                                                                 of the sales invoices processed.
• Unique passwords- Necessary to ensure that access
   authorization cannot be compromised through use of another                  • Total items-Verification that th e total number of items
   individual's authorized data access. Indi vidual passwords                    included on each document in the batch agrees wi th the total
   also provide account ability for data changes. (see                  llulll ber of items processed. For example, the total num ber
   5• Protecti on of In formatio n Assets, for controls regnrdlllg               ofullit s ordered in the batch of invoices ag rees with t~e total
   passi,vord use and stl1lcture.)                             .                 number of units processed. .
• Term inal or client workstatioit idcntifica tion--=-Used to                 ; Tota l docum ents- V      erification that the total number of .
   limit iilput to specifjc terminals'or workst~tions 3S we ll as to ' .         doc uments in the. bat~h equa ls the·tota l number of documeI}ts
   individuals. Teril1 ina ls or client workst~tioits in a network c~n be.   . processed. F~r example, the totaillumber.·of invoic~s in a batch
  co,;figured with a uiliq(~e form of'identifi.c?tion sllch ·as senal            agrees with the tota l mimber of i l1voi~es processed.
  number or computer name that is authe nt icated by the system.              • Ha'sh totals- Verification that th e tota l in a batc~ agrees with
• Source do clIm cnts- Tbe'forms used to re~o'rd data. A source                 the total calculated QY the system.

                                                                             ~atch                     ~e~for'm ed th~'ough
  documcnt Inay be a piece of paper, a tu rnaround document or an
  image.displayed .for an Iinc data input. A wc ll-designed source                    balancing can. be                        manual or a.utoniatcd
  document achieves scveral purposes. It inc reases the speed                reconcil ia tion. Batcn totaling must be comb ined with adequate
  and accuracy with which data can be recorded, controls work                 fo llow· up procedures. Adequate controls should exist to ensure
  f10\\~ fac il itates preparntion of the datu in machi nc·readable          that each transaction creates an input doc ument, all documents
  rorm for pattern recognition devices, increases the spced and              are included in a batch, all batches are submitted for processing.
  acc uracy with which data can be read, and fac ilitates silbsequent        all batches are accepted by the computer, batch reconcil iation
  reference checking.                                                        is pe rfOJ:ll1e(~ procedures for the illVestigation and timely
                                                                             correction of differences are followed, and contro ls ex ist over the
                                                                             rcsubmission of rejected items.

CISA Revi.ew Manual 2011
~."":""";_,     ,
              __ _.   ..:.!.'~"", ' .l'.:;,,   ......,, ••

   Section Two: Content
                                                             Chapter 3-lnformation Systems Acquisition,
                                                                        Development and Implementation                        ~C~IS~A    Certified AudItor·
                                                                                                                                         .. __·c...•....


  Types of batch balancing include:                                         • Anticipation- The user or contro l group anticipates the receipt
  ~ Batch registers- TIlese registers enable manual recording of batch        of data
    totals and subsequent comparison with system reported totals.           • Transmittallog- DoclItncnts transmission or rece ipt of data
  • Co ntrol acco unts- Control account use is perfomlcd throu gh           • Cancellation ofsollrcc documents- Procedures to c;:lIlcel
    an initial edit file to determine batch totals. The data arc then         source documen ts such as by punching with holes or marking
    processed to the master file, and a reconci liation is performed          them 10 avo id dupli cate entry
    between the totals processed during th e initial edit file and the
    master file.                                                            Batch Integrity in Online or Database Systems
  • Computer agree ment- Computer ag reement with batch totals              Online systems also require control over input. Batches may
    is performed tlu'ough the input of batch header details that            be eSlablished by the time of day, the specific termin al or the
    reco rd the batch totals; the system compares these to calculated       individ ual inputting the data. A supervisor should then review
    totals, ei ther accepting or rej ecting the batch.                      the online batch and release it to the system for processing. This
                                                                            method is preferred over review of the outptit by the same person

  Error Reporting and Handling                   _                          preparing the input.
  Input processing requires that controls be ideJlti fj~d to verify th at
  on ly corrccr data arc accepted into the·system and input errors are      3.13.2 PROCESSING PROCEDURES AND CONTROLS
  recognized and corrected.                                                 Processing procedures and controls are meant to ensure the
                                                                            reliability of applicat iOIl program processing. IS aud itors need
  Data cOllversion error correcti ons are needed duri ng the data           t.o understand the procedures and controls that can be exercised
  conversion process. Errors·can occur due to duplication of                ove r processing to eva luate what exposures are covered by these
  transactions and inaccurate data entry. These errors can, i·n turn.       cOl1t ro ls an d what ex posures remain.
  impact tllCcompleteness and accuracy of the data. Correcti ollS
  to data shou ld be processed through normal data.conversion
                                                                            Data Validation and Editing Procedures
  processes and should be·ver ifi ed. au thori zed and reentered into
                                                                            Procedures should be cswblished 10 ensure tlmt input da ta are
  the systelll as a part of the nonnal processing.
                                                                            va lidated and edi ted as close to the time and point of origi nat ion
                                                                            as possible. Preprogra l11111cd input formats ensure thm dala are
  Input er ror handling can bq processed ~y:
                                                                            input to the correc t field in the correct format. "if input procedures
  • Rej ectin g onl y tra nsacti ons ,,,ith errors- Only transact ions      allow supervisor ove rrides of data val idation ane!" editin g.
    co ntaini ng errors would be rejected; the rest Of lhc batch would      automatic logging should occur. A manager who did 110t initiate
    be processed,                                                           the override shou ld review this log.
  • Rej ecting the whole batch of transactions-:-Any bateh
  · containi ng errors would be rejected for correction prior to            Data \~alidation is meHll t to identify data errors, incomplete or
    processing.                                                             missing data and inconsistencies among related data items,
  • J-Iolding Ule. batch in .suspense- Any batches contai ni ng             Front·end data editing and validation can be performed if
    erro rs would not be rejected; however, the batch would be held         in telligent termi na ls are used.               .
    in slispense, pending correction.
  • Accepting the batc h and fl agg in g error tr3nsactions-                Edit controls are preve nt ive controls fha t are used in a program
    Any balch containing errors wou ld be processed; however,               before data are processed. Ifnol in place or not working
    those transactions con taining erro rs would be flagged for             effectivcly, the preve nt ive cont rols are not effective. This may
    identificati on,. enabling subsequent error correction.                 cause processi ng of inacc urate data. Ex hibit 3.30 describes
                                                                            vari ous types of data va lidation edits.
  Input. control tec hniques include:
  • Transaction log""T"""Colltains. a det~il ed list of aU·updates.         Processing Controls
    Th ~ log can· be ei.ther manually maintail)cd·or p·rovided              Processing controls arc n~eatl1 to ensure ·the completeness ·and
    through automatic computer logging. A trimsactioll ·log Can be          accuracy pf aCCuillulated·'fata, they WOliid ensure that ·data in a ft·leI
   -reco nciled to the Ilumber of source docu ments received to veri fy     database remain complete and accurate lintil changed as ~ result of
    that all transactions have been input.                                  authorized p[Qcessing or modification routines: rhe following are
  • Recorid fiatiori·of"data- Controls whether al l,data received are       processing'control techniques. that can be used to address the issues
    properly recorded and processed                                         of completeness and a~cu:acy of accumulated data:                     .
  • Documentation- Written evidence of user, data entry and ·data           • Manual reca lculatiolls- A sample of transactions in<lY
    control procedures                                                        be recalculated manually 10 ensure that processing is
  • Er ror correc tion procedures- These include:                             accomplishing the anticipated task.
    - Loggi ng of errors                                                    • Editing- An edit check is a program instructi on or subroutine
    - Timely co rrections                                                     Ihat tests the accuracy, completeness and valid ity of data. II
    :- Upstream resubmission
    - Approva l of corrections
                                                                              may be used·to control inpu,t or late r processing of data.
                                                                            • Run·to·run totals- Run·to-rul1 totals prov ide the ability to verify             •
    - Suspense file                                                           data values through the stages of application processing. RUlHO-
    - Error file                                                              rUlltowl verifi cation ensures that data read into the computer were .
    - Validity of corrections                                                 accepted and then applied to the updating prQ(:css.

  224                                                                                                         CISA. Review Manual 2011
            ~C:ls3A "'"........
                    .......... ,_
                                             Chapter 3-lnformation Systems Acquisition,
                                             Development and Implementation                                                                    Section Two: Content


                                                                   Exhibit 3,30-Data Validation Edits and Controls
                              Edits                                                                         Description
             Sequence                               The control number follows sequentially and any sequence or duplicated control numbers are rejected or noted on
             check                                  an exception report for follow-up purposes. For example, invoices are numbered sequentially. The day's invoices
                                                    begin with 12001 and end with t 5045. 1 any invoice larger than t 5045 is encountered during processing, that
                                                    invoice would be rejected as an invalid invoice number.
             Limit check                             ata                                                                                                 1
                                                    D should not exceed a predetermined amount. For example, payroll checks should not exceed US $4,000. 1 a
                                                    check exceeds U $4,000, the data would be rejected for further verificationlauthorization.
             Range check                            Data should be within a predetermined range of values. For example, product type codes range from 100 to 250. Any
                                                    code outside this range should be rejected as an invalid product type.
             Validity check                         Programmed checking of the data validity in accordance with predetermined criteria. For example, a payroll record
                                                    contains a field for marital slatus and the acceptable status 'codes are Mor S.II any other code is entered, the record
                                                    should be rejected.
             Reasonableness check                                                                                                    or
                                                   Input dat? are matched to predetermined reasooable limits or occurrence rates. F eXample, a widget manufacturer
                                                   usually receives orders for no more than 20 Widgets. II an order for more than 20 widgets is receiveG,-Ule computer
                                                   program should be designed to print the record with a warning indicating that the order appears unreasonable.
             Table lookups                         Input data comply with predetermined criteria maintained in a computenzed table 01 possible values. For example, the
                                                   input clerk enters a city code of 1 to 10. This number corresponds with acomputerized table that matches the code to a
                                                   city name. '
             Existence check                       Data are entered correctly and agree with valid predetermined critena. For example, a valid transaction code must
                                                   be entered in the transaction code field.
             Key verification                      The keying process is repeated by aseparate individual using a machine ttlat compares the original keystrokes to the
                                                   repeated keyed input. For example, the.worker number is keyed twice and compared to venly Ihe keying process.
             Check digit                           Anumenc value that has been calculated mathematically is added to data to ensure that the onginal data have not been
                                                   altered or an incorrect, but valid, value substituted. This control is effective in detecting transposition and transcnption
                                                   errors. For example, a check digit is added to an account number so it can be checked for accuracy when it is used.
             Completeness check                    Afield should always contain data rather than zeros or blanks. Acheck of each byte of that field should be pertormed
                                                   to determine that some form of data, not blanks or zeros, is present. For example, a worker number on a new
                                                   employee record is left blank. This is identified as a key field and the record would be rejected, with a request that
                                                   the field be completed before the record is accepted for processing.
             Duplicate check                       New transactions' are matched \0 those previously input to ensure that they have not already been entered. For
                                                   example, a vendor invoice number agrees with previously recorded invoices to ensure that the current not a
                                                   duplicate and, therefore, the vendor Will not be paid twice.
             Logical relationship check            II a particular condition is true, then one or more additional conditions or data input relationships may be required to
                                                   be true and consider the input valid. For example, the hire date of an employee may be required to be more than 16
                                                   years past hislher date of birth.

            • Programmed co ntrols- Software can be lIsed to detect and                         • Exception reports- An exception report is generated by a
               initiate carre.clive a" ti on for errors in data and processing. For
                                      c                                                           program that identifies transact ions or data that appear to be
              example, if the illGorrcct fi le or fi le version is provided for                   incorrcct:Thesc items may be outside a predetermined 'range or
              processing, the application program could display messages                          may not conform 10 specified criteria.
              Instructing that the proper file and version be used. .
            • Rca sol1abl e ilc's~ verification of calculated am.oll nts....:..-               Data File Control Procedures '
              Application programs can verify the reasonableness of                             File contro1~ should cnslll:e.that only authorized processing occurs
              calculated amounts. The reasonableness 'can be tested to ensure                   to stored data. Types of contro ls over data files are shown in
              appropriateness. to predetermined crj ter!a. Any transacti on                     exhibit 3.31.
              that is detCn11incd to be unr~asonablc may be rejected pending
              further review.                                                             .    Contents of data files, or ilideed da~abase tables, genera lly fall
            • LiI11\t checks on amounts-An edit check can provide                              i.nto four categories:
              assurance, through the use of.predetermined lim its, that                        • System control parameter The entries in these files change
              amounts have been keyed or calculated correctly. Any                                 the workings of the system and may alter controls exercised by the
              transaction exceeding the limit may be rejected for further                          system; for example, the tolerance allowed before an exceptional
              investigation ..                                                                     trnnsactiol1 is rcponed or blocked. Any change to these files
            • Il f,! conciliation' of fil e totals....---Reconci li at ion of file totals         'should be.controlled in a similar way to program changes.
              should be perform ed 011 a routine bas is. Reconciliations                       · Standing dara- These "master files" include data, sllch as
              l11.ay be 'pcrfbrmed through the usc of a manually maintained                        supplier/customer names and addresses. that do not frequ ently
              aCCOUI1I, a fi le control 'record or an independent control file .                   change and are referred to during processing. These data should

            CISA Review Manual 2011                                                                                                                                      225
                                                                                                                                             • ,,.?    ..•...

                                                                Chapter 3-lnforma tion Systems Acquisition,
                                                                                                                                          ~C IS~A
 Section Two : Conten t                                                    Development and Implementa tion
                                                                                                                                                           - -- ..
                                                                                                                                                      Certified Auditor'
                                                                                                                                                      Systems Information
                                                                                                                                                      ..        J

                                                               Exhibit 3.31-Data File Controls
         Method                                                                          Description
 Before and after           Computer data in a fite prior to and after a transaction is processed can be recorded and reported. The before and after images
 image reporting            make it possible to trace.the impact transactions have on computer records.
 Maintenance                Control procedures should be in place to ensure that all error reports are property reconciled and corrections are submitted
 error reporting            on a timely basis. To ensure segregation of duties, erro"corrections should be reviewed properly and authorized by personnel
 and handling               who did not initiate the transaction.
 Source                     Source documentation should be retained for an adequate time period to enable retrieval, reconstruction or verification of data.
 documentation              Policies regarding the retention of source documentation should be enforced. Originating departments should maintain copies
 retention                                                                                                     hen
                            of source documentation and ensure that only authorized personnel have access. W appropriate, source documentation
                            should be destroyed in a secure, controlled environment.
 Internal and               Internal and external labeling of removable storage media is imperative to ensure that the proper data are loaded for
 external labeling          processing. External labels provide the basic level of assurance that the correct data rnediurn is loaded for proceSSing. Internal
                            labels, including file header records, provide assurance that the proper data files are used and allow for automated checking.
 VerSion usage              It is critical that the proper version of a file be used as well as the correct file, for processing to be correct. For example,
                            transactions shOuld be applied to the most current database, while restart procedures should use earlier versions.
 Data file security         Data file security controls prevent unaulhorized access by unauthorized user.; that may have access to the application to alter
                            data liles. These controls do not provide assurances relating to the validity of data, but ensure that unauthorized users who
                            may have access to the application cannot alter stored data improperly.
 One-lor-one                Individual documents agree with a detailed listing of documents processed by the computer. It is necessary to ensure that all
 checking                   documents have been received for processing.
 Prerecorded input          Certain information fields are preprinted on blank input forms to reduce initial input errors.
 Transaction logs           All transaction input activity is recorded by the computer. A detailed listing, including date of input, timeof input, user 10 and
                            terminal location, can then be generated to provide an audit trail. It also permits operations personnel to determine which
                            transactions have been posted. This will help to decrease the research time needed to investigate exceptions and decrease
                            recovery time if a system failure occurs.
 F updating and             Proper authorizalion for file updating and maintenance is necessary to ensurethat stored data are safeguarded adequately,
 maintenance                                                                 ay
                            correct and up to date. Application programs m contain access restrictions in addition to the overall system access
 authorization              restrictions. The additional security may provide levels of authorization as well as an audit trail of lile maintenance.
 Parity checking            Data transfers in a computer system are expected to be made in ·a relatively error-free environment. However, when programs
                            or vital data are transmitted, additional controls are needed. Transmission errors are controlled primarily by error-detecling
                            or correcting'codes. rhe former is'used more often because error-correcting codes are costly to' implement and·are unable
                            to correct all errors. Generally, error detection methods such as a check bit and redundant transmission are adequate.
                            Redundancy checking is a common error-detection routine. A transmitted block of data containing one or more records or
                            messages is checked for the number of characters or patterns of bits contained in it. If the numbers or patterns do not
                            conform to predetermined parameters, the receiving device ignores the transmitted data and instructs Ihe user to retransmit.
                            Check bils are often added to Ihe Iransmitted dala by the telecommunications conlrol unil and may be applied eilher
                            horizonlally or vertically. These checks are sim 10 Ihe parity checks normally applied 10 data characlers wilhin on-premises
                            equipment. Aparity check on a single character generally is referred 10 as avertical or column check, and a parity check on all
                            Ihe eQui~alenl bits is known as a horizonlal, longitudinal or row chec.k. Vse of both checks greatly improves the possibilities of
                            detecting a Ira n smi~s i Gn error, which may be missed when eilher of Ihose checks is used alone.

 be authorized before emfy. or manitcnance: Input c'ontrols 1)1tIy                 Output conirol~ includ¢:
  include a report of ehanged data tha~ i ~ checked and approved.                  • Logging and stor:agc of ncgoti able, sensitive and criti ca l
  Audit trails may log all changes_ .                                                fonns in a secure place-Negoti able, senS     itive or criti cal
• Mas t ~ r data/bal ance data- RUlU1ing balances and totals that .                  forms sholild .be.prope,Fly logged and secured to provide
  arc updated by transactions should not be capablc of aC      ljustment             adequa te safegua rds agai nst theft, damage or disclosure. TI.1 c
  except.under stri c;t a p p r~val and review contro ls. Audit trai ls              form log should be routinely'reco nciled to have inve ntory 0 11
  are important here since there may be financial reporting                          hand, and any discrepancies should be properly researched.
  implications for th e change.                                                    • Co mput er generation of negotiable instruments, forlll s
• Transaction fil es- These are contro lled lIsing validation                        and signatu res- The computer gencration of negotiable
  checks, control totals. exception reports. etc.                                    instruments, forms and signatures should be properly controlled.

                                                                                     A detailed.listing of generated forms should be co mpared to the
                                                                                     physical fOl"l11S t:eccived. One should propcdy account for all                        •
Output controls are meant to provide nssurance th at the data
dcliycred to ~I s ers will be prc s c ll te(~ formatted and deli ve red in a.
                                                                                     exceptions, rejections and mut ilati ons.
                                                                                   •.Rcpo·rt distribution- Output reports should be distribLlted                            •
consistcnt and secure manner.                                                        according to authorizdl distribution parameters, which may
226                                                                                                                      CISA Review Manual 2011
                                        Chap ter 3- lnformation System s Acq uisition,
                                        D e v elopment and Implem entation                                                         Secti on Two: Conten t

             be automated or manual. Operations persOIUlcl should veri fy                   • Bcncluriarking with best pmctices
             Ihat output reports are comp lete and deli vered accord ing to                 • Roles and responsibilities
             schedule. All reports should be loggcd prior to distribution. In               • Activities and tasks
             most environments) processing output is spooled to a buffer or                 • Data restrictions
     r       pri nt spool on completion of job processing, where it waits for
             an available pIi ntcr. Controls over access to the print spools arc           3,14 AUDITING APPLICATION CONTROLS
             important to prevent reports from being deleted accidentally from
             print spools or directed to a different printc[ In addition, changes          The IS aud itor's tasks include the following:
             to the output print priority can delay printing of critical jobs.             • IdentiFying the signiFi cam app licat ion components and the
             Access to distributed reports can compromise confidentiality.                   flow of information through the system, and gni ning a detailed
             Thererore, physical distribution or reports should be controlled                understanding of the applicatioll by reviewing the avai lable
             adequately. Reports containing sensitive dala should be pr~tcd                  documentation and interviewing appropriate personncl
             unde r secure, controlled conditions.. Secure output drop·ofl:                • Identifying tbe application control strengths and evalunting the
             points should be established. Output disposal should also be                    impact of the cOIHrol weaknesses to develop a testing strategy by
             secured adequately to ensure that 110 ullnu thorized access cnn                 analyzing th e accumul ated in Formation
             OCCllI". Reports that arc distributed electronically through the
                                                                                           • Revie\ving application system doc.umenta tioll to provide un
             computer system also need to be co nsidered. LOgical ac~ess to                  understanding oF the fu nctionality of the appl icat ioll. Jnlllany
             these reports should also be controlled careFully and subject to                cases- mai nly in large systems or packaged software- it is not
             authorization. When distributed man ually, assurance should be                  feas ible to review the whole application docllmentation. Thus, II
             provided that sensiti ve reports are properly distributed. Such        should be perFormed. ff an application is vendor
             assurance should include the reci piem signing a log as evidence                supplied, tcchn icnl nnd lIser manuals should be reviewed. Any
             or rece ipt of outpul (i.e., manual nonrepudiat ion).
                                                                                             changes to applications should be docume nted propcrly.
           • Ba lancing and reco nciling- Data processing application
             program ou tput should be balanced rou tinely to the cOl1lrol.            .     Thc following doclIlllentation should be reviewed to gain an
             tota ls. Aud it trai ls shou ld be provided to Ibcil itate the tracklllg 01     understandin g of all applicat ion's clevelopment:
             transaction processing and the reconcilintion of data.                         - System development met hodology doc uments- These
           • Outpu t error handling- Procedures for reporting nnd                             d~ul11ents inc lude cost·benefi t analysis and lIser rcquirements.
             controllinu errors contained in the appl ication program output                - f un cti onal design speci licati ons- This docum ent pmvides
 ;':'•.1     should be~stabl ished. The error report should be timely and                     a detailed explallation oF the app lication. Anllnderstandillg of
             del ivered to the origina ti ng department for review and error                  key control poi nts should be noted during revicw of the design
             correctioll.                                                                     specifications.
           • O utput rcport rclcntion- A record retention sc hedule should
                                                                                            - Program' changes- Documentati on of any program change
             be adhered to firm ly. Any governing legal reguintions should be                 should be avn ilable for review. Any change should provide
             included in the retention policy.                                                evidence of authori zation and should be cross-referenced to
           • Verifi cation of rece ip t'of reports- To provide ass ur ance                    source' code.
             th at sensitive reports ar e prope rly distri but ed, the                      - User manuals- A review of the user manuals provides the
             reci pient should sign a log as evidence of rcccipt of output.                   Foundation fo r understanding how the user is utilizing the
                                                                                              applicntion. Often control weaknesses can be noted from the
           The IS auditor should be aware of existing concerns rega rding                     review of this documen t.
           rccord·retenti"on policies for the organ izntion and add~es s legal              - Technical reference doc um entation- This documentation
           rcquirements. Output can be restri cted to particular IT resQurces                 includes any vendor·supplied technical manuals for purchased
           or a~v i c~s'(e.g·., a particular printer)                                         appl"ications 'in addi tion to any in-house documentation. Access '
                                                                                              rules and logic usua lly are included in these documents.
           1n an irllebTf"(lt~d nppUcation cnv'iropment, controls arc embedded             3.14.1 FLOW OF TRANSACnONSTHROUGH THE .
           and de:signe:d .illto the application that SlIPPO the processes.          .     SYSTEM
           Business process control nSSUl11l1Ce involves evaluating controls at .
           the process nnd activjty level.1l1esc cOl~irols l11~y be a cQI)lb.ina~on        A'tra;lsaction flowc har.t provides informat ion rega rdin g key
           of management, progmn"imed and mrumal controls. 1 addition to.11
                                                                                           JrOccssillg controls.' Points where transactions arc en tered,
                                                                                           p~ocessed m!d posted should be reviewed for control weaknesses.
           evaluating genemL       conlrOls that affect the processes., business process
o·         owner·specific controls-sllch as establishing proper security and
           S\..""'greg~tion of duties, peliodic review and approvrtl of access, and         3.14.2 RISK ASSESSMENT MODEL TO ANALYZE
           application controls within the business process-are evaluated.                 .APPLICATION CONTROLS
                                                                                           Risk assessment, as discllssed in chapter I>provides informati on
           Specific matters to consider in the business process control                    relating to the inherent ri sk oFun application.
           assmance are:
j          • Procc~s mnps                                                                  A risk assessment model can be based on many Factors, whi ch
           • Process controls                                                              may include a combinat ion of the Following:
           • Assess ing business risks within the prottss

                                                                                           • The qua lity of internal contro ls

            CISA Review Manual 2 011                                                                                                                      22 7
                                                                                                 ...• : ....   -;;~:.   .........;. .." '.

 Section Two: Content
                                                          Chapter 3-lnform,ation Systems Acquisition,
                                                                     Development and Implementation                                          ~C~IS~A'   Certirled Infarmation
                                                                                                                                                        Systems. AUditor'
                                                                                                                                                        ,. ..."tttw<no

  • Economic conditions                                                       Testing can be performed through the review of access ni les to
  • Recent accoullt ing system changes                                        ensure that access has been granted as management intended.
  • Time elapsed si nce last audi t
  • Complexity of operations                                                Activity reports provide details, by user, of act ivity vo lume and
  • Changes in operations/environment                                       hours. Activity reports should be reviewed to ensure that act ivity
  • Recent changes in key positions                                         occurs only during authori zed hours of operation.
  • Ti me in existence
  • Competitive environment                                                 Violation reports indicate any unsuccessful and unauthorized
  • Assets at risk                                                          access attempts. Violation reports should indicate the term inal
  • Prior audit results                                                     location, date and time of attempted access. These reports should
  • Sta ff turnover                                                         evidence managerial review. Repeated unaut hori zed access
  • Transaction volume                                                      violations may indicate attempts to ci rcum vent access controls.
. • Regulatory agency impact                                                Testing may include review of follow-up activities .
  • Monetary vo lume
  • Sensi tivity of transacti ons                                           3.14.4 DATA INTEGRITY TESTING
  • Impact of applicat ion fai lure                                         Data integrity testing is a set of substantive (csts that examines
                                                                            accuracy, completeness, consistency and authorization of data
 3.14.3 OBSERVING AND TESTING USER PERFORMING                               presently held in a system. It employs testi ng similar to that used
 PROCEDURES                                                                 fo r input control. Data integrity tests wi ll indicate fa ilu res iil
 Some of the user procedures that should be observed and tested             input or processing controls. Controls for ensuri ng the integrity of
 include:                                                                   accumu lated da ta in a fi le ca n be exercised by regularly checki ng
 • Separat ion of duties- Ensures thaI no individual has the                data in the fi le. When this .checking is do ne aga inst au thorized
   capab ilit y of performing marc than one of tile fo llowi ng             source documen tati on, it is COITUllOIl to check ouly a portion of th e
   processes: originatio n, authoriza tion, verification or distribution.   file·at a time. Since the·whole file is regularly checked in cycles,
   Observati on and review of job descriptions and review of                the COJltrol technique is often referred to as cyclical checking.
   authorization levels and procedurcs may provide informat ion
   rega rding the exist.ence and enforcement of se parati on of duties.     Two common types of dat(). integrity tests are relat ional and
                                                                            refcrent ial integrity tests. Relational integ ri ty tests are performcd
 • Authorization of input- Evidence of input aut horizati on can
                                                                            at the dala elcment and record-based.levels. Relational integrity
   be achieved via wri tten authorization on input documen ts or
                                                                            is enforced through data validation routi nes built into the
   wi th t.he use of unique passwords. One may test this by looking
                                                                            application or by defini ng the input condition constraints and
   tlnough a sampling of input documen ts for proper authorization
                                                                            data characteristics at the table definition in the database stage.
   or reviewing computer-access rul es. Supervisor overrides of
                                                                            Sometimes it is a combination of bot h.
   data validation and editi ng should be reviewed to cnsu re that
   a\ltomatic logging occurs. This override activity report should
                                                                            Referent ial integ ri ty tests define existence relationships
   be tested for evidence of managerial rev iew. Excessive ovcrrides
                                                                            between entities in different tables of a database thai needs to
   may indicate the need for modification of val idation and editing
                                                                            be maintained by the DBMS. It is required for mainta ining
   routines to improve efficiency.
                                                                            interrelation integri ty in the relat ional data model. Whe never
 • Balancing- Perfor med to ve rify that run-to-run control totals
                                                                            two or more relations are related through referential constra ints
   and other app licati on lotals are reconciled on a time ly basis.
                                                                            (prinlary and fore ign key), it is necessary that references be kept
   Th is may be tested by indepe ndent balancing or review ing past
                                                                            consistent ill the event of inserti O del etions and updates to·th ese
   reconciliations.                              . .          .
                                                                          . relations. Database software gC~lera lly provides va rious bllilt~
 • Error control and cor rection- In the fonn of reports that               in automated procedures for chec king and ensuring referential
   provide evidence. of approp\iale review, research, timely                in tcgrity. Refercntial in tegrity checks 'i nvolve e.nsuring that a" ·
   c~rrection and ·resubrnission. Input errors and rej c~tip n s should·
                                                                            refc'rences to a primary key from another table (i.e., a foreign key)                               I
  .be reviewed prior 10 reslibmission. 'Managerial review and·             -actually cxist in their original table: In nonpointer databases
   authorization of corrections should be evidenced. l esti ng oftlu s .(e.g., relational), referential integrity checks involve making
   effort can be achieved by ret(lbulat ing or reviewi ng past crror        that all f,? ~e.i gn keys .exisl in their original ~able.                                           !.
 • Distribution of reports~Crit i cal output. reports shou ld be
   produced and main tained in a secure area and distributed in an
                                                                            3.14.5 DATA INTEGRITY IN ONLINE TRANSACTION
   authorized manner. The distribution process can be tested by             PROCESSING SYSTEMS
   observation anCi review of distribution output logs. Access to           III muiliuser transaction systems, it is necessary 10 manage parallel
   online O  utput reports sllould be restricted. Online access may be user access to stored data typically controlled by a DBMS. and

   tested through a review of the access rules or by monitoring user deliver fault tolerance. Of particular importance are fOllr online dara

   output.                                                                  illt~grity requirements known collectively as the AC ID principle:
 • Review and testing of access authorizations .and                         • Ato micit y- From a user perspect i\·c . .. transacti on is either
   ca pabilities- Access contro l tables provide infol111a tion               completed in its enti rety (i.c., all relevan t database tables arc
                                                                               updated) or not at all. If an errar or' i;itcn~lIption OCC~lrs, all

   regarding access levels by individpals. Access should be hased
   on job dcscriptions and should provic!c.for a separnl ioll of duties.       c h a n g~s made up to that point arc backed out.

 228                                                                                                                       elSA Review Manual 2011
           C SA
          ~~I~       Cellilie<llnformation
                     Systems AOOifoc'
                                             Chapter 3 - ln fo rm ation Sys tem s Acquisition,
                                             Developme nt and Implem en tation                                                                 S ection Tw o : Content

          • Consistency- All integrity cond itions in the database arc                            To facilitate the evaluation of application system tests, an IS
            mainta ined with each transaction , taki ng the database from one                     aud itor may also want to use gencmlized audit software (GAS)
            consistent state into another consistent stale.                                       also known as computer-assisted audit tools (CAATs). This is
          • Isolat ion- Each transaction is isolated from other transactions                      particularly useful when specific app lication cont rol weaknesses
            and hence each transaction on ly accesses data that are part ofa                      are discovered that affect, fo r example, updates to master file
            co nsistent database state.                                                           records.and certain error conditions on specific transac tion
          • Durabili ty- Ifa transaction has been reported back to a user                         records. Additionally, GAS can be used to perform certain
            as com plete, the resulting changes to the da tabase surv ive                         appncatioll control tests, such as parallel simulati on, in compari ng
            subsequen t hardwa re or soft ware fai lures.                                         ex pected outcomes to live data.

          This type aftesting is vital in loday's vast ar ray of onli ne Internet-                3.14.7 CONTINUOUS ONLINE AUDITING
          accessible. mu ltiuser DBMSs.                                                           Continuous online aud iting is becomi ng increasingly important
                                                                                                  in today's e-busi ness world sjnce it provides a method fo r the IS
          3.14.6 TEST APPLICATION SYSTEMS                                                         aud itor to collect evidence on system reliabili ty while normal
          Testing the effect iveness of appl icat ion controls involves                           processing takes place. The approach allows IS auditors to
          analyzi ng compu ter application programs. testing computer                             monitor the operation of such a system on a contin uolls basis
          app lication program controls, or selecting and monitori ng data                        and gather selective audit evidence through the computer. If the
          proccss transactions. Testing contro ls by app lying appropriate                        selective infonnation collected by the computer technique is llot
          audit procedures is import..1l1 t to ensure their functionality and                     decmed serious or"material enough to warra nt immediate action.
          effccti vencss. Methods and techniques for each category are                            the information is stored in separate audi t fil es for verification
          descri bed in cxhibil3.32.                                                                                           til11e. The continuous nudi t appronc h
                                                                                                  by thc IS audi tor at a Inter"

                                                                    Exhibit 3.32-Testing Application Systems
                                                                        Analyzing Computer Application Programs
                 Technique                                 Description                                 Advantages                                   Disadvantages
.'        Snapshol                           • Records flow of designated                • Verifies programlogic                    ~   Requires extensive knowledge of the
                                               transactions through logic paths                                                         ISenvironment
                                               within programs
          Mapping                            • Identifies specific program logic Ihat    • Increases efficiency by identifying     • Cost of software
                                               has not been tested. and analyzes           unused code
                                               programs during execuiion to indicate     • Identifies potentiaf exposures
                                               whether program statemenls have

          Tracing and tagging                • Tracing shows the Irail of instructions
                                               executed during an application.
                                               Tagging involves placing an indicalor
                                               on selected transactions at input and
                                               using traCing to track them.
                                                                                         • Provides an exacl piclure of sequence • R
                                                                                           of events, and is effective with live
                                                                                           and simulated transactions
                                                                                                                                     equires extensive amounts of
                                                                                                                                   compuler time. an intimate knowledge
                                                                                                                                   of the application program and
                                                                                                                                   additional programming to execute
                                                                                                                                   trace routines
          Test data/deck                     •S  imulates transactions through real      • May use actual master files or          •D ifficult to ensure that the proper
                                              . prog_
                                                    rams                                                           program is checked
                                                                                         •S  ource code review is unnecessary.     • Risk of nol including all transaction
                                                                                         • Can be used on a surpose basiS               ~cenarios
                                                                                         • Provides objectivereview and'           • Requires good knowledge of
'--                                                                                      · verification of program controls and      application systems
                                                                                           edits                                        oes
                                                                                                                                   • D not test master.file and master
                                                                                         • Initial use can belimned 1 specific       file records
                                                                                           program functioos m   inimizing scope
                                                                         .   -             and complexity. . - . .
                                                                                         • Requires minimal knowledge of the IS
          Base-case system                   • U test data sets developed as
                                                 ses                                  • Comprehensive testing verification          • Extensive effort to maintain data sets
          evaluation                           part of a comprehensive testing of       and compliance testing                          lose
                                                                                                                                   .• C cooperation is required among
                                               programs                                                                               all parties.
                                             • Verifies correct system operations
                                               before acceptahce, as well as periodic

          CISA Revie w Manual 201 1                                                                                                                                    229
  Section Two: Content
                                                                Chapter 3-lnformation Systems Acquisition,
                                                                           Development and Implementation                                  ~C~IS~A   Certilie<l Auditor'
                                                                                                                                                                ,           I


                                                      Exhibit 3.32-Testing Application Systems (cont.)
                                                           Analyzing Computer Application Programs
         Technique                           Description                                  Advantages                                 Disadvantages
  Parallel operation            • Processes actual production data         • Verifies new system before                   • Added processing costs
                                  through existing and newly developed       discontinuing the old one
                                  programs at the same time and                                                                                 -
                                  compares results. and is used to
                                  verify changed production prior to
                                  replacing existing procedures
  Integrated testing facility   • C ates a fictilious file in the database • Periodic testing does not require
                                    re                                                                                   • Need for careful planning
                                  with fest transactions processed           separate test process.                      • Need to isolate test data from
                                  simultaneously with live data                                                            production data
  Parallel simulation           • Processes production dala using          • Eliminates need to prepare test data        • Programs must be developed.
                                  compuler programs that simulate
                                  application program logic
  Transaction selection         • Use audit software to screen and         • Independent of production system            • Cost of development and
  programs                        select transactions input to the         • Controlled by Ihe auditor                     maintenance
                                  regular production cycle                 • Requires no modilication 10
                                                                             production systems
  Embedded audit data           • Software embedded in host computer       • Provides sampling and productions           • High cost of development and
  collection                      applications screens. It selects           stalistics                                    maintenance
                                  input Iransactions and generated                                                       • Auditor independence issues
                                  transactions during production.
                                  Usually, IT is developed as part of
                                  syslem development. Types include:
                                  - Systems control·audit review file
                                     (SCARF)-Auditor determines                                                                        -
                                     reasonableness of tests
                                     into normal processing. It provides
                                     information for furtller review.
                                  - Sample audit review file (SARF)-
                                    Randomly selects transaclions
                                    to provide representative file for
  Extended records              • Gathers all data that have been          • Records are put into one convenient         • Adds to dala slorage costs and
                                  affected by a particular program           file.                                         overhead, and to system development

  cuts down on needless paperwork and leads to the conduct of an                   COl1t i J1UO~I S   audit very often re lies on calls to GAS/CAAT services. ·
  essentia lly paperless audit. In such a settin g, ·an IS audi tor
. repo'rt directly through the microcomput er 011 significant errors·              3.14.8 ONLINE AUDITING TECHNIQUES
 -or other irregularities Ihat may require im mediate managcl1)cm
                                                                                  Thero arc. fi~/e types of autom~ted eval uation te·chniqu·es
  action:-This appr?ach . ~cduces audit cosl .and time.                           <I])p lieable to continuous onl ine aUditing:
                                                                                  I. Systems Co nt rofA lIdit Revie w File and Emb'cd ded
  Continuous audit techniques arc il~lportant IS audit lools,
  particularly W11el) they' are l!scd in time-sharing environments that              Audit Mo dules (SCA RF/ EAM)- The use or thi s technique
. P.fOCeSS a large numbcroftnlllSt1ctions blllicave a scarce paper                   invol ves embedding specially wrift en audit so n\va re in the· '
  trai l. By pennitting IS auditors to evaluate operati ng cont rols·on              organization'Shost application system so the application
  a continuous basis without dismpling the organiZ:1tioll's uS    lial               systems are monitored on a selective basis.
  openllions, continuous audit techniqucs improve the security of a               2. Snaps hots- This technique involves laking what might
  system. When a systcm is misused by someone withdl1lwing money                     be termed pictures of the processing path that a lransaction
  fron~ an inoperative account, a continuous audit technique wi ll                   follows, from the input to thc output stage. With the lise of

  report this wit hdra.wa·1in a timely fashion to the IS audi tor. Thus              th is tec hniq ue, transactions are ta gged by app lying idenrifiers
  the time lag between the misuse ofthc system and the detection                     to input data and recording selected infonnation abollt what
                                                                                     occurs for the <lllditor's subsequellt review.

  of that misuse is reduced. The rea lization tl1al 1~lilurcs, improper
  manipUlation and lack of controls will be detected on a timely basis            J. Audit hooks- This tec hn ique invo lves embedding hooks in
                                                                                     app lication systems to function as red flilgs anJi to· i;ld uc- IS

  by thc usc of continllous audi t procedures gives IS auditors aJld
  management greater confidence in a system·s reliability:                           auditors to ae l before.<ln error or irregularity gets out of hand.

 230                                                                                                                     CISA Review Manual 2011
         ~C~IS'A   Sysl!ms,tllormation
                   Certified Auditor'
                                         Chapter 3-lnformation Systems Acquisition,
                                         Development and Implementation                                                               Section Two: Content
                   "$'U~_         '

         4. Integ ral ed lesl racility (ITF)- In this technique, dummy                        development and lIser project team members to identify controls
            cllIitics arc sel up and included in an auditee's production                      to mi tiga te the risks to and exposures of the sys tem.
            fil es. The IS auditor can make the system ei ther process live                 • Evaluate ava ilable controls and participate in discussions
            transactions or lest transacti ons duri ng regular processing rUIlS,              with systcms development and user project team members to
            and have these transactions update the records of the dummy                       advise the project team regarding the design of the system and
            entity. TIle operator enters the test transactions simultaneollsly                implement ati on of controls.
            with the live transactions that arc entered for processing.                     • Periodically meet with systems development and user project
            The auditor then compares the output with the data that have                      team members, and review the documentati on and deli verables

    .~                                                                                        to monitor the systems development process to enSlire that
            been independent ly calculated to veri fy the correctness of the
 '~'.~      COlll pll tcr-processed data.                                                     controls arc implemented, user and business requirements are
         5. Co ntinuous and intermitt ent simulation (CIS)- During a                          met, and the systems development/acquisition methodology is
            process run of a transaction, the compu ter system simulates the                  being followed. Also rev iew and evaluate the application system
            instruction execution of th e app lication. As each transaction is                audit trails to ensure that documented controls are in place to
            ent ered, the simul ator decides whether the .transaction meets                   address all security, edit and processing controls. Audit trails are
            ce rtain predetermined criteria and, if so, audits the transacti on.              tracking mechanisms that can help IS audit ors cnsure program
            If not. the simulator \Vaits lIllt il it cncounters the ncx t                     change accountability. Tracking information in a change
            Inlllsaction that meets the criteria.                                             management system ind udcs:
                                                                                              - History of all work order activity (date of work order)
         In ex hibit 3.33.lhe relative advantages and disadva ntages of the                     prog rammer assigned, changes made and date closed)
         vl.l ri olts concurrent audit too ls are presented.                                  - History of 1   0gol1s and 10goO's by programmers
                                                                                              - l-Iis.lory of program deleti ons
         The usc or each of the continuous aud it techn iqucs                               • Participate in post implemcntation reviews.
         has ad\'alltages and disadva ntnges. Thcir select ion and                          • Review appropriate docultlentat ion, discliss with key personnel,
         implemclllatioll depends: 10 a large extent, on the complexi ty                      and lise observa ti on to evaluate system ma" intcnance standards
         of <I n organization's computer systems a1!d applications_ and -the                  and procedures to ensure their adequacy.
         IS auditor's ab ility to un derstand and eval uate the systcm wit h .              • Discuss and exa mine supporting records to test system
         and without the lise of co nt inuous au dit techniques. In addition,                 maintenance procedures to cnsure that th ey are being applied as
         IS aud itors must recognize tl1a t cont inuous audi t tech niqu es                   described in .the standar4s,
         arc'not'a C re for all cont ro l problems and that the use of these
                     lI                                                                     • Analyze test rcsulls and other audit evidence to evaluate the
         techn iqucs provides only limi ted assurance that the informati on                   system maintenance process to determ ine w hether control
         processing systems examined are operating as they were intended                      objectives were achieved.
         to fu nction.                                                                      • Identify and test ex isting controls .to determine the adequacy
                                                                                              of producti on library security to ensure the integ rity of the
                                                                                              producti on resources.
         ACQUISITION AND MAINTENANCE                                                        3.15.1 PROJECT MANAGEMENT
                                                                                       Throughou t the project managemen t process the IS auditor should
         The IS auditor's tasks in system development, acqu isition                    analyze the assoc iated risks and exposures inherent in each phase
         and mai ntenance may take place once the project is fin ished                 of the SOLe and ensure that the appropria te co nlrol mechanisms
         or- infrequenlly-during the project itself. Most tasks in the                 are in place to min imize these risks in a cost-effecti ve manner.
         following list cover both scellarios and the reader is ex pected ·            Cauti on should be exerciscd to avoid recommending cont rols
         to determine which task applies. Tasks generally include the                  that cost more to adm inister than the ~'ssoc ia ted risks they are
         rollowing:                                                                    designed to minimi ze.
         • Meet with key systems developmen t and user project team .
            mC  Il1bers to detc:rm inc th e mai'n components, objecti ves and          W~en reviewing the SOLe Proi;css, th~ IS !ludi IO'r should obtail;
          . user requi rements of th.c syslein to identi fy .the" area'S tbat requ ire documeiltalioll lI'om the various phases and attend project team
            conlrols.                                                                  meetings, offeri ng advice to the project team throughout th~
         • Discuss the selection of appropriate controls with SYStC S'     ll1         system developl11cnt process. The IS auditor should also assess
  ...       development and user projec t team members to detenn ine and . .lhe project team:s ab ility 1 produce·key delive rables by Ihe
            rall"k the major risks 10 and exposures of the system.                     promised dates.
         • Discuss references to auth oritative sources with systems
                                                     Exhibit 3.33-Concurrenl Audit Tools-Advantages and Disadvantages
                                              SCARF/EAM                      ITF                  Snapshots                   CIS                    Audit Hooks
          Complexity                     V high
                                          ery                     High                      Medium                 Medium                      Low
          Useful when:                       lar
                                         Regu processing          II is not beneficial to   An audit trail is       Transactions·meeting         nly
                                                                                                                                               O select
                                         cannot be interrupted.   use test data.            required.              ·certain criteria need 10   transactions or
                                                                                                                    be exam in~d .             processes need to be

          elSA Review Manual 2011                                                                                                                              231
 Section Two : Conten t
                                                          Chap ter 3-ln f or m ation S ystem s A c quisition,
                                                                     Devel op ment and Implem entati on                   ~C:IS~A'     Certified Auditor'
                                                                                                                                     . Syslems,Information
                                                                                                                                     .. -...t"!"'-

 Typically, the IS auditor should review the adequacy of the              3.15.4 SOFTWAR E ACQUISITION PROCESS
 fol lowing project management activities:                                The IS auditor should perform the following functions:
 • Levels of oversight by project committee/board                         • Analyze the documentation from the feasibility snldy to
 • Risk management methods within the project                               determine whether the decision to acqui re a solution was
 • Issue management                                                         appropriate.
 • Cost management                                                        • Review the RFP to ellsure that it covers the items listed in this
 • Processes for plmming and dependency management                          section.
 • Reponing processes to senior management                                • Determine whether the selected vendor is supported by RFP
 • Change control processes                                                 documentation.
 • Stakeholder management involvement                                     • Att end age nda-based prese ntat ions and conference room pilots
 • Sign-ofT process- At a minimum, signed approvals from                    to ensure that the system matches the vendor's response to th e
   systems deveioljment a~d user manageme nt responsible for the
   cost of tile projecL ancVor lise of the system
                                                                          .. Review the vendor contrac t prior to its signing to ensure that it
                                                                             includes the items listed.
Additionally. adeq uate and complete documelllation of all                • Ensure the cont ract is reviewed by legal counsel before it is
phases of the SDLe process should be evide nt. Typical types of              signed.
doculllcntatibn may include, but sJl0uld not be limited to. the
o Objectives defining whnt is to I:>e accompliShed during that phase      3.15.5 DETAILED DESIGN AN D DEVELOPMENT
o Key deliverables by phases with project persOIlJlel assigned
                                                                         The IS auditor should perform the following functions:
  direct responsibilities for these deliverables                         • Review the system flowcharts fo r adherence to the genera l
o A project sc hedule with highli gl)ted dates for the completi on of
                                                                           design. Verify that appropriate approvals were obtained for
  key deliverables                                                         any changes and all changes were discllssed and approved by
• An economic forecast for that phase, defi ning resources and Ihe         approprintc user management.
  cost of tile resources re. uired to complete the phase                 o Review the input, processiilg and output controls designed into

                                                                           the system for appropriateness,
3.15.2 FEASIBILITY STUDY                                                 • Interview the key users of the system to determine their
The IS auditor should perfo rm the following func ti ons:                  understanding of how the system wi ll operate, and assess their
• Review the doclIlllemation produced in this phase for                    level of input into the design of screen fonnats and output reports.
  reasonableness.                                                        o Assess the adequacy of aud it trails to provide traceability and

o Detemiine whe ther .all cost justifications/benefits are verifiable      accoulllability of system transactions.
  and, showing the antic ipated costs and penefils to be realized.       • Verify the integrity of key calculat ions and processes,
• Identify and determine the criticality of the need.                    • Verify that the system can identify and process erroneous data
• Determine if a solution can be achieved with systems already in          correctly.
  place. If not, review the evaluation of alternative ·sol utions for    • Review the quality assurance results of the programs developed
  reasonableness.                                                          during this phase.
• Determine the reasonableness of the chosen solution.                   • Verify that all recommended corrections to programming errors
                                                                           were made and the recommended audit trai ls or EAMs were
3.15.3 REQUIREMENTS DEFINITION                                             coded into the appropriate prog ram s.
 The IS auditor should perform the following functions:
 • Obtain the detailed· requirements definition document and             3.15.6 TESTING
    verify· its accu,:acy thi·ough interv iews with the relevant user     Testing is crucial· in determining thc·l!Ser req uirements have been
   -departments.               .                                          va lidated the sys~em · is performing as antic ipated and internal
.·ldentify·the key t~am me'mbers.on ·the project team al)d                cOll trols work as. intended. Therefore it1Sessen tial I1mt the IS
    verify that all nlTected user groups have/had appropriate           , au·ditor be involved in rc.viewing·this phase and perfOJm the
    represelltation.                                                       following:                         .
 • Verify that project initiation and cost have received proper:          • Review the test plan for completenesS"; indicate evidence of use r
   management approval . .                                                   participation sllch as user developmt:nt of test scenarios and/or
 • Review the conceptual design specifications (e.g" transforms,             lIser sign·off of results; and consider rerunning cri"tic'ilI4cstS.
   data descriptions) to ensure that they address th e needs of           • ·Reconcile control totals and converted data,
   the user,                                                              • Review error reports for th eir precision in recognizing ·erron eous
 • Review the c'onceptual design to ensure that cont rol                     data and resolution of errors.
   specifications have been de fined.                                     • Verify cyclical processing for correctness (month·end, yea r-end
 • Determine whether a reasonable numper of vendors received a               processing, etc.).
   proposal covering the project scope and use r requirements.            o Interview .end users of the system for their understanding of new

• Review the UAT speci fication .
o Determine whether the application is a candidate for the use
                                                                             methods. procedures and operating instruct.ions.
                                                                          • Review system and end· user documcntation to determin e its                       •
   of an embedded audit rou tine. Ifso, request that the routine be
   i~lcorporated in the conceptual design of the system.
                                                                             completeness and verify its accuracy during the test phase.
                                                                          o Review parallel testing results for accuracy.
2 32                                                                                                      CISA Review Manua l 2 011
               Ceflirle(l At.dtor'
                                     Chapter 3-lnformation Systems Acquisition,
                                     Deveiopment and Implem entation                                                 Section Two: Content

     • Ve ri fy that system security is function ing as designed by           3.15.9 SYSTEM CHANGE PROCEDURES AND THE
       deve loping and execLiting access tests.                               PROGRAM MIGRATION PROCESS
     • Review unit and .system test plans to determine whether tests for
       internal controls arc planned and perfonned.                           Following implementation and stabilization, a system enters
     • Review the user acceptance testing and ensure that the accepted        in to the ongoing development or maintenance stage. This
       software has been delivered to the implementation team. The            phase continues until the system is retired. The phase involves
       ve ndor should not be able to rcp lace this version.                   those activities required to either correct errors in the system
                                                                              or enhance the capabi lities of the system. In t hi s "Tegal'(~ the IS
     • Review procedurCs used for recording and fo llowing through on
                                                                              auditor should consider the foll owing:
       erro r rcports.
                                                                              • The existence and use of a methodology fo r authori zi ng,
                                                                                prioritizing and tracking system change requests fro m the user
     3.15.7 IMPLEMENTATION PHASE                                              • Whether emergency change procedures are addressed in the
     This phase is initiated only anc r a sliccessful testing phase. The        operations manuals
     system should be installed accordi ng to the orga ni zati on's change    • Whether change control is a formal procedure for fhe user and
     control procedures. The IS aud itor shou ld verify that approp riate       the development groups
     sign-offs have been-obtained prior to implcmentation and perform         • Whether the change conlrollog ensures all changes shown were
     the following:                                                             resolved
     • Review the programmed procedures used for schedulin g and              • The tlser's satisfaction with the turnaround- time liness and
       run ning the system along with system parameters llsed in                cost-of change req uests
       executing the production schedule.                                     ~ The adequncy of the security access restrictions over production
     • Review all system documentation to ensure ils completeness               source and executab le modules
       nnd thm all recent updntes f)·om the testing phase hnve been           • The adequacy of tile organi zation's procedures for dealing with
       incorpomtcd.                                                             emergency program changes
     • Vei·ify all data conversion to ensure that they are co n·ecl and       • The adequacy of the security access rest rictions over the lise of
     complete before implementing the system in producti on.                    the emergency logon IDs

     3.15.8 POSTIMPLEMENTATION REVIEW                                         For a selection of changes on the change control log:
     Afte r the new system has stabilized jnlhe producti on                   • Determine whether changes to rcquirements resulted in
     environment, a postimplcmcntatioil review shou ld be performed.            appropri ate change-development docum·ents sllch as progmm
     Prior to this review, it is important th~t sufficient time be              and operati.ons docllments.
     allowed for the system to stabili ze in producti on. In this way, any    • Determine whether changes were made as documented.
     significant problems will have had a chance to surface.                  • Determine whether current documentation reflects the changed
      The IS auditor shou ld perform the following func tions:                • Evaluate the adequacy of the procedures in place for testing
      • Determine if the system's objectives and requirements were              system changes.
        achieved. During the· postimplementation review, careful              • Review evidence (test plans and test results) to ensure that
        attention should be paid to the end users ' utilization and overall     procedures are carried out as prescribed by organ izationa l
        sat isfact ion with the system. This will indicate whether the          standards.
        system's objecti ves and req uirements were achieved.                 • Review the procedures established for ensuring executable and
      • Determ ine if the cost benefit s identified in the feasi bility         source code integrity.
        shldy are being measure(~ analyzed and accurately reported 10         • Review productioil executable modules and veri fy there is one
         management.                                                            and only one corres ponding version of the program
      • Rev iew program change requests performed to assess the                 source code.
        tYpe of changes required of the systclll. The type of changes
    .. requested may indicate problems in the design, progranll~ling or       Additionally, the [S auditor si10llid review theoveraU
         interpretation of llSer requiremen ts. . .                           ch.ange management process for possible improvements in·
      • Revie\v conlrots bullt.into the system io ensure ~hat they are        acknowledgement, response tim·e, response effecliYeness and lIser
        operating according to design. If an EAM was included in the          satisfacti on with the process.
J     · system, lise this modlll~ to test key ope rations.
      • Rev iew operators' error logs to dete rmine if there are any
       . resource or operating problems inherent within the system.
        The logs l11ay indicate inappropriate planning or testing of the
        system prior to implementation.
      • Rev iew input and output control balances and reports to verify
         that the system is processi ng datJ accura tely.

     CISA Review Manual 2011                                                                                                                  233
~,. ~   ::--'      ~   "   .   ',."     ~".                     '." "       ".   ;.~,:",   .~   ..

                Section Two : Content
                                                                                       Chapter 3-lnformation Systems Acquisition,
                                                                                                  Developm ent and Implementation                               EC~'S~A
                                                                                                                                                                  .          Syslems Auditor'

                3.16 CASE STUDIES                                                                       3.16.2 CASE STUDY B
                                                                                                        Case Study B Scenario
                The following case stud ies nre included as a learning tool to                          A large industrial concern has begun a complex IT project, with
                reinforce the concepts introduced in this chapter. Exam candidates                       ERP, to rep lace the main component systems of its accounti ng
                should note that the e lSA exam does not currently usc this format .                    and project control departments. Sizeable customizatiolls were                              ,
                fo r testing.                                                                           anticipated and arc being carried out with a phased approach of                             I
                                                                                                        partial dcliverablcs. nlCse deliverables are released to users for
                3.16.1 CASE STUDY A                                                                     pilot usage on real data ancl actual projects. In the meanwhile,
                                                                                                        detailed des ign and programmi ng of the next phase begins. After
            Case Study A Scenario
                                                                                                        a period of in itial adjustment, the pilot users start ex periencing
            A major retailer asked the IS auditor to review their readiness for
                                                                                                        serious difficulties. In spite of positive test results, already
            complyi ng with credit cmd company requirements for protecti ng
                                                                                                        stab ilized functionali ties began 10 have intermittent problems;
            cardholder information. The JS auditor subsequently l earn~d
                                                                                                        transactioilS hang during cxeclllioll and, more and more
            the fo llowing information. The retai ler uses wireless pojnt-of-
                                                                                                        frequent ly, project data get corrupted in the database. Additional
            sale (POS) registers that COlUlect to application servers located
                                                                                                        problems sho\v·up- errors ~l ready corrected started occurring
            <II each slore. These registers use wired equivalent protection
                                                                                                        agai n and funct ional modifications already tested tend to
            (WEP) encryption. The apjillCation server, usua lly located in ·the
                                                                                                        present other errors. The project, already late, is now in a criti cal
            middle of the stores customer service area, forwards all sales
                                                                                                        situation. The IS auditOJ~ after collecting the evidence, requests
            data over a frame relay net work to database serve rs locatcd at
                                                                                                        an immed iate mccting with the head of the project steering
            thc rctni ler's corporate he<ldquarters, and using strong encryption
                                                                                                        committee to cOllllllunicate findings and suggest actions capablc
            over an Internet virtual private network (VPN) to the credit
                                                                                                        to improve the situatioll .
          . ca rd processor for approva l oflhe sale, Corporate databases are::
            located 0 11 11 protected screened subset of the corporate local area
            network. Additionally. weekly aggregate sales data by prodLtct                                                        CASE STUDY B QUESTIONS
            line is copied frolllthe corporale databases to magnetic media                                B1 .      The IS auditor should indicate to the head of the project
            anclmailed to a th ird party for analysis of buyi ng palterns. It was·                                  steering committee that:
            noted that the retailer's d atab~se software has 110t been patch.ed in
            over two yea rs. This is because vendor SUppO for the database                                          A. the observed project problems are a classic example of loss
            package was dropped due to management's plans to eventually                                                of control of project activities and loose
            upgrade to a new ERP system.                                                                               discipline in following procedures and methodologies. A
                                                                                                                       new project leader should be appOinted.
                                                                                                                    B. relays due to an underestimation of project eHorts have
                                                 CASE STUDY A QUESTIONS                                                led to failures in the versioning and modilication control
                 A1.                  Which of the following would present the MOST significant                        procedures. New programming and system reso"urces
                                      risk to the retailer?                                                            must be added to solve the root problem.
                                                                                                                    C. the problems are due to excessive system modifications
                                      A. Wireless point-of-sale registers use WEP encryption.                          after each delivery phase. Tile procedure for control of
                                       . atabases patches are severely out-of-date.                                    modifications must be tightened and made more selective.
                                      C. C cardholder information is sent over the Internet.
                                          redit                                                                     D. the nature of initial problems is such as to lead to doubts
                                      D Aggregate sales data are mailed to a third party.
                                       .                                                                               regarding the adequacy and reliability of the plattorm. An
                                                                                                                       immediate technical review of the hardware and software
                 A2.                  Based on the case study, which of the following controls                         platterm (parameters, configuration) is necessary.
                                      would be the MOST important to implement?
                                                                                                          B2.       In order to contribute more directly to solve the situ!ltion, the
                                      A. Store application servers should be located in a                           IS auditor should:
                                         &ecur. .area.
                                      B Point-of-sale registers should use two-factor authentication.
                                       .                                                                            A. research theproblems furthe(to identify root causes and
                                      C. Wireless access points should use MAC addressliltering.                       deline appropriate couritermeasures.·                      .
                                      D. A!]gregate sales data se·nt oHsiteshould ~e encrypted.                     8. review the validity of the·functional project specifications as
                                                                                                                       the basis for an improved software baselining definition.
                See answers and expfanations to the practice questions atthe end of the cf,apte(                    C. propose to be included in the project team as a consultant
                (page 235).                                                                                            for the quality ·control of deliverables.
                                                                                                                    D. contact the project leader anO discuss the project plans and
                                                                                                                       recommend redefining the delivery schedule
                                                                                                                       using the PERT methodology.
                                                                                                        See answers and explanations (0 tile practice questions at HIe end of the chapter
                                                                                                        (page 235).

                234                                                                                                                           CISA Review Manual 2011
           ~c:'s3A   Systems,tnfOfmatiM
                     Cerlilied AuditOl'°
                                             Chapter 3-lnformation Systems Acquisition,
                                             Development and Implementation                                                Section Two: Content
                     .. ""..,...... .
 " ,."
"',.- .}
           3.17 ANSWERS TO CASE STUDY QUESTIONS                                      ANSWERS TO CASE STUDY B QUESTIONS
           ANSWERS TO CASE STUDY A QUESTIONS                                         131.   D    The IS auditor has on ly found symptoms but not the
                                                                                                 root causes of the severe problems, so attribu tin g
                                                                                                 them to limited ski lls of the project leader is
           A I.. A         Use ofWEP encryptioJl wo ul d present the most                        inappropriate. The sequence of negati ve events
                           significant ri sk ~ i n ce WEP uses affi xed secret                   is to show that a fundamental problem is causing
                           key that easy to break. Transmission afcred it                        seriolls techn ical difficulties and, as a consequence
                           cardholder information by wireless reg isters woul d                  of delays, nega tively impacting the versioning and
                           be suscept ible to interception, and would present                    change co nt ro l procedures. It is necessary, howevcr,
                           a very serious risk. With regard to the unpatched                     to ascertain inullcdiately the nature of the suspect
                           database servers, since they are located"on a                         problems so if such a problem exists and is not
                           screened s~lbnet, this wou ld mitigate ,the risk to the               foun(~ it could sevcrely affect the final result or kill
                           organizat ion. Sending credit cardholder data over                    the project. Also, the added techn ical programm ing
                           the Internet \\'ould be less of a risk since strong                   resources are an inappropriate remedy because the
                           encryption is being ut ili zed. Since the sales data                 .development methodology is RAD, which is based
                           being sent to the third party·atc aggregate data, 110                 on lIsi ng a small group of skilled and experienced
                           cardholder inforrnat i.on shou ld be included.                        techni cal rcsources. The initial etTects of the problem
                                                                                                 (intermittent block of transactions, corrupted data)
           A2.   A         Locating application servers ill allullsecured                        lead to the slispicion that a problelll might exist iJl lhc
                           area creates a significant risk to the retailer since                 setup and configuration of the technical hardwa rel
                           cardholder data may- be retained 011 tilCserver.                      software environment or platform.
                           Addi tiona lly, physicai access to the server could
                           potentia lly allow programs or dev ices 10 bc installed   132.   A   The only appropriate action is additional resea rch,
                           that would capture sensitive cardholder information.
                           Two-factor aut henticati on for pas registers is not
                                                                                                even if the appare nt ly techn ical nature oflhe
                                                                                                problcm renders it unlikely that the auditor may find
                           as critical. In the case of MAC address filtering, this              it alone. Functi onal project specifications should
                           can be spoofed. Siilce sales data are aggregate, no                  be exec uted by lIsers and systems ana lysts, and not
  .....                    card holder information should be at risk when the
                           data are sent to a third party.
                                                                                                by the auditor. To propose to be project consultant
                                                                                                for qua li ty would not bri ng about an essenti al
                                                                                                contribution since qua li ty is a formal characteristi c,
                                                                                                whereas in the current C(lSC the proble~n is a
                                                                                                substantia l system instabi lity. To contact the project
                                                                                                leader and redesign the schedule of de liveries would
                                                                                                not solve the problem. FUlthermore, the definiti on
                                                                                                of rea l ca uses may sensibly alter the project
                                                                                                envirolUllcnt, inva lidati ng in this way all prcceding
                                                                                                plans and projections .


    c       CISA Review Manual 2011                                                                                                                  235
                                                ~C~IS~A Certir~
                                                         Systems Information
                                                         .. 1S>tO'c.u......

      Page intentionally left blank


236                                   elSA Revie w Manual 2011
'. !          €~,
         Certified Information
                                                        Chapter 4:

          Systems, Auditor'
                                                       Information Systems
                An lSACA' Certi ticahon
                                                       Operations, Maintenance
                                                       and S.u p.p 0 r t

         Section One: Overview
         Defin ition ......... ............ ~ .............................................. :.. H ...... ... .............. : .......................... .. .. .. ..... . . : • •••• • • • • • ••• ••• • •••••••• • ••••• • ••• • ••• •• • •• • •••• 238
         Objectives ...........................................................·.............................................. :............................................... ,................................ 238
         Task and Knowledge Statemcnts ........................................ .............. ,.......................................................................... ................... 238
         Sugges ted Resource.--fOr Furth er Study ..................:................ :............................................................. :...................................... 244
         Practice Question s ........................................................................................................... .................. ,...... ............................... ......... 245
         Ans\1'crs to Practice Questioll s.... ............................ :............:.. ........................................... ..... ........ .............................. ... ................ 246

         Section Two: Content
         4. 1       Qui ck Reference ..............................................:.............. .. .......... ....... ..................... .... ............................................................ 248
         4.2        Infornl at ion Systellls Operations...................... :......... .... ...................................... ....... ........................................................ . 249
         4.3        Infol'Ination Systcrl1S I-Ial'<hvarc .......: .............. :.......:.... .. ................. ...................... ........... .............................................. .. ...... 2SS
         4.4        IS Arc hitecture an d Software .. ....................... :................... ......................................... ................................... " ..................... 260
         4.5        IS Nct\\'ork Infrast ru ct ure ....................................................... ........................................ ..................................................... 267
.. : j
         4.6        Audi tin g Infrastr ucture and Operat ions ............................... .................................. .... ... ;............................................ :....... 288
         4.7        Disaster Recovery Planning ....' ................................................................. ,............ ....... ..................... ...................... .............. 29S
:" '--
 0       4.8        Case Stu d ies ...... ,........... " ............................. :.............................. " .... " .... " .................................................. "."" ............ ......... 305
         4.9        Answe rs to Case Study Q uestions ................,.......................... :.. ...................... ;........ , ........ " ......." ...................... :................ 306





          elSA Review M iwual 2011                                                                                                                                                                                                             237

Se ction One: Overview
                                                         Chap te r 4 - lnforma tion System s Op erations,
                                                                              Maintenance and Support
                                                                                                                           C IS A
                                                                                                                          ~:~        Certified Infat;malion
                                                                                                                                     Systems Auditor"
                                                                                                                                     "<$OU'(oM . . ''''

                                                                           T4. 10 Evaluate the ad equacy of backup and restore provisions
Section One: Overview                                                             to detemline the avai lab ility of information requircd to
                                                                                  resume processing,
                                                                           T4.11 Evaluate the organization's disaster recovery plan to
DEFINITION                                                                        determine whether it enables the recovery of IT processing
                                                                                  capabilities in the eve nt of a disaster.
Informati on systems operations, maintenance and support
practices are important to provide assurance to use rs as well as          KNOWLEDGE STATEMENTS
manage ment that the expected level of service wi ll be delive red.
                                                                           The e l SA cand idate must have a good understandin g oreach
Service level expectations are deri ved fro m th e organi zation's
                                                                           of the domains de li neated by the knowledge statements. These
business objectives. IT service de live ry includes IS operations,
IT services and management of IS and the groups responsible for            statements arc the basis for the examination.
support ing them.
                                                                            Thcrc are 19 knowledge statements within the information systems
                                                                            operations, maintcnance and support domain:
OBJECTIVES                                                                  KS4.1 . Knowledge ofser vic~ level management prac ti ces and
                                                                                      the compeAents wi thin a servi~e level agreement
The objective of this domain is to ensure that the e l SA candidate         KS4.2 KmlVledge of techniques for monitoring tll ird-party
understands aJld can provide assura nce that the processes for                        complia nce' with the organization's internal cOlllrQls
informat ion systems operati ons. mai nte nance and support meet            KS43 Knowledge of operations and end-user procedures for
the organization's strategies und objectives.                                         managing sched uled and nonscheduled processes
                                                                            KS4.4 Knowledge of th e technology concepts related to
This domai n represents 23 percent of the e lSA examina tion                          hardwa re and ne twork components, systcm soflware.
(approximately -16 questious).                                                        and database managcment systems
                                                                            KS4.5 Knowledge of contro l techn iq ues that ensure the
TASK AND KNOWLEDGE STATEMENTS                                                         in tegrity of system interfaces
                                                                            KS4.6 Knowledge of sonwarc licensing and inventory
TASKS                                                                        KS4.7 Knowledge of system res iliency tools and tec hniques
There are I I tusks wit hin the in forlllrt tion systems operations,                  (e.g., fa ult tolerant hardwnre, elimination of single point
main tenance and SlIpport domain:                                                     of failure, clusteri ng)
T4.1 Conduct periodi c reviews of in format ion systems to                  KS4.8 Knowledge of da tabase administ ration practi ces
       .determine \vhethcr they co ntinue 10 meet the orga nization's        KS4.9 Know le~lge of capacity planning and related monitori ng
        objectives.                                                                   tools and techniq ues
T4,2 Evaluate service level ma nagement practices to detenni lle             KS4. IO Know ledge of s),stems performance monitoring
        whether the level. ofservice from inlernal and c>\ternal                      processes, tools and tec hn iques (e,g., network an alyzers.
        serv ice providers is defined and managed.                                    system ut ilization reports, load balancing)
T4.3 Evaluate thi rd-party manage ment practices to determine                KS4.11 Knowledge of prob lem and incident management
        whether the levels of cont ro ls expected by the orga ni zation               practices (c.g., help desk, escalat ion procedures,
        arc being adhered to by th e provider.                                        tracking)
T4.4 Evaluate operations and end-llser procedures to determine               KS4. 12 Knowledge of processes for managing scheduled and
        whether scheduled and nonsched uled processes are                             nonschedu led changes to the production. systems and/or
        managed .to completion.                                        .              infrastructure jncludi ng change, coitfigurati on, re l ~ase
T4.5 Evaluate the process of in fo rma tion systems main tenance                      and patch management practices
        to cletenni ne whelh~r they arc cOIl.t rolled e;ffectively ~nd       KS4. 13 Knowledge of data bhckup, storage, maintenance,
        continue to ,support th~ organ izuti9n's objectives.                          retention and restoration practices
j 4.6 Evaluate data admi n'istrati on practices to d~term il1e the         . KS4.14 Knowledge of regulatory, legal, contractual and
        integrity and optimization of dat·abases.              ,                      insurance issues related to disaster re.covery
T4 .? Evafuate the use·of capacity and performance monitoring                KS4.15 Knowledge ofbllsiness impact a~la l ys is (~IA) re lated to
        tools and lechiliques 10 determine whctherTf services                         disaster recovery planili bg
        mee t the organization's objectives.                                 KS4. t6 Knowledge of the developmerlt and ma intemince of
 T4:8 Eva luate problem and incident managemen t practices                            disaster recovery plans                    .
        1 determine whether incidents, problems or errors arc
         0                                                                   KS4.17 Knowledge of types of alter'nate processing sites and
        recordecl analyzed and resolved in a timely manner,                           methods lIsed to mon itor the contracmal agreements
 T4.9 Evalua te change, configurat ion and release management                         (e.g., hot sites, wa rm sites, cold sites)

        prac ti ces to determine whcther scheduled aryd                      KS4.I S Knowledge of processes used to invoke the disaste r
         nonscheduled changes made to the organization's                              recovery plans
        production environment are adequately controlled and

                                                                             KS4.19 Knowledgc of disaster recovery testing methods

2 38                                                                                                        CISA Review Manual 2011
               ~C~IS~A    SyslelT\$,Iniolmalion
                          Certified Auditor'
                                                  Chapter 4- Information Systems Operations,
                                                  Maintenance and Support                                                                 Section One: Overview

. .

               Relationship of Task               to Knowledge Statements
 .         \   The task statements are what the elSA candidate is expected to know how to do. The know ledge statements delineate what the e l SA
t, ~
       1       candidate is expected to know ill order to pe rform the tasks. The task and knowledge statements are mapped in exhi bit 4.1 insofar as it
               is possible to do so. Note that although there is often overlap, each task statement will generally map to several knowledge statements.
~.vJ                                                              ExhibiI4.1-Task and Knowledge Statements Mapping
                                    Task Statement                                                         Knowledge Statements
                T4.1 Conduct periodic reviews of information KS4A            Knowledge of the technology concepts related to hardware and network compooents, system
                     systems to determine whether they                       software, and database management systems
                     continue to meet the organization's     KS4.6           Knowledge of software licensing and inventory practices
                     objectives.                             KS4.7           Knowledge of system resiliency tools and techniques (e.g., fault tolerant hardware,
                                                                             elimination of single pOint of failure, clustering)
                T4 .2 Evaluate service level management             KS4.1 Knowledge o(service level management practices and the components within aservice level
                      practices to determine whether the                  agreement
                      level of service from internal and            KS4.2 Knowledge of techniques lor monitoring third-party compliance with the organization's
                      external service' providers is defined              internal coottols
                      and managed.
                T4.3    Evaluate third -party management            KS4.2    Knowledge of techniques for monitoring third party compliance with the organization's
                        practices to determine whether the                   internal controls
                        levels of controls expected by the
                        organization are being adhered to by
                        the provider.
                T4,4     Evaluate operations and end-user           KS4.3    Knowledge of operations and end-user procedures lor managing scheduled and
                         procedures to determine whether                     nonscheduled processes
                        ·scheduled and nonscheduled
                         processes are managed to completion.
                T4.5    Evaluate the process of in'forrnation       KS4.4 Knowledge of the technology concepts related to hardware and network compone      nts,system
                        systems maintenance to defermine                  software, and database management systems
                        whether they are controlled effectively     KS4.5 Knowledge of control techniques that ensure the integrity of systern intertaces
                        and continue to support the                 KS4.6 Knowledge of software licensing and inventory practices
                        organization's objectives.                  KS4.7 Knowledge of system resiliency tools and tecbniques (e.g., fault tolerant hardware,
                                                                          efimination of single point of failure, clustering)
                T4 .6 Evaluate data administration practices        KS4.B    Knowledge of database administration practices
                      to determine the integrity and
                      optimization of databases.
                T4.7    Evaluate the use of capacity and            KS4.9 Knowledge of capacity planning and related monitOring tools and techniques
                        pertormance monitoring tools and            KS4.10 Knowledge of systems pertormance monitoring processes, tools and techniques (e.g.,
                        techniques to determine whether                    network analyzers, system utilization reports. load balancing)
                        IT services meet the organization's
                T4.B    Evaluate problem and incident              . KS4.11 Knowledge of problem and incident management practices (e.g., help desk, escalation
                        management pracfices to determine                   procedures, tracking)
                        whether incideilts, problems or errors
                        are recorded, analyzed and resolved in
                        a timely·manner. .
                T4.9      valua.
                         E te change, Configuration and             KS4.12 Knowledge of processes lor 'managing scheduled and nonscheduled changes to the
                         release management practices to                   production systems and/or infrastructure inclUliing cha.nge, configuration, release and patch
                         determine whether scheduled and                   management practices
                         nonscheduled changes made to the
                       . organization's production environment
                         are adequately controlled and
                T4.10 Evaluate the adequacy of backup and           KS4.13 Knowledge of data backup, storage, maintenance, retention and restoration practices
                      restore provisions to determine the

                      availability of information required to

                      resume processing .

     C         CISA Review M anual 2011                                                                                                                              239
 Section One: Overvie w
                                                                  Chapter 4-lnformation Systems Operations,
                                                                                   Maintenance and Support
                                                                                                                                           ~C~IS~A     Cerlilled Auditor'
                                                                                                                                                       Svstems Information
                                                                                                                                                       .. <UC.o',.. _

                                               Exhibit 4.1-Task and Knowledge Statements Mapping (cont)
                  Task Statement                                                                   Knowledge Statements
  T4.11 Evaluate the organization's disaster          KS4.13 Knowledge of data backup, storage, maintenance, retention and restoration practices
         recovery plan to determinewhether it         KS4.14 Knowledge of regulatory, legal, contractual and insurance issues related to disaster recovery
         enables the recovery of IT processing        KS4.15 Knowledge of business impact analysis (BIA) related to disaster recovery planning
       _ capabilities in the event of adisaster.      KS4.16 Knowledge of the development and maintenance of disaster recovery plans
                                                      KS4.17 Knowledge of types of alternate processing sites and methods used to monitor the
                                                             contractual agreements (e.g., hot sites, warm sites, cold sites)
                                                      KS4.18 Knowledge of processes used to invoke the disaster recovery plans
                                                      KS4.19 Knowledge of disaster recovery testing methods

 Knowledge Statement Reference Guide
 Each knowledge statement is ex plained in terms of underlying concepts and re levance of the knowledge statement to the IS auditor. II
 is essential that the exam candidate understand the concepts. the knowledge statements are what the IS auditor must know in order to
 accomplish the tasks. Consequently. only the knowledge statements are detai led in this section.

 The sections identi fied in KS4.1 ·KS4.19 arc described in greater detail in secti on two of this chapter.

 KS4.1 Knowledge of service level management practices and th e components within a service level agreement
                                    Explanation                                      I        Key Concepts          I Reference in 2011 CISA Review Manual
  Service level management ensures IhaIIT services meet customer's                       Understanding best          4.2.2 IT Service·Management
  expectations and that service level agreements (SLAs) are conlinuously                 practices for service
  maintained and improved as needed. SlAs are generally separate documents               level management
  from the contracts with external vendors. Although generally associated with
  oulsourced functions, the ISauditor should be aware that SlAs may also be
  created "ternally to assure key process owners of the level of service that
  the IT organization has agreed to provide. SlAs may include technical support
  elements such as expected response times; system availability (e.g., 08.00 to
  18.00, Monday through Friday);'help (or service) desk responses and escalalion
  procecures, etc. Therefore, SlAs specify tile underlying operational specincs
  for agreed-upon services which, if measured and managed, will deliver the
  commitments that meet custom r expectations. .

KS4.2 Knowledge of techniques 'for monitoring third-party compliance with the organization's internal controls
                                    Explanation                                      I        Key Concepts          I Reference in 2011 CISA Review Manual
   With the increasing trend of outsourCing infrastructure to third-party service        Impact of sourcing          2.1 0.2 Sourcing Practices
   providers, it is essential to know the latest approaches in contracting               practices on IT
   strategies, processes and contract management practices. Outsourcing                  governance
   IT (and related splutions such as process management and infraslructure
   management) can help reduce costs and/or complement an enterprise's                   Relationship between        2.11.1 IS Roles and Responsibilities
                                                                                         vendor management .
   own expertise;.however, outsourcing also may introduce additional risks.
   Thus, it is essential for the IS auditor to understand the latest approaches in       and IT governance of the
   contracting strategies, processes and contract management practices, such             outsourCing entity
   as which critical concepts must be included in an outsourcing contract and        Contractual lerms and .         2.12.2 Reviewing 'Contractual Commitments
 . business case requirements:                                                       their impact on driving
                                                                                     ITgovernance of the
                                                                                     outsourcing entity .

. KS4.3 Knowledgeof oper~tlons and end-user procedures for managing scheduled and nonsch~rJ.ulediJrocesses ·
                                    Explanation                                      I        Key Concepts          I Reference in 2011 CISA Review Manual
 Operations management is critical in providing effective, efficient and        . Understanding best                 4.2   Information System Operations
 appropriate technical solulions. Theroles and responsibilities of operations     practices for operalions           4.6.4 Network Infrastructure and
 management represent a high risk, not only to the day-to-day running of the management                                    Implef1)entalion Reviews

 IT organization, but to the protection of information assets both in the areas                                      4.6.6 Scheduling Reviews
 of restricting access to authorized people and the availability of IT. The IS
 auditor must understand operations management practices and controls to
 ensure the deliverY of quality IT services to the business and to ensure the
 security of the information.
240                                                                                                                       CISA Review Manual 2011
     ~C~IS~A    Ceilirledlfl'ormation
                                        Chapter 4- Information Systems Operations,
                                        Maintenance and Support                                                                Section One: Overview

     KS4.4 Knowledge of the technology concepts related to hardware and network components, system software, and
=.   database management systems
                                           Explanation                               I        Key Concepts         I Reference in 2011 CISA Review Manual
      The IS auditor must be familiar with the functionality of information              Understanding the key       4.5   ISNetwork Infrastructure
      system hardware and network components. This includes understanding                network and hardware        4.6.1 Hardware Reviews
      the importance of the physical part of alilSllTsolutions that support the          components of a typical     4.6.4 Network Infrastructure and
      organizational objectives and goals. Although the elSA exam does not               data center                       Implementation Reviews
      test technical knowledge of the working of individual components, an
      understanding of the 'risks associated with and possible control functions         Understanding the           4.4.1 Operating Systems
      of each component is expected-for example, the risk that router access             key controls and risks      4.6.2 Operating System Reviews
      passwords may be shared but that, if properly programmed, passwords can            involving system            4.4.7 Utility Programs
      make a m contribution to network resilience.
                 ajor                                                                    software

      The IS auditor should understand basic concepts related to system software.
      Application software resides within the environment controlled by the
      operating system, but other system software such as utilities, security
      management, etc., may have a material effect on the security, reliability,
      integrity and availability of both applications and data. System software
      issues are extremely important since all applications within the environment
      will be impacted and controls at the application level may be subject to
      circumvention at the system software level.

     KS4.5 Knowledge of control techniques that ensure the integrity of system interfaces
                                           Explanation                               I        Key Concepts         I Reference in 2011 CISAReviewManual
      System intertaces, including middleware, application program intertaces            Understanding the          4.5.6 Application of the OSI Model in
      (APls), and other similar software, present special risks because they may         key controls and risks           Network Architectures
      not be subject to the same security and control rigor that is found in             involving system
      large-scale application systems. The IS auditor needs to understand how            intertaces
      these system intertaces are controlled and secured. Management 'should
      ensure that systems are properly tested and approved, modifications are
      adequately authorized and implemented, and appropriate version control
      procedures are followed.

     KS4.6 Knowledge of software licensing and inventory practices
                                           Explanation                               I        Key Concepts         I Reference in 2011 CISA Review Manual
      The IS auditor should be aware that the use of unlicensed software, also      Understanding key                                 se
                                                                                                                    4.2.4 Monitoring U of Resources
      known as piracy, is regarded as unlawful throughout the world, although       controls for software
~     specific legislation may not be in force in every country. Software licensing licensing
      should be subject to controls to ensure that the number of copies in
      circulation within an organization does not exceed the number purchased.
      The ISauditor should understand the different methods of software licensing
      (per seat, concurrent users, enterprise licenses, etc.) and the wayS in which
      automated tools can be utilized to inventory the number of software products
      in use and to prevent and detecf the use of unlicensed software.
       '          .                    .               .             .
     )(S4.7 Knowledge of system resiliency tools and techniques (e.g" fault tolerant hardware, £iiimination of single '
     paint of failure, ciustering)                                             .
                                           Explanation                               I       Key Concepts          I Reference in 2011 CISA Review Manual
       System resiliency tools and techniques are important to ensure uninterrupted Understanding best '            4.2.2    IT Service Management
-,     service. The IS auditor should be able to·identify potential single pOints of . practices for ensuring
       failure wilhin aprocess and understand related tools and techniques-such system resiliency
       as high availability, load balancing and clustering solutions-utilized to
      improve system resiliency.

     CISA Review Manual 2011                                                                                                                                241
Section One: Overview
                                                               Chapter 4-lnformation Systems Operations,
                                                                                Mainte nance and Support                               ~C~ls3A   Certirled Auditor'
                                                                                                                                                 Systems IlllGl'malion
                                                                                                                                                 .. I!.AU'~ . ...

KS4.8 Knowledge of database administration practices
                                  Explanation                                     I        Key Concepls         I Reference in 2011 CISA Review Manual
 It is necessary for the IS auditor to understandtheconcepts of database              Understanding key           4.4.5 Database Management System
 design, database administration, relationships between database objects,             control areas within        4.6.3 Database Reviews
 potential problems in transaction processingand secunty issues associated            database administration
 with database management systems (DBMSs), especially when auditing such              and security
 systems. The roles and responsibilities of, such as those of
 the database administrator (DBA), shouldbe understood as should the control
 practices associated with those roles and responsibilities, and the technology
 managed by key personnel.

KS4.9 Knowledge of capacity planning and related monitoring tools and techniques
                                  Explanation                                     I        Key Concepls         I Reference in 2011CfSA Review Manuaf
 Capacity planning ensures that all the current and future capacity and               Capacity planning and       4.3.4   Capacity Management
 pertormance aspects of business requirements are antiCipated in advance;             monitoring
 assessed; and, as necessary. provided in acost-effective manner. Capacity
 of information systems must be monitoredon acontinuous basis to meet
 business needs and should be planned using projections of future expecfed
 demands. Capacity includes the size andspeed of the processor, infernal
 system memory, and storage and communications media. The IS auditor
 is expected to be aware of the concepts of capacity management and the
 essential information requirements of fhe task, such as technical pertormance
 reports and information on projected business needs. Adetailed knowledge of
 the ottencomplex mathematical models usedby the process is not esSential.

KS4.10 Knowledge of systems performance monitoring processes, tools and techniques (e.g., network analyzers,
system utilization reports, load balancing)
                                  Expfanation                                             Key Concepls          I Reference In 2011 efSA Revfew Manual
 IT pertorrnance monitoring of critical processes andassets should be conducted U  nderstanding best             4.2.4    Monitoring Use of Resources
 on acontinuous ba~s to ensure reliable IT services that meet SLAs and achieve practices for systems
 defined business objectives. Pertormance monrtoring processes must be            monitoring
 establishedwith supporting tools and techniques, and although the C exam
 does not test knowledge of specific tools, the IS auditor should be aware of the
 importance of monitoring and of basic techniques that may be employed.

KS4.11 Knowledge of problem and Incident management practices (e.g., help desk, escalation procedures, tracking)
                                  Expfanation                                     I       Key Concepts          I Reference in 2011 efSA Review Manual
 An incident is any event that causes tem    porary disruption to the business.A Understanding best              4.2.4                se
                                                                                                                          Monitoring U of Resources
 problem may develop when such incidents are unresolved. The underlying          practices for problem
 cause of an incident may also be identified as a problem and addressed as       management
 such. All incidents or problems must be detected, reported, managed and
 resolved in a timely manner. A problem management tool should be used
 that can be checked by the IS auditor for evidence of satisfactory problem.
'resolutionand for the ability to identify irends in incidents and root cause'
 analysis, which'may'point 10 an underlying problem . .

KS4.i2 Knowledge of processes for managing scheduled.and nonscheduled changes to' the productio'n .syste~s
and/ or infrastructure inclucJing change, configuration, releas.e and patch management practices
                                  Expfanatlon                                     I       Key Concepls          I Reference in 2011 elSA Review Manual
All changes to the production systemor infrastructure should be approved          Best practices for             4.2.6    Change Management Process,
according to an established change management process. Adequate                   change management
separation of duties should be enforced-for example. to ensure that the
person making the change is not the same person approving the change.
The IS auditor should also be aware of the need for established procedures
to control changes made to systems inan emergency situation-such as
when aprogrammer has been called in tnaddress issue$ following asystem                                                                                                   •
stoppage. In such Circumstances, it is often necessary for the programmer
to have access to production systems, which then breaches the control                                                                                                    •
of "division of duties." Logging of activity:together with management
verification and postamendment approval, is an essential requirement.
242                                                                                                                      CISA Review Manual 2011
           IS~A .......... ...
         ~C~ ",,,r..,0'''''''''''
                    Systems Auditor'
                                       Chapter 4 - Information Systems Operations,
                                       Maintenance and Support                                                                            Section On e: Overview

         KS4.13 Knowledge of data backup, storage, maintenance, ret ention and rest oration practices
                                           Explanation                                                Key Concepts           I Reference In 2011 CISA Review Manual
          An IS auditor should undersland the relationship between backup/recovery               U nderstanding backup         2.14 Auditing Business Continuity
          plans and business process requirements. It is essential that critical data be         strategies including:         4.7.2 Recovery Strategies
          available in the event of data loss or contamination. Data must be backed              media rotation and            4.7.6 Backup and Restoration
          up, available at a location ttrat is not likely to be impacted by a disaster           proper storage,
          at the primary site and protected (i.e., physically secure and encrypted, if           data protection, and
          necessary). An organization should have documented policies, processes,                relationship to recovery
;.-. ?                                                                                           time objective (RTO)I RPO
          procedures and standards that clearly explain data backup and recovery.
          The IS auditor is expected tounderstand that without backup, no disaster
          recovery plan (DRP) can work; that backup should be taken at appropriate
          intervals according to bu~ness need, as determined by the recovery point
          objective (RPO); and that backup must be securely transported for storage
          in an "offsite" location so backup will be available in the event of a seriously
          disruptive incident affecting its host site.

         KS4.14 Knowledge of regulatory, lega l, contractual and insurance issues relat ed to disaster re covery
                                           Explanation                                       I        Key Concepts           I Reference in 2011 CISA Review Manual
          An IS auditor should know how to analyze the degree to which the business              Understanding BCP        2.13.1 IS Business Continuity Planning
          continuity plan(BCP)/D Pis aligned with regulatory, legal, contractual
                                  R                                                              regulatory requirements, 4.7.3 Recovery Alternatives
          and insurance requirements. Business continuity and disaster recovery                  third-party contract
          strategies often depend, to varying degrees, on third-party service providers.         provisions and insurance
          Contractual terms determine the obligations 01 third-party vendors. who
          are part of the D P/BCPsolution. BCP may also be mandatory depending
          on various regulatory or legal requirements. Additionally, insurance is an
          important component of the risk mitigation strategy, in terms of transfer of
          risk, and the IS auditor must be aware of the need to maintain an insurance
          valuation commensurate with the enterprise technology inlrastructure.

         KS4.15 Knowledge of business impact analysis (BIA) related to disaster recovery planning
                                           Explanation                                       I        Key Concepts           I Reference in 2011 CISA Review Manual
          An IS auditor must be·able to determine whether a BIA and BCP are suitably         Understanding the BIA            2.13.6 Business Impact Analysis
          aligned. To be effective arid efficient, BCP should be based on a well-            as a key driver of the
          documented BIA. ABIA dnves the focus of the B.GP efforts of an organization        BCP/disaster recovery
          and helps in balancingcosts to be incurred with the corresponding benefits                     RP)
                                                                                             planning (D process
          to the orga·nization. Agood understanding of the BIA concept is essential for '
          the IS auditor to audit the effectiveness and efficiency of aBCP.

         KS4.16 Knowledge of the development and maintenance of disaster recovery plans
                                           Explanation                                       I       Key Concepts            I Reference in 2011 elSA Review Manual
          An IS auditor well'versed in the practices and techniques followed       Understanding the                2.13.1 IS Business Continuity Planning
           for development and maintenance of BCPS/DRPs, including the need to .             life cycle of BCP/DRP            2.13.3 Business Continuity Planning Process
           coordinate recovery plans acroSs the organization. Plans should be tailored       development and                  2.13.4 Business Continuity-Policy .
           to fit tlie individual needs of organizations because diHerences in industry,     maintenance                      2.13,5 Business Contin'uilY Planning Incident
           size and scope of an' or~an ization ; and_even geographic'location, can affect                                             Manageinent .
           the contents of the plans.The size and nature of the selecied recovery facility                                    2.13.7 Development of Business' Continuity
           for technology will materially depend on the financial risk associated with                                               . Plans
j          disruption. In essence, the faster the required recovery, as determined by the                                     2.13.8 .Other Issues in Plan Development
         . RTO, the greater the potential cost. Once established, recovery plans must be                                      2.13.11 Summary of Business Continuity
           kept up-to-date with changes in the organization and with associated risks ..                                      4.7.2 . Recovery Strategies·           .

         elSA Review Manual 2011                                                                                                                                     243
 Section One: Overview
                                                                 Chapter 4-lnformation Systems Operations,
                                                                                  Maintenance and Support
                                                                                                                                                      Certified I"formation

                                                                                                                                                      . ,
                                                                                                                                                        ~' '''''''-

KS4.17 Knowledge of types of alternate processing sites and methods used to monitor the contractual
agreements (e.g., hot sites, warm sites, cold sites)
                                   Explanation                                         I       Key Concepts           I Reference In 2011 elSA Review Manual
 An IS auditor should be able to analyze whether an enterprise's selection of             Understanding alternate      4.7.2 Recovery Strategies
 an alternate processingJacility is appropriate. given the cornpany's recovery            processing options.          4.7.3 Recovery Alternatives
 requirernents. The company should make provision for alternate processing                the advantages and
 facilities to sustain critical inforrnation systems in the event that the prirnary       disadvantages of each,
 inforrnation systems become unavailable. The alternate processing site                   and the methods used to
 should meet the defined business requirements. An IS auditor should                      monitor the contractual
 understand the differences among alternate processing site types and be                  agreements with a third -
 able to evaluate whether the type selected is aligned with and adequate to               party provider
 meet the defined business requirements. as established in the BCPIDRP.
 Solutions may involve. in descending order of recovery speed •. duplicate
 facility; a hot site; a warm site; cold Site; a contracted site, including the
 provision of a mobile facility; and reliance on vendors and a reciprocal

 Tile IS auditor should understand how contracts can be structured to
 accommodate the differing requirements of the business, depending on the
 intensity of the disaster.

KS4.18 Knowledge of processes used to invoke the disaster recovery plans
                                   Explanation                                        I        Key Concepts           I Reference in 2011 elSA Review Manual
 An IS auditor should understand the concepts behind the decision to declare            Understanding                  2.13.5 Business Continuity Planning Incident
 a disaster and to invoke a·BCP/DRP and should understand the impact o(the              best practices for                      Management
 decision on an organization, remembering that invocation of the BCP/DRP                communicating the              2.13.7 Development of Business Continuity
 can, in itself, be a disruption. Key elements that the IS auditor should ensure        declaration of a disaster               Plans
 are in place include clear instructions to individuals who have tile authority to                                     2.14.6 Reviewing Alternative Processing
 declare a disaster. identification of staff who will step into a decision-making                                               Contract
 role if the primary decision maker should be incapacitated or otherwise
 unavailable, and' steps to ensure that the disaster declaration is properly

KS4.19 Knowledge of disaster recovery testing methods
                                   Explanation                                                Key Concepts            I Reference in 2011 elSA Review Manual
  An IS auditor should know the testing approaches and methods for BCP/DRP             Understanding the types         2.14.4 IntervieWing Key Personnel
  to evaluate the effectiveness of the plans. To ensure that the BCP/DRP will          of Bep tests, factors fa
  work in the event of a disaster. it is important to periodically test the            consider when choosing
  BCP/DRp, and ensure that the testing effort is efficient. The role of the IS         the appropriate test
  auditor is to observe tests, ensure that all "Iessons.learned" are properly          scope, methods for
  recorded and reflected in a revised plan. and review write-ups documenting           observing recovery
  previous jests. Key items to look for include the degree to which the test           tests and analyzing test
 ·Ieverages resources or extensiVe prepl111lning meetings that would noi               results
  be available during an actual disaster. The objective of a test should be
  to identify gaps that can be improved, rather than to have a flawless fest.
  Another important aspect of·DRP/BCp·testing is to'provide training for
  management and staff who may be involved in fhe recovery process.

SUGGESTED RESOURCES FOR FURTHER STUDY                                                 ,Schneier, Bmce;Secrets & ties: ' Digit(l/ Securi'), in'a Networked
                                                                                       If" rld, John Wiley & SOilS, USA , 2004
Hiles, And rew; Busilless COlltilluity: Best PractiCes- World-
class Business COlllillui(r Management, 2nd Editioll , Rothstein                      Slledakel, Susall ; Busil1ess Continuity & Disaster Recove/J'
Associates Inc., 'USA, Dccember 2003                                                  Plallllil/g/or IT Pro/essiol1uJs Syngress Publishin g Inc.,

                                                                                      USA, 2007
ISACAj Cybercrime: Incidem Rejpollse alld Digital ForeJlsics.
USA,2005                                                                              \Vells,April j Ch.1rlyne "Valkcrj Timothy Walker; Disaslel'
                                                                                      Recovel)': Pri/lcll )i es a/lei Plne1ices, USA, Pearsoll-Prentice                       •
Nationa llllstitute or Standarcls and Tec hnology, "Security
CO ll s id~ ration s for Voice Over IP Systems," USA , 2005
Note: Publicflfiolls ill bold (Ire stocked ill the ISA CA Boo/(store.                                                                                                         •
244                                                                                                                         elSA Review Manual 2011

Shared By: