Docstoc

Risks

Document Sample
Risks Powered By Docstoc
					ID Issue




   High LOE of implementing a standalone
I1 credentialing service




I2 Development scope for Year 1 is too high




I3 QA testing has fallen far behind development
   The lightweight approach to identity
   verification, authentication and digital
I4 signatures was deemed unsatisfactory.
Description                                                Owner




Our original proposal of leveraging the NES code base to
implement the credentialing service was rejected by the
ESST. The mandate to implement a standalone
credentialing service significantly increases the LOE.     Rashmi Srinivasa




As the use cases and user stories for various parts of the
application get fleshed out, it has become clear that all of
the scope originally proposed for Year 1 will be impossible
to accomplish with the current team size.                    Rashmi Srinivasa


After the latest QA team member transition, QA has made
little to no progress on testing of the delivered
functionality. As of June 28, 5 iterations (24, 25, 26, 27
and 28) are untested by the QA team.                       Rashmi Srinivasa
The originally-proposed lightweight approach to identity
verification, authentication and digital signatures was
deemed unsatisfactory, and the LOE for multi-factor
authentication is much higher.                             Rashmi Srinivasa
                                           Priorit   Statu
Actions                                    y         s

Since there is no known user for the
credentialing service other than
FIREBIRD, the credential managemen
service implementation has been pushed
to Year 2. This has been approved by the
COTR and John Speakman.                  High        Closed




We have proposed an alternative scope
and schedule which has been accepted by
the COTR, John Speakman and Dr. Sheila
Prindiville.                            High         Closed

The new QA tester started in July 2011,
and we will work with him to ensure that
QA catches up with the last 5 untested
iterations, and starts on an automated
regression testing suite.                  High      Closed
For the moment, we are proceeding with
the single-factor approach in the interest
of getting the rest of the application built.
The schedule will need to be revisited
once the scope for multi-factor
authentication becomes clearer.               High   Open
Comments




After clarifying requirements and understanding the system, it has
become clear that a full-featured FIREBIRD 3.0 application cannot be
delivered by this summer. Some of the issues that caused the revision
of schedule are:

1. We could not leverage any significant portions of the existing code
base from the old FIREBIRD. This was because of the old tech stack it
used, as well as the need to redesign and structure our application
front-end and the database/object model.

2. After requirements clarification, some features turned out to have
a higher LOE than originally estimated. E.g., investigator invitations
is not simply a matter of sending an email inviting an investigator to
register, but is more complex due to the interactions between the
statuses of an in progress registration packet and how the user is
interacting with that packet. The system can't merely send out an
email to an Investigator, it must manage what the Investigator says
they will do, handle what to do when an Investigator doesn't exist and
how they might be linked with this after coming into the system,
handle what happens when a user will not do things in our system at
all, etc.

3. BDA / Systems requirements for application installation changed,
causing us to spend a large chunk of time to rework the installation
set up due to changed requirements from the Systems team.

4. Integrating with the NES took more time than anticipated, in part
because of the need to understand the intricacies of the information
that needs to be passed to and received from the NES. This involved
not just reading the available documentation but also interacting with
the NES team to fully understand the complexity of the integration.
The documentation is out there, but it takes a reasonable amount of
research to find all the right information and the understand the




QA testing has now caught up with the latest iteration.
ID      Risk                                     Description
                                                 Credentialing work started earlier for COPPA may
                                                 bias the expectations for the FIREBIRD credentialing
      1 COPPA credentialing bias                 service.
                                                 It is unclear who has the authority to approve
        No clear authority to approve our        requirements and design for digital signatures,
        digital signatures (Part 11              including Part 11 compliance, which may delay
      5 Compliance) design.                      approval and increase the risk of rejection.


                                                 Due to negative experiences with previous
                                                 incarnations of FIREBIRD, DCP may not fully
      6 DCP reluctance in adoption               embrace the project.




        Poorly defined scope for digital     The price proposal assumed a lightweight identify
                                             verification requirement. If the requirement is more
        signature services (including identify
        verification) means that the LOE     complex, the implementation may be more costly
      8 may have to be revised later.        than the contract can sustain.
                                             The scope of the credentialing service is poorly
        Poorly defined scope for             defined at the moment. When it does become well
        credentialing service means that LOE defined, the LOE may turn out to be higher than
     11 may have to be revised later.        originally expected.

                                                 As the use cases for various parts of the application
                                                 get fleshed out, it is becoming unlikely that all of the
        Development scope for Year 1 may         scope originally proposed for Year 1 will be possible
     13 be too high                              to accomplish with the current team size.

                                                 Since FIREBIRD supports FDA-regulated transactions
                                                 and maintains FDA-regulated documentation, it is
                                                 beholden to 21 CFR Part 11 Compliance. Compliance
                                                 is typically demonstrated through independent
                                                 validation against Part 11 requirements, which is not
                                                 in the remit of the Firebird team and is not typically
                                                 the remit of the development team but rather an
     15 No testing of Part 11 compliance         independent QA body.




                                                 Leveraging the NES may limit performance and
      3 NES Limitations                          functionality
                                      If we're able to help OEWG implement some
   Not aligning to CTEP OEWG goals    mandates, we may be able to enlist their support;
   may push FIREBIRD to the bottom of conversely, if we can't help, FIREBIRD may be
 4 their priorities.                  pushed to the bottom of their priorities.




                                       Currently, curation of persons and organizations
                                       depends on the CTRO/CTRP contract and their
                                       deployment timelines. If curation isn't available
                                       when FIREBIRD is ready to deploy, CTEP and DCP
 7 Dependence on CTRO curation         won't use it.

   CTEP-NES integration and presence Any delay in CTEP integration and use of NES for
   of investigators and organizations in person/org management may impact CTEP's ability
 9 NES may be delayed.                   to use Firebird.




                                       Old incarnations of FIREBIRD developed a reputation
   Poor legacy reputation may impact   for being excessively complicated and difficult to use,
10 adotion                             which may impact adoption going forward.




                                       Investigators and others users may not be available
                                       to support the gathering and refinement of
                                       requirements. Limited input from end users may
                                       negatively impact the quality and usability of the end
12 Unavailability of users             product.
                                        The DCP has expressed a desire for all of the NCI to
   DCP's desire to move to one NCI-     move to one way of doing registrations. If they
   wide process for registration may    switch to doing annual registrations only, then our
   make per-protocol registration       per-protocol registration implementation will become
14 unnecessary                          wasted effort.

                                      Several user stories that were not previously
                                      recorded on the backlog have now been identified
                                      after clarifications with the DCP. This means that the
                                      backlog is quite bigger than it was at the time the
16 Previously-unrecorded requirements project schedule was determined.

                                        There are duplicate organizations (same name but
                                        different addresses) and misspelt organization names
                                        in NES. This makes it difficult for users of a consumer
   Some Person-Organization data in     application like FIREBIRD to determine which person
17 NES is incorrect.                    or organization to select.
   Increase to scope if CTEP IAM
18 integration is required              Increase to scope if CTEP IAM integration is required
   Investigator dissatisfaction and
   increase to scope if Dorian to NCI   Investigator dissatisfaction and increase to scope if
19 LDAP migration is required           Dorian to NCI LDAP migration is required
Owner         Impact   Mitigation Plan                                           Priority


Nobody        Medium   Define vision and get it reviewed by stakeholders.        Low


                       Identify the stakeholder(s) with authority to approve
Nobody        High     requirements and design.                              Low
                       Demonstrate early on that FIREBIRD 3.0 is addressing
                       some the key issues that contributed to DCP's earlier
                       negative experience. Continue to demonstrate in each
                       subsequent demo that we continue to address their
David Loose   High     concerns.                                             High

                       Clearly document our understanding of the vision for
                       this service based on the historical FIREBIRD
                       requirement discussions and current industry
                       practices. Identify the stakeholder(s) with authority to
                       approve requirements and design (Braulio Cabral,
                       George Komatsoulis and Bruce Woodcock - CIO).
                       Review our proposed approach with them, highlighting
David Loose   High     the impact of increasing the complexity.                 Low




Nobody        High     Get our vision and design reviewed by the ESST.           High

                       Break down the high-level adoption priorities into
                       tasks, estimate the LOEs and compare to the team
Rashmi                 velocity to see if scope needs to be revisited.
Srinivasa     High     Reprioritize tasks instead of full adoption priorities.   High




Hemant Undale Medium   QA team responsible for Part 11 compliance testing.       Low
                       Monitor the performance and functionality
                       improvements of NES as we approach our release.
                       Raise visibility of issues through demos and briefs.
                       Consider saving up updates and sending a batch
Eric Tavela   High     request to NES.                                           High
                       Discuss with CTEP PIO Chief on possible ways for
                       Firebird to aide OEWG response. Due to divergent
                       timelines, we may only be able to assist with the
                       broader long-term goals of reduced study inititation
                       timelines, but the CTEP investigator registration
                       process is not currently a major delay to study
                       initiation due to a number of established release points
                       (i.e. ways to push through study activation without
David Loose   Low      waiting on PI registration)                              Low




                       We are looking to SAIC-F to manage this risk as this
Hemant Undale High     has to do with the CTRO/CTRP contracts.                 Medium


                       We are looking to SAIC-F to manage this risk as this
Hemant Undale High     has to do with the CTRP contracts.                      High

                       Engage with DCP in the first year to gain their
                       sponsorship; Hold regular stakeholder meetings to
                       solicit feedback on requirements and design, and
                       demo newly implemented functionality. Expand to
                       CTEP in the following year. Engage with outside/other
                       stakeholders throughout to keep them apprised of
David Loose   Medium   progress and demo new developments.                   Low

                       The team has begun regularly engaging with DCP, as
                       per the adoption plan for engaging stakeholders/users.
                       We have executive sponsorship at DCP in Dr. Leslie
                       Ford; but we also need to keep the operations people
                       happy. To that end, David and Tanya we plan to do
                       monthly updates at a minimum to keep the core
                       operations group engaged: Judy Smith and Margaret
David Loose   High     Schetrum.                                              High
                       The likelihood of such a switch to annual registration is
                       low, especially in the short term. We are keeping DCP
                       engaged so that we are aware of any such
                       developments. Our team follows an Agile approach, so
                       changes in requirements can be handled relatively
David Loose   Medium   smoothly.                                                 Low




Rashmi                 Streamline process, prioritize backlog tightly,
Srinivasa     Medium   communicate risk.                                        Medium




                       Display more information than merely the name so
Hemant Undale Medium   that the user can select appropriately.                  Medium
Rashmi
Srinivasa     High                                                              High


Jose Galvez   Medium                                                            Medium
Status   Comments/Resolution
         Braulio Cabral, ESST, and other stakeholders have reviewed
         and accepted the vision for this service described in the
Closed   Credentialing Service Vision Document.


         Braulio Cabral and George Komatsoulis have been identified as
Closed   the authorities.


         We have set up regular monthly meetings with the DCP, and
         are keeping them engaged. They have provided positive
Closed   feedback.




         The lightweight approach was deemed unsatisfactory, and the
         scope of identity verification and authentication will be higher if
         multi-factor authentication is desired. This increases LOEs, and
Closed   this risk has turned into issue I4.

         The ESST has reviewed and mandated a standalone
         credentialing service that does not sit in the NES codebase.
Closed   This increases LOEs, and this risk has turned into Issue I1.




Closed   See Issue I2.




Closed   21 CFR Part 11 compliance testing is not in scope.




Open
       We can help the overall issue of study initiation, but CTEP has
       already had to implement OEWG responses, and we're not
Open   ready yet


       We believe that adding Firebird entity curation (e.g. Persons &
       Orgs) as a CTRO curation task will not result in a greater total
       sum of work. Almost any entity requiring curation in Firebird
       would ultimately need to be curated for CTRP purposes since all
       CTEP/DCP protocols will be registered in the CTRP database,
       including all PIs. Further, the vast majority of PIs (15000) are
       already in CTEP's system and have been made available to the
       CTRO/CTRP. DCP may result in the addition of a few hundred
       additional PI person records over the entire lifecycle of Firebird,
       but will largely rely on the already existing universe of Persons
       in CTRP/CTEP-ESYS. Creating another curation body within the
       NCI will require greater coordination both technically and
Open   operationally between the CTRO and Firebird curation bodies.


       CTEP-NES integration is expected to be done as part of the
Open   CTRP contract, but this effort is currently stalled.




       We have had several meetings with the DCP so far, and have
       set up monthly meetings to keep them engaged. We will
       engage CTEP more closely once we have implemented annual
Open   registration.




       We have had several meetings with the DCP so far, and have
       set up monthly meetings to keep them engaged. DCP is
       definitely interested in participating in UAT of the sponsor side,
       and discussed finding a DCP PI to perform UAT of the
Open   Investigator side.
Open




       The team has streamlined its process and has been tackling
       more items in every iteration. We hope that we can still the
Open   meet the schedule since our velocity is now greater.




Open

Open


Open

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:2
posted:10/21/2011
language:English
pages:17