Data Retention

Document Sample
Data Retention Powered By Docstoc
					Data Retention
DAMA - Portland      February 15, 2005

          More Value…
               Less Filling!
    •   Introduction
    •   The Sea of Data
    •   The Mountain of Regulations
    •   Swimming With the Sharks
    •   The Data Retention Roadmap
    •   A Data Lifecycle
    •   Draining the Swamp
    •   A Data Retention Topology
    •   Example Data Reduction
    •   Wrap-up

                  2005 DAMA Presentation   2

   “ One of the tests of leadership is the
     ability to recognize a problem before it
     becomes an emergency. ”

                                         Arnold Glasgow

                2005 DAMA Presentation                    3
    • Unstructured Data Is growing at an alarming rate with
      media now an integral part of many businesses.
        – Email, Images, Scanned Forms
        – Video and Audio clips
    • Regulatory Impacts on data retention are growing, a
      trend not likely to change.
    • Virtually all corporate litigation now includes some form
      of electronic information forensics in discovery
    • Companies are now fined not only for not having data
      that is under retention regulation, but for not having it
      readily available and understandable.

    IT needs to help put the business on a “diet” that sheds
       itself of risky, hollow information assets.

                       2005 DAMA Presentation                     4
The Sea of Data

          2005 DAMA Presentation   5
The Sea Of Data…
     •   In 1967 there were less than 5000 Computers world wide.6
     •   2002 UC Berkley Worldwide Data Volumes 1
          –   >3 million Server / Multi-user Class Machines
          –   400 million PCs
          –   250 Mb / person / year as of 2002.
          –   97% of corporate documents are electronic.
          –   < .003% of all information is in paper print.

     •   IDC Worldwide E-Mail Study – 2002 2
          –   ~ 60 Billion email / day – increasing at a rate of 25% / yr
          –   ~ 25 Billion wireless text messages / day
          –   Average corporate mailbox contained >1000 messages with
          –   > 50 Mb of content.

                       2005 DAMA Presentation                               6
Common Data Centric Architecture
    Operational                                               Analytics
    App and Web                            Data                 and
      Services    App and Web
                                        Warehousing           Reporting

      HR           Attached

      GL                                        Data
                    Content                     Warehouse
     Sales        Management

     Prod          Near Line


                     2005 DAMA Presentation                               7
Reasons For Data Growth
  •   Everyone Wants a Data Warehouse
  •   Everyone Wants Content Management
  •   Nobody wants to determine the Cost and Value
  •     of the data they manage.
  •   “Storage is cheap.”
  •   “Regulations require me to.”
  •   “Don‟t know what it is, but someone might need it”.
  •   Failure to develop “Enterprise” data management
  •   Bloated applications.
  •   Just plain sloppy development.

                    2005 DAMA Presentation                  8
The Mountains of Regulations

           2005 DAMA Presentation   9
Compliance and Regulations

   • “Regulatory compliance initiatives share a
     requirement for businesses to collect, securely store,
     analyze, and share critical business information, and
     to have business processes in place to act on that
     information. Regulatory compliance represents the
     most difficult challenge because it requires
     information transparency and management across
     functional silos.”

   IDC Insight, December 2003

                    2005 DAMA Presentation                    10
Some Fundamental Changes
  • A Business‟ relationship with the information it manages has
    changed from one of ownership to one of stewardship.
    Businesses no longer have absolute control over the
    information they manage.

  • Regulations enforce this new relationship and are meant to
    protect the public at large, customers, patients and

  • The definition of the business / information relationship is
    ongoing and independent of politics, industry and region
    with no real end insight.

  • Virtually all litigation now involves electronic data in the
    discovery phase.

                        2005 DAMA Presentation                     11
The Regulations

     • Currently there are over 10,000 statutes in the
       US covering Data Retention Regulations for
       Federal, State and Self Regulated Agencies in
       the US alone. 3

     • Most Regulations are comprised of:
        –   A Stated Rule
        –   A Retention Period
        –   A Retention Media Requirement
        –   A Violation Consequence

                  2005 DAMA Presentation                 12
Regulations with Retention Rules
   • SEC Rule 17a-4 Electronic Storage of Broker Dealer
   • Graham-Leach-Bliley Act - Financial Services
     Modernization Act -1999
   • Sarbanes – Oxley Act of 2002
   • FDA 21 CFR Part 11
   • DOD 5015.2 Department of Defense
   • Health Insurance Portability and Accountability Act (HIPAA)
   • Fair Labor Standards Act
   • Occupations Safety and Health Administration (OSHA) Act
   • Internal Revenue Service Reform Act
   • Food and Drug Administration
   • Health and Human Services

                     2005 DAMA Presentation                        13
Regulations with Retention Rules
  • Federal Employee's Compensation Act
  • Employers Retirement Security Act
  • Welfare and Pension Plans Disclosure Act
  • Civil Rights Act and Equal Pay Act
  • Patriot Act
  • Basel II Capitol Accord
  • BaFin – Germany
  • MoReq CRFB - France
  • Additional Regulations are applicable in Canada, the
    EU and Asia.
  • What parts of the business aren’t covered?

                     2005 DAMA Presentation                14
SEC 17a
  SEC Rule 17a-4 Electronic Storage of Broker Dealer Records

  • Retention – Minimum of 3 Years

  •   Related to the retention of correspondence between the securities
      company and its customers.
           • Purchase and sale documents,
           • Customer and associated persons‟ records,
           • Customer complaint records
           • Written supervisory procedures

  •   Additional rules have been established by both the NASD (sect 2210 and
      3010) and NYSE(SECT 342 ) that require members to comply with SEC
      17a-4 or risk fines by both the SEC and the members SRO.

                         2005 DAMA Presentation                                15
SEC 17a
    •   "preserve the records exclusively in a non-rewriteable, non-
        erasable format.“ This requirement does not mean that the
        records must be preserved indefinitely. Like paper and
        microfilm, electronic records need only be maintained for the
        relevant retention period specified in the rule.

    •   The electronic storage media must verify automatically the
        quality and accuracy of the storage media recording
        process; serialize the original and, if applicable, duplicate units
        of storage media, and time-date for the required period of
        retention the information placed on such electronic storage
        media; and have the capacity to readily download indexes and
        records preserved on the electronic storage media to any
        medium acceptable under paragraph (f) as required by the
        Commission or the self-regulatory organizations of which the
        member, broker, or dealer is a member.

                     2005 DAMA Presentation                                   16
Grahm - Leach

   Graham-Leach-Bliley Act or Financial Services
       Modernization Act of 1999

   •   Retention Period – 6 Years or “Best Practices”

   •   Related to limited privacy protection against the sale of
       private financial information to third parties.
         •    Personal financial information must be securely retained.
         •    Customers must be advised of the policies in place for sharing
              personal financial data.
         •    Customers must be able to easily opt out of the sharing of some
              financial data

                       2005 DAMA Presentation                                   17
  Name - Health Insurance Portability and Accountability Act

  •   Retention Periods
       – Complaints – 6 Years
       – Medical and Diagnostic Records – 6 Years
       – Medicare Records – 5 Years
       – Special Consideration for Minors
       – Records must be retained for 2 years after a patients death

  •   Relates to documents on uses and disclosures, authorization forms,
      business partner contracts, notices of your information practice,
      responses to a patient who wants to amend or correct their
      information, the patient's statement of disagreement, and a complaint

                         2005 DAMA Presentation                               18
Sarbanes – Oxley - SOX
   The Sarbanes-Oxley Act of 2002

   •   Retention Period – 7 Years

   •   Deals with the falsification, destruction, alteration of documents or data with
       the intent to impede, obstruct or mislead an investigation by any federal
       agency. Includes the destruction of materials used in the creation of audits
       or financial assessments
   •   Applies directly to publicly held companies
   •   US Companies valued at over 100 million dollars will spend a combined 2
       Billion dollars on implementing SOX 4
   •   Privately held companies with US ties are adopting SOX as well.

                           2005 DAMA Presentation                                        19
Sarbanes Oxley
    “ And today I sign the most far reaching reforms of American
      business practices since the time of Franklin Roosevelt.
      This new law sends a very clear messages that all
      concerned must heed. This law says to every dishonest
      corporate leader: You will be exposed and punished; the
      era of low standards and false profits is over; no boardroom
      in America is above the law. ” George Bush – July 30, 2002

    Since 2002 the DOJ has had over 250 successful

                    2005 DAMA Presentation                           20
The 3 rules of Sarbanes Oxley
   •   The first rule deals with destruction, alteration, or falsification of
       records. - Sec. 802(a) "Whoever knowingly alters, destroys,
       mutilates, conceals, covers up, falsifies, or makes a false entry in any
       record, document, or tangible object with the intent to impede,
       obstruct, or influence the investigation or proper administration of any
       matter within the jurisdiction of any department or agency of the
       United States or any case filed under title 11, or in relation to or
       contemplation of any such matter or case, shall be fined under this
       title, imprisoned not more than 20 years, or both."

   •   The second rule defines the retention period for records storage.
       Best practices indicate that corporations securely store all business
       records using the same guidelines set for public accountants. - Sec.
       802(a)(1) "Any accountant who conducts an audit of an issuer of
       securities to which section 10A(a) of the Securities Exchange Act of
       1934 (15 U.S.C 78j-1(a)) applies, shall maintain all audit or review
       workpapers for a period of 5 years from the end of the fiscal period in
       which the audit or review was concluded."

                           2005 DAMA Presentation                                 21
Sarbanes Oxley
  • This third rule refers to the type of business records that need
    to be stored, including all business records and
    communications, including electronic communications.

  • Sec. 802(a)(2) "The Securities and Exchange Commission shall
    promulgate, within 180 days, such rules and regulations, as are
    reasonably necessary, relating to the retention of relevant
    records such as work papers, documents that form the basis of
    an audit or review, memoranda, correspondence,
    communications, other documents, and records (including
    electronic records) which are created, sent, or received in
    connection with an audit or review and contain conclusions,
    opinions, analyses, or financial data relating to such an audit or

                       2005 DAMA Presentation                            22
New SOX “Data” sources
  • Website Records - Section 403 - Posting stock ownership

  • Internal Control Reports – Section 404 - Audit notes on how the
    internal control reports are created

  • Corporate Officer Certification – Section 302 – Who certified
    which reports and audits and when.

  • Complaints – Section 301 – The collection, retention and
    treatment of complaints, external, internal, anonymous as they
    relate to financial audit and disclosure. Also, a description of
    how the complaint was addressed.

  • Penalties – Section 906 – False certification can result in
    $5,000,000 in penalties and/or 20 years in prison.

                       2005 DAMA Presentation                          23
The Next Wave – Basel II

   • The Basel Capital Accord, or Basel II, updates 1988
     European capital rules for risk-management
     practices that align capital with operational, credit,
     and market risks for banks operating internationally.
   • Basel II also mandates new regulatory methods for
     calculating capital to support operational risk.
   • Banks must determine operational risk in its methods
     of calculating the amount of capital an institution
     must set aside to cover risk.
   • Banks must retain data on how risk was determined.

                    2005 DAMA Presentation                    24
Basel II Pillars of Compliance

       First Pillar - Minimum Capital Requirements
       Standardized calculation and norms to determine “haircut”
       between collateral and exposure

       Second Pillar - Supervisory Review Process
       Principle 1: Overall Capital Adequacy in Relation to Risk
       Principle 2: Supervisors Must Monitor and Take Action
       Principle 3: Banks Must Operate Above Minimum Capital Ratios
       Principle 4: Supervisors Must Intervene and Take
                    Remedial Action

       The Third Pillar - Market Discipline
       Quantitative and Qualitative Disclosure of Credit Risk

                     2005 DAMA Presentation                           25
Basel II Impact

   Guillermo Kopp, of TowerGroup, on 2005 trends in
     financial services:

   “ [T]he Basel II Accord introduces the concept of
     operational risk across the company, and
     fragmented silos can't offer an enterprise view of
     operational risk. As a result, companies are starting
     to think more horizontally across the hierarchy,
     looking at ways of combining and integrating their
     systems for each compliance mandate. “ 6

                    2005 DAMA Presentation                   26
What Do The Regulations Want?

  • Proof of Compliance through the adherence to Policy
  • Retain data / information covered by the regulation in
    a fashion that demonstrates it is in a pristine and
    unaltered condition.
  • Regulations REQUIRE retention of
     – Relational Data
     – Content Managed Data such as scanned forms and media
     – E-Mail
  • Once you touch it, it‟s yours to keep for the entire
    retention period and longer.

                    2005 DAMA Presentation                    27
Swimming With the Sharks…

          2005 DAMA Presentation   28
   • Forensic Data Analysis
     – Do you know where the data is?
     – Can you get to it?
     – Is it related through multiple systems
   • Information Discovery Process
     – Outsiders view data as a “Corporate Asset”
   • Halt Destruct Orders
     – Similar in form and function to Regulatory
       retention rules except they will (hopefully)

                   2005 DAMA Presentation             29
Andersen Consulting

   “It might be useful to consider reminding the
       engagement team of our documentation and
       retention policy. It will be helpful to make sure that
       we have complied with policy. Let me know if you
       have any questions.”
   E-mail fro an Andersen Attorney in October 2001

   In June 2002 Andersen was found guilty of obstruction
      of justice.

                      2005 DAMA Presentation                    30
Crime and Punishment…
   •   In response to the WorldCom bankruptcy filing, the Securities and
       Exchange Commission (SEC) takes swift and dramatic action to
       deal with what was perceived as a wholly inadequate records
       management program and imposes an $800-an-hour monitor on
       WorldCom (now MCI). The monitor’s task is to ensure that the
       company “has developed document retention policies and ... has
       complied with these policies.

   •   The SEC fines five brokerage firms $8.25 million for failure to
       retain e-mail records. In addition to the monetary penalty, the firms
       are required to “review their procedures to ensure compliance with
       recordkeeping statutes and rules.”

   •   The CEO of a pharmaceutical company is found guilty, sentenced
       to seven years in jail, and forced to pay a $3 million penalty for
       obstruction of justice because he “directed another individual to ...
       delete certain computer files ... Containing phone messages he
       received ... and documents evidencing [his] instructions.”

                         2005 DAMA Presentation                                31
Crime and Punishment
  •   Procter & Gamble: P&G was sanctioned $10,000 for failure to
      preserve corporate e-mail communications despite its knowledge that
      the e-mail would be relevant to an action.

  •   Applied Telematics Inc. v. Sprint Communications Co.: Sprint’s failure
      to preserve electronically-stored routing plans resulted in an order for
      payment of plaintiffs’ costs and attorney fees.

  •   Prudential Ins. Co. of America Sales Practice Litigation: Prudential’s
      “haphazard and uncoordinated” approach to document retention
      denied its opponents potential evidence to establish facts in dispute,
      and was grounds for a $1 million fine.

  •   Fen/phen wrongful death class action suit Experts estimated that the
      cost to restore emails from backup tapes to satisfy the discovery
      process could go as high as $1.75 million. Facing a hostile court,
      defendant Wyeth settled for over $3 billion dollars.

                          2005 DAMA Presentation                                 32
Retention Caveats

   • Organizations need to consider the statute of
     limitation or time period for suing, in determining
     their retention policy.
   • Many organizations will retain the records of minors
     for a longer periods of time. Best practice is until
     the minor involved is at least 21 years of age.
   • Organizations select longer retention periods
     because of the concern of having records available
     for defense purposes for litigation.
   • There is a balance between keeping data to defend
     yourself and getting rid of data that can be used
     against you.

                    2005 DAMA Presentation                  33
The Data Retention Road Map

           2005 DAMA Presentation   34
The Conflict


                                                             Regulatory Requirements

   Data Management

Improved Data Practices

                                                                Litigation Exposure

                          2005 DAMA Presentation                                       35
What are organizations doing? 5
   • 44 percent of organization ensure the integrity of
     their electronic records

   • 50 percent automatically delete e-mail messages
     on a periodic basis

   • 39 percent address electronic records in their
     retention policies

   • 43 percent train e-mail users about record

   • 28 percent monitor compliance with their
     electronic records-retention policies
                   2005 DAMA Presentation                 36
Four Legs of Compliance
   Compliance is the result of integrated Policy and Process

   The Policy - Information Records Management Policy is established by corporate
      Legal. Specific measure for compliance are tied to the policy. What’s the policy
      and how do you measure compliance?

   The Leadership – The Policy is reflected in the visibility, adoption, enforcement
      and compensation by and for senior management. Does Leadership walk the

   The Technology – The Policy is reflected in all aspects of data management. IT is
      using and NOT establishing The Policy. Does the Procedure tie to the policy?

   The Training – The Policy reflected in all aspects of training, education, procedure
      and compensation. Does everyone understand their responsibility, liability and

                             2005 DAMA Presentation                                       37
The Compliance Team

  • The Compliance Team Provides an enterprise
    understanding of data retention through
     – Comprehensive understanding of corporate policy and
       procedures related to regulatory compliance.
     – Elimination a fragmented responses to regulatory inquiry
     – Optimizes response to Litigation Discovery

                     2005 DAMA Presentation                       38
The Information Compliance Team
   •   General Counsel – Identifies current and future regulatory requirements
       and potential impact on various business units.

   •   Compliance Officer – Interprets regulatory requirements for all
       information assets within the organization. Identifies operational
       deviation from company Data Management Policies. Ensures consistent
       application of Policy to Process.

   •   Information Architect – Understands information assets by structure and
       function from the metadata content to full subject area. Ensures that
       regulatory components are built into data.

   •   Application Architect – Ensures that applications incorporate retention
       and audit capabilities in compliance with stated policy.

   •    Content and Messaging Manager – Understands Content Management
       (unstructured) Data and e-mail service resources. Ensures compliance
       for unstructured digital information assets.

   •   Training Supervisor – Supports compliance related education services
       for all individuals who can access data under retention regulatory

                           2005 DAMA Presentation                                39
Data Under Management
    Understand ALL the Data Sources
        Covered By Regulations

              Archived       Paper Copy

                                          Content Managed
    Off Site Backup      Relational


                      2005 DAMA Presentation                40
Data Under Management

       Data Under Management

              Archived       Paper Copy

                                          Content Managed
    Off Site Backup      Relational


                      2005 DAMA Presentation                41
Integration of DR Rules

   • Shared Data must be coordinated
   • Structured, Unstructured (ECM) and Email must be
     related from a retention standpoint with common
                 Primary Key                     Common Rules
                                                For Retention and
              Content Managed                     Management
                Document ID

         Email, Paper and Other –
        Email Address, Folder ID, Etc.

                       2005 DAMA Presentation                       42
The Roadmap

  • Establish Policy
     – Relational Data
     – Unstructured Data
     – E-Mail, Training Materials etc.
  • Identify data value for all data under management
     – Relational by subject area
     – Content Managed as Related to Relational
     – Email, Backup and Offsite
  • Dispose of non-regulated low value / low access
    data with an appropriate audit trail.
  • Develop process to periodically dispose of disposal
    eligible data with audit and reporting systems.

                      2005 DAMA Presentation              43
Risk Mitigation
   • Less Data = Less Exposure = Less Risk
   • Understand the Regulations and keep what is
   • Develop Policies and keep them current.
   • Develop Compliance Training, Measures and
   • Improve Data Management.
   • Implement Auditable Processes

   “ Luck is the residue of design… “

                                  Branch Rickey
                    2005 DAMA Presentation         44
New Regulation Process
   1.    New and modified regulations reviewed and interpreted by
         legal department designee.
   2.    Regulation requirements reviewed with Chief Compliance
         Officer (CCO). If warranted, Implementation team assigned.
   3.    Business Analyst reconciles new requirements with existing
   4.    BA reviews Data Requirements with Data Architect
   5.    DA identifies impacts to data and data lifecycles.
   6.    New rules developed by BA and validated with Legal
   7.    New rules reviewed with DA and implemented
   8.    New rules test cases run and approved by legal and CCO
   9.    New rules applied to current data set (production and
   10.   New rules in production

                       2005 DAMA Presentation                         45
The Data Retention

       2005 DAMA Presentation   46
The Data Lifecycle
   •   The Data Lifecycle is the collection of standard events applied
       to a domain of information.
   •   The Lifecycle is comprised as follows:
       Mutable events can modify, refresh and change the content
           and context of the information as in the following Lifecycle
       –   Create – The derivation or creation of the record
       –   Update – Modification of the record
       –   Distribute – Share data with other services
       Immutable events allow the data to remain observable but
          with fixed and unchanging content and context.
       –   Archive – physical or logical repositioning of the record with
           immutable content
       –   Dispose – Physical removal and destruction of the data record

                         2005 DAMA Presentation                             47
A Data Lifecycle

                          Data Lifecycle

                  Data Workers                  IT Management

         Create     Update       Distribute   Archive    Dispose

                          Audit and Reporting

                   2005 DAMA Presentation                          48
Data Value and Lifecycle

            < 20%

           Online                               Nearline / Offline



                                 2005 DAMA Presentation                       49
Data Lifecycle Caveats

   • All data has a lifecycle composed of events triggered
     by rules.
   • These events and rules can be captured as part of
     the administrative or operational metadata
     associated with an entity.
   • Additional Lifecycle events may be required to
     support business, governance or regulatory
   • The Events become the metrics for the audit.
   • The rules behind the events may change requiring
     version management.
   • Halt destruct orders need to be accommodated as
                    2005 DAMA Presentation                   50
Draining the Swamp

    Disposing of Data, Mitigating Risk and
      Achieving Compliance

                2005 DAMA Presentation       51
Data Reduction
   • Get rid of data eligible for disposal
      – Relational and Unstructured
   • Identify and keep “Value” data. What purpose does
     this information serve?
      –   Required for Analytics
      –   Business Continuity
      –   Business Operations
      –   Regulatory Controlled
   • Archive
      – Low value
      – Low frequency accessed
   • Keep aggregate, dispose of atomic
   • De-Dupe Data
      – One source of data = “Enterprise” view.
      – Authoritative content
      – Eliminate Replication
                        2005 DAMA Presentation           52

   Business Case benefits are primarily related to
     Legal Exposure Risk Avoidance.

   •   Storage Recovery is minimal
   •   Improved performance
   •   Improved Disaster Recovery Time
   •   Less complex data integration
   •   Lower Development Costs
   •   Reduced Discovery Costs
   •   Paper File Retention and Storage

                    2005 DAMA Presentation           53
Data Reduction

                No DR Disposal

                  Remove Delete Eligible Transactions
14m     16.5M

10m                De-Dupe Operational Data
                                DR with Disposal
6m                7.3M

 2003                    2005                           2007

                      2005 DAMA Presentation                   54
Data Audit
                Used in last Year
                   500,000                                       Retention
                                            Disposal             Required
                                            Eligible             3.6million

   Unused Over
      1 Year
    (6.5 million)
                                    (8 million)

                            2005 DAMA Presentation                            55
Data Audit

   Active in Last Year                                  Online
        500,000                         Retained by     500,000
       Inactive for         1.9m           Policy
      Over 1 Year        (Replicated)    4.6 million
       (6.5 million)

       Offline                            Eligible
       Archive              8.0M
                                        For Disposal
     Inactive for                                      Nearline
                                         9.9 million   4.1 million
     Over 2 Years
       (8 million)

                          Numbers in Transactions

                             2005 DAMA Presentation                  56
Audit Data Under Management
                    Transactions Under Management by application as of 7-1-2004

                                                                        CDE XTN in ABC
                                               CDE XTN 3,100,000 16%
                                                                         3,100,000 16%
              ABC XTN 4,400,000 23%

                                                                             CDE/ABC XTN in STU
                                                                           (STU Mirror) 400,000 2%
                                               Replicated Data
             STU XTN 56,000 <1%                 4,122,000 21%
                                                                                   CDE XTN in STU (STU
                                                                                  Integrated) 42,000 <1%

                Archived ABC XTN
                                                                                     CDE XTN in DEF 570,000
                  4,000,000 20%
                                             Archived STU XTN            XTN in Panther 10,000
                                               3,900,000 20%

   •   Out of 18 Million Transactions under management less than
       600K were required to do the business
   •   Of the remaining 17.4 Million, 10 Million had exceeded data
       retention thresholds and were eligible for disposal.

                               2005 DAMA Presentation                                                         57
Wal-Mart Retention

   In a recent interview Linda Dillman, CIO of Wal-Mart
      stated that:
      – 690 million line item transactions are saved daily
      – Every customer transaction is saved
      – The information is used to identify product velocity, mix and
      – The transaction data retention period for a fixed 2 years.
      – Data is made available to suppliers real time. “Right product
        to the right store at the right time…” Assed value in the
        ability to monitor customer, product, department, store and
      – Wal-mart keeps daily weather statistics for each store.

                      2005 DAMA Presentation                            58
E-Mail Retention

   • Companies Identify Email covered by regulation and
     Do Not Destroy Orders based on job function, source
     and title.
   • Retention eligible are then saved separately
   • Other non-regulated e-mail are disposed of after 90
   • Retaining email copies at the desktop is discouraged
     or prevented.
   • Recent fines have ranged from 1.7 to 10 Million

                   2005 DAMA Presentation                   59
A Data Retention

        2005 DAMA Presentation   60
DR Framework Requirements
  • Retain and delete transaction data according to the data
    retention policy
      – Maintain transactions for the period noted in the data retention
        policy statement
      – After the transaction maintenance period has elapsed (generally 6
        years) the transaction will be deleted
  • Comply with court orders to halt destruction of data meeting
    given sets of criteria
      – Maintain the ability to apply the stated retention policy while also
        making exceptions for „halt destruction orders‟ requiring the
        cessation of deletion for electronic transaction files meeting a
        given set of criteria
      – Be able to immediately apply halt destruction orders to the delete
  •   Maintain audit records of all transactions that are deleted
      – For transactions that have been deleted maintain the primary key,
        date of close, date of destruction, and the version of the rule set
        under which the transaction was deleted

                          2005 DAMA Presentation                               61
DR Framework Requirements

  • Provide a report or message service that can be utilized
    for managing the destruction of no relational data such as
    Content Management supported information, E-Mail and
    paper files. Be able to provide a report of the destruction
    of transaction files so that paper files may be destroyed
    along with their paper and unstructured counterparts.
  • Provide a one stop shop to determine the disposition of
    any data from a data retention stand point.

                     2005 DAMA Presentation                       62
DR Frame Work Components

  • Source systems – Data where Data Retention status will
  • Rules Engine – Rules engine used in determining current
    data retention status for any applied data.
  • Audit Repository – Keeps track of key life data lifecycle
  • Reporting Services – Provide on demand and scheduled
    reporting and notification services related to DR subject areas
    or individual rows / records.
  • Archive – Records repositioned from the primary storage to
    Secondary Nearline and Offline storage.
  • Disposal services – Service to eliminate disposal eligible
    information based on retention policy.

                      2005 DAMA Presentation                          63
Data Retention Landscape
                           Rules Engine           Audit And
                                                  Reporting                Content
                                                                       Disposal Process
              Other               Retention
                                  Rules          Retention

                             Retention         Reporting      Management
                                               Messaging           Email

    Archive                                            Messaging
    Tapes                                                            Paper Disposal
                                                             Paper      Process

                      2005 DAMA Presentation                                          64
Data Retention Rules Engine
   A Retention Rules Engine
       – One Stop Shop for determining
          the Retention Status of any data
          based on a set of business rules.
       – New rules can be added to cover
          additional data subject areas as
       – Sends messages to other
          systems such as paper data
          archive, content management,
          third parties to notify those
          systems of the change in the
          retention status of specific data.
       – Audit component to track the
          Lifecycle of the data.

                          2005 DAMA Presentation   65
Data Retention Infrastructure

       Applications using          Compliance and               Applications using
       Unstructured Data           Litigation Audits             Structured Data

                                 Retention Rules
      Content Manager                                  Relational Data Manager
                               Retention Manager

                        Storage Manager for Data Retention

                                 Storage Devices
                        Disk, Tape, WORM Tape, Optical, DVD

                                   Disposal Manager

                            2005 DAMA Presentation                                   66
Data Retention Infrastructure

   • Implemented as an Enterprise Service
   • Use common features
      – Rules Engine
      – DR Audit Services
      – DR Status Reporting Services
   • Rules retained in metadata under governance meta
      –   Rules
      –   Regulatory documents
      –   Impacted roles
      –   Data Sources, Archives, Backups etc.
   • Integrate with Change Management

                       2005 DAMA Presentation           67
DR Landscape Features
   Operational                                               Analytics
   App and Web                           Data                  and
                 App and Web          Warehousing
     Services                                                Reporting
     HR           Attached

     GL             SAN                    Data
                   Content                Warehouse
    Sales        Management

                  Near Line
                   Storage               Backup

                                     Engine            Services

                    2005 DAMA Presentation                               68
Technologies to Support DR

   • Rules Engines
      – EII, ETL, BPM
   • Audit and Reporting
      – Standard Reporting process
   • WORM Storage
      – long term archive – Non rewritable media backup.
      – WORM Tape Cartridge – Password Protected
   • Content Management
      – EMC Documentum
      – IBM Records Manager
   • Email Archive
      – Ziplip

                      2005 DAMA Presentation               69
Vendor Solutions

   • Solution Types
      – Enterprise Information Integration (EII)
          • Composite, Avakia
      – Hardware and Software Blend
          • IBM, HDS, StorageTek
      – Software only or Software partnered
          • Outerbay, Princeton Softech
      – “Build” solution based on ETL Tools
          • Informatica, Data Stage Etc.
      – Business Process Management Tools
          • Staffware

                        2005 DAMA Presentation     70
EII Tool

   Composite Software
   Purpose: All in one tool to
     rapidly develop views and
     queries across multiple data
   • Used to develop statistics
     about data under
   • Ability to link to multiple data
     sources to
   • Dashboard presentations
   • Support for scripting
     language, XML and SQL
   • Integrated Scheduling tool

                         2005 DAMA Presentation   71
EII Tools

   • Insitu data investigation
   • Cross database investigation
   • Rapid development of “Views”
   • Federated Data Management allows views of
     information assets across the enterprise.
   • Uncouples information from the application.
   • Levels hub and spoke data sources to a single view.
   • Great “Swiss Army Knife” tool
       – ETL, Query & Select, Dashboards

                      2005 DAMA Presentation               72
Vendors – HW/SW

        IBM – DR-450
    •     Rules Engine based
    •     Records Management oriented.
    •     Supports unchanging data.
    •     Requires integration with applications.
    •     Provided with P-615 for Near line Storage
    •     Focus on Content / Record Management
    •     Integrates with Tivoli
    •     Acquired from Tarian Software in 2002

                       2005 DAMA Presentation         73
Vendors HW/SW

  • StorageTek Lifecycle Director
     – Records Management Oriented but can support row level
       data movement.
     – SQL Generator for Rules Management
     – No Rules Versioning or Hierarchy
     – No nesting of the rules
     – Integrates with other StorageTek storage Management
     – Automates migration of older data to inexpensive storage.
     – Improves performance of primary storage.
     – Frees up more expensive online disk space.
     – Provides access to all archived data.

                     2005 DAMA Presentation                        74
Vendors HW/SW

  EMC Documentum

    –   Records Management
    –   Federated Search
    –   Relevancy Ranking
    –   Keyword Search
    –   Workflow Definition
    –   Integration with EMC HW
    –   Ability to “Freeze” data for
        litigation hold.

                         2005 DAMA Presentation   75
Vendors - Build

   • Information Builders
      – iWay Adapter Manager, iWay Data Migrator, WebFocus
      – Ability to connect to a wide range of data sources
        JDBC, COM, ODBC, J2EE etc.
      – Use of Data Migrator as the Rules Engine
      – Use of WebFocus for analytics
      – Combination of products to meet requirements.
   • Informatica
      – PowerCenter, SuperGlue, Power Analyzer, Power
      – Use for doing initial heavy lifting
      – Combination of products to meet requirements

                    2005 DAMA Presentation                   76
E-Mail Retention Management

  • Compliance Management
    – Index, Store, Retrieve
    – Tamper-Proofing
    – Monitor & Control
  • Email Content Management
    – End-User Search & Restore
    – With Security, Policies, Privileges
  • Storage/Performance Mgmt
    – Offloading Large Email/Attachments
    – Boosting Email Server Performance

                    2005 DAMA Presentation   77
Wrap Up
  • Develop Policy statements that have specific actions and
    measures associated with them.
  • Reduce your total volume first by eliminating content that has
    expired policy retention limits.
  • Incorporate retention requirements in your metadata under
  • When implementing, ensure you have:
      –   Rules Engine
      –   Audit Services
      –   Reporting Services
      –   Disposal Services
  • Ensure you a incorporating all data sources
      – Relational
      – Unstructured / Content Managed
      – Email, Paper etc.

                        2005 DAMA Presentation                       78

    1.  Peter Lyman and Hal Varain – How Much Data – 2000
    2.  IDC – Worldwide Email Usage Report – 2002 to 2006 – Gretel Jonston – October
    3. CFO – Drowning in Data – Nov.4 2003
    4. Carol Hildebrand - Profit, November 2004 “Basel II Opens the Door to Opportunity
        Coxx Media
    5. 21st Century Complete Guide to the SEC by U.S. Government Printing Office
    6. Securities Regulation: Examples and Explanations by Alan R. Palmiter
    9. The IBM Archives

                          2005 DAMA Presentation                                          79
Some Light Reading

  • Randolph Kahn - Information Nation: Seven Keys to
    Information Management Compliance
  • Jesus Mena – Homeland Security: Techniques and
  • John Vacca – Computer Forensics

                  2005 DAMA Presentation                80


            2005 DAMA Presentation   81
Contact Information
   Apex Solutions
     Easy? No
     Worth it? Yes.

   John Murphy – 303-670-8401

   Suzanne Riddell – 303-216-9491

                      2005 DAMA Presentation   82