Data Governance by pengxuebo

VIEWS: 27 PAGES: 160

									Data Governance
A common thread runs through in a vast
  number of business problems that most
  problem solvers cannot see
Data that is not designed to be
Data design is the heart and soul of
how IT enables government to fulfill
            its mission
What is badly designed data?
     One type of bad design is
information that is locked into data

    Data on one island cannot talk to data on another island
Another term to describe locked data is
 “siloed” data or “stovepiped” data
Example of how data models
 affect business processes
A programmer is hired to design a system
  for a new library
 For convenience of illustration, let's say that
patrons can be uniquely identified by first name

•Redundant address
•Change of address difficulty
•Redundant book author problems
         Data Problems

This example shows exactly how business
processes are affected by poor data
 Bad data design forces business
side to work harder than necessary
   and without good information
New layers of inefficient business
processes form to adapt to the poor
data design
 For example, a business process could exist
where librarians have to hire an army of data entry
staff to do error checking to make sure patron
names are correctly spelled
Bad data design forces creation of layers
      of bad business processes
Twenty-year problem lock-in
Data design plays a large part in controlling
  programming and business processes
Enter Data Governance
Data governance cleanly resolves
 inefficient business processes

           PATRON FILE
Book file only contains info about
              BOOK FILE
Finally, we have a concept that handles the
transaction of a patron checking out a book

computer screen display of what
          clients see
• Only have to type in the data once
• Address changes handled cleanly. Old
  address is updated for all current books
  checked out
• Automatic error checking on patron or
  book name
Data governance improves service to the
business side
Minor changes in data design
 translate into huge business
improvements throughout the
Client satisfaction is increased

A centralized government database from
the Library of Congress of all books can
be connected to so that library staff
doesn’t have to type in book information
such as author name, title, etc.
The business side has benefited
from even more savings in costs
           and work
The data islands are now connected

  Imagine the work savings from not having to type in new book information
  such as title and author name
This is how data governance
     brings cost savings
 Additional improvements are in
time, quality, and organizational
These are not one-time savings, but ongoing
     savings that leverage each other

• Converting to the centralized national book database
  was easier because the data was already modeled
• They just replaced their book file with the national
  centralized one.
• The error checking and streamlined book check-out
  process were already in place.
Modeling data guarantees
    business agility
Data design is the heart and soul of
how IT enables government to fulfill
its mission
The library system is an example of how tightly data design is coupled
to business processes
 Where is bad data design
reflected in the real world?
Business clients, e.g., the librarians, can’t visualize that
      data design caused the various problems
  Clients see the problem camouflaged as a lack of
functionality in many areas their business processes
Dept. of Veterans Affairs
• How did it happen?
• What was missing?
 Answer: a comprehensive way looking at
  the interconnection of problems and
  prioritizing them in coordination with the
  enterprise data vision
 Do you see the correlation to the
  centralized remote book file for the library?

FBI Director Robert S. Mueller, III
          FBI had competent vendors
    and project managers install their system
•   On time
•   On budget
•   On scope.
•   Good data modelers
                         QUIZ: What was
Answer: a comprehensive way looking at the
interconnection of problems and prioritizing them in
coordination with the enterprise data vision
• Hidden requirements were not identified.
• No organizational structure existed to ensure enterprise-wide
• A process that continually takes the enterprise-wide view of how to
  implement change. They just added systems to the rest like a
  manager of a fleet of cars adds a new car. They didn’t check for

The problem was repeated over and over
 again for each new FBI project.

Data governance was missing.
New way of looking at the
1. Connections
2. Potential connections not yet implemented
Using this new method to view an organization,
how pervasive is this problem?

• The problem is huge and expensive. E.g.,
  Fi$cal, 21st Century projects were
  designed to handle lack of comprehensive
• When someone leaves their organization
• We’re basically talking about everything.
A interpreter is needed that identifies
business problems stemming from bad data
design and then designs the data models
that fix the problems at an enterprise-wide
         Data governance
Data governance is the practice of
 organizing and implementing policies,
 procedures and standards that maximizes
 data access and interoperability for the
 business mission
Why is data governance needed?
• Business needs drive data governance
  This is what interoperability
looks like both to business side
         and the IT side
Each link represents a business
value where data can reach the
 business people that need it
But beyond the requirements that are represented
in the links, many more business requirements are
invisible to all except the enterprise data modeler
Requirements gathers cannot find them
What integration opportunities look like to an
          enterprise data modeler
Currently, much data is stovepiped
Many connections are missing
 Each potential link between data
that is not implemented represents
      a loss of business value

How do stovepiped systems get built?
    Forces that create data stovepipes
•  Contractors and vendors
•  Programmers
•  Managers
•  Budget structures
•  Security concerns
•  Methodologies: SDLC, PMBOK, WATERFALL,
• Change
In sum, an absence of data governance
Organizations mistakenly think
installing new applications are like
replacing one of the cars in their

It’s broken, so….replace it…

What is a better metaphor for representing a
 new software system than replacing a car?
A component of a
 Neural Network
It is important to add to the project
 team, a person that can check for
       enterprise interoperability
This is a key method regarding how
to look at new projects
The neural network will ensure that one hand will know
what the other hand is doing
A small sampling what types (part
of a much larger list) of
improvements the links represent
•   Eliminates manual operations, e.g., when redundant tables or fields are
•   Business side has the data it needs when it needs it
•   Organizational agility

Closer IT alignment to organization’s mission.
 Example of one of the links at the
California Dept. of Consumer Affairs
Calif. Secretary of State website
Corporation data interoperability
   was an invisible business
Because data interoperability problems are invisible,
business processes are unnecessarily limited throughout
the enterprise due to undiscovered data integration
What is data governance’s goal?
 Bring together the business side, with the
 people that can see the invisible business
 requirements in order to build the data
 structures that will give business the
 information it needs when it needs it
Why is the need to implement
 data governance urgent?
The design phase for projects is relatively
The opportunity to correct it at the right
stage consisted of only a tiny, fleeting time
So why is lack of data governance
 Project development is currently going on
 in many areas without the benefit of data
 governance, which could result in
 permanent stove-piping of data.

 Result: Many stovepiped systems.

How many years can bad data design
affect a system?
 Inefficient business process may
get locked in for 20 years or more
Twenty-year problem lock-in
  How does data governance
The data governance process acts as a
central planning center to coordinate data
design across organizations
 It builds the enterprise-wide blueprint
that guides all IT development towards
           data interoperability
         Simple process!!!
Data Governance Council reviews changes
 or new development against a checklist
The governance council reviews all
new software development to see if
it could be shared enterprise-wide
           New table or field checklist

•   Correct data modeling (3rd normal form), correct keys. (Data Architect, IT)
•   Cross agency data and process sharing opportunities, including SOA and Web
    Services (Data Architect, IT and business side) [Image: waiting clocks: text: “Clients
    don’t have to wait until they request data sharing. Comprehensive data sharing
    opportunities check is done at the outset.”
•   Business Intelligence – data mart/data warehouse opportunities. (Business side)
•   Metrics generation opportunities - Can this field or table create useful metrics or
    appear on a DCA dashboard? Customers could include boards, licensees, the public,
    finance, governor's office and the legislature (business side)
•   KPI - Key performance indicators opportunities. Are there opportunities to use the
    field/table to measure performance? (Data Architect, IT and business side)
•   Is data subject to legislative oversight or mandates? E.g., Health Insurance
    Portability and Accountability Act (HIPAA), California Database Breach Act (California
    SB 1386),, FIPS, HSPD-12. Create table
    of federal, state and departmental regulatory mandates or voluntary guidelines that
    reviewers check data against (Data Architect, business side)
                Checklist continued…
•   Are there opportunities from making this available to a broader audience? Customers that are not
    immediately evident could include boards, licensees, the public, enforcement, DOJ, finance,
    governor's office and the legislature (Data Architect, IT and business side)
•   Can this data be replaced by a better source of data elsewhere or replace other data? Can whole
    tables be eliminated by consolidation and sharing? (Data Architect, IT and business side)
•   Prioritize super connector fields and super connector tables. (Super connector fields are those
    that cross agency boundaries. Super connector tables are the most important tables that can be
    shared. These must be listed in a transparent, centralized database and reviewed for use
    whenever a new system is designed. Vendor created systems must be reviewed for table/data
    sharing and naming conventions.) If this is a new super-connecter, then it should be transparently
    registered to the repository so that other strategic planners can see it. (Data Architect, IT and
    business side)
•   Can it be used to validate data or does it need validation performed on it? (Data Architect, IT and
    business side)
•   Data harmonization problems or opportunities. (Data Architect, IT and business side)
•   Standards evaluation – Are there standards to be adhered to or created? Data standards,
    business standards, naming conventions, etc. For example, every state agency could have the
    same standard for this field: Corporation_Tax_ID_Number 40 characters – alphanumeric. Does
    NIEM. gov already have a standard name for this field? (Data Architect, IT and business side).
    Example, CAS Class field.
              Checklist continued…
•   Alignment to organizational mission. Strategic planning problems or
    opportunities. (Data Architect, IT and business side)
•   Enterprise Architecture planning. How does it align with to the “To Be” architecture?
•   Impact on other systems. Entered into Architect’s dependency database (what
    systems does it impact or is it impacted by) (Data Architect, IT and business side)
•   Metadata opportunities (business side)
•   Risk (Data Architect, IT and business side)
•   Security. Should it be encrypted? What controls should be applied (Data Architect,
    IT, business side and ISO)
•   Should client be given control of the data? Would a data steward be useful for this
    data? (Data Architect, IT and business side)
•   Backup considerations - how often. How does it get refreshed when there is a
    crash? When should it be purged? (IT and business side)
•   Can data quality be improved? Is data cleansing applicable? (IT and business side)
•   Quality management – are clients satisfied? Is quality management and continual
    process improvement built into this system? (Data Architect, IT and business side)
•   Automated duplicate detection (IT)
             Checklist continued…
•   Timeliness. Is there value to the organization if the data is refreshed sooner
    or by other ways? (IT and business side)
•   Is the data coming from the best sources (lineage; most reliable, timely)? (IT
    and business side)
•   Enterprise architect’s calendar scheduled for periodic data design review
    every two years
•   Priority on architect’s data design inventory. (Data Architect)
•   Optimization by combining multiple projects (past, future or ongoing
    projects). (Data Architect, business side, IT)
•   Review for entry into a table of future opportunities and linked to a calendar
    of related opportunities or future change events. For example, if a related
    component was scheduled for updating, that would trigger an automatic
    reminder to review opportunities for this component. (Data Architect)
•   Audit policy – should the field or table be have its edit or use history
    recorded (IT and business side)
•   Error management (IT and business side)
Generally, client requests for government
interoperability arrive inconsistently as clients
struggle to understand how to improve their
The above checklist ensures that a
whole series of government
improvement opportunities are
checked for at the precise
movement when it’s most important
Clients don’t have to wait until they request
data sharing. Comprehensive data sharing
and all other opportunities checks are made
at the outset.

What it would it look like 10 years from now
 if every government agency used this
 checklist for every new project?
Which one of the check list items was used
 to discover the SOS opportunity?

• Correct data modeling (3rd normal form), correct keys.
  (Data Architect, IT)
• Cross agency data and process sharing opportunities,
  including SOA and Web Services (Data Architect, IT and
  business side)
• Business Intelligence – data mart/data warehouse
  opportunities. (Business side)
To handle the complexity of data, a simplified
super-connector check list would assist the Data
Governance Council identify data sharing

  Tables and fields that have the most intradepartmental
  and statewide connectivity. Examples:
  –   Corporation Number
  –   License type and license number
  –   SSN
  –   Address (including apartment number)
  –   Criminal case number
  –   Civil case number
  –   Agency code
Data governance concept is simple
 Whenever there’s change we ask a
 Can this be shared in or outside of our
Data Governance flow chart
    Sample integration priority list
•    Licensing data model changed to make individual
     unique identifier (QUIZ: how was this identified?)
•    Remove status code constraints from programming
     code and move them to table-based system (Where
     did this idea come from? Data modeler)
•    Enforcement measurement fields (QUIZ: Where did
     this come from?)
 Without data governance, how do data
improvement opportunities traditionally
         become known to IT?
• Business client submits a ticket for a problem.
  This involves delay.
• IT manager or OCIO recognizes a pattern. This
  involves delay.
• Business managers recognize a pattern. E.g.,
  LUG and EUG (Board user groups) collectively
  discover a problem. This involves delay.
• Vendor products and recommendations. This
  involves delay.
All of the current methods involve
delays and do not provide a
consistent and continual process
for identifying and addressing
problems enterprise-wide
    How do new opportunities come to
     the Data Governance Council’s
•   Project conception
•   PMO
•   SDLC
•   FSR
•   SPR
•   PIER
•   RFP
•   Change Management Board
•   IT Governance
•   PIT (Process Improvement Team)
•   Strategic plan
•   Executive strategic discussions
•   BCP
•   ITPP - Information Technology Procurement Plan
•   PSP - Proposal Solicitation Package
•   Table and field creation process (DBA, programmer, etc.)
•   List of business-side data related requests
•   Informal business projects, such as potentially sharable spread sheet data
•   Programmers or business clients
Data Governance shortens the time
that it takes to determine business
    clients need a data change

 Otherwise, IT waits until business side submits a ticket
 for a specific problem. They don’t know they have a
 data modeling problem or don’t realize that their
 business process can change through data design.
A comprehensive array of
discovery points above speed up
identification of improvement
    At what discovery point was the SOS opportunity found?                       (Hint…)

•   Project conception
•   PMO
•   SDLC
•   FSR
•   SPR
•   PIER
•   RFP
•   Change Management Board
•   IT Governance
•   PIT (Process Improvement Team)
•   Strategic plan
•   Executive strategic discussions
•   BCP
•   ITPP - Information Technology Procurement Plan
•   PSP - Proposal Solicitation Package
•   Table and field creation process (DBA, programmer, etc.)
•   List of business-side data related requests
•   Informal business projects, such as potentially sharable spread sheet data
•   Programmers or business clients
          Was this too late?

When is the best time to discover data
  How can data governance be

Stage One: To handle urgent problems.

 A quick Stage One with a short time line is
 envisioned without lengthy discussions
 DBAs simply email Data Architect
  any planned schema updates
• Simple, cheap and effective.
• Reasoning: Project development is currently going on in
  many areas without the benefit of data governance,
  which could result in permanent stove-piping of DCA
• We want to catch any emergency problems in the bud.
              Stage Two
Volunteers from business and IT form an
 initial, first version of Data Governance
 Council. The Data Governance Council
 designs itself and processes are to be
 improved as we collectively gain more
Together, business and IT build the
      data integration vision

• E.g., data warehouse
• Enterprise connectivity
• Etc.
   What factors make data
   governance successful?

Data governance is between 80 and 95
percent communication.
 The most important factor in most
   successful data governance
   programs is communication

Clearly, data governance is not a typical IT
       How effective is data

• Very cost effective
• Vast scope of business processes
• Money savings example: Fi$cal
All areas of the organization are
Data Governance advances the
efficiency, cost savings and agility
for every service
•   PMO
•   Process Improvement
•   Ticket system
•   IT Governance.

    Data governance will help every project become
    more successful wherever it’s included
 What are the benefits of data
Each time there is a single integration improvement, it
removes roadblocks to the organization's mission. Data
silos become accessible, clients' problems are reduced,
maintenance problems are reduced and connectivity
opportunities open up across departments. This
incremental method is also the least expensive
    Benefits of data governance
•   Greater department-wide and statewide interoperability
•   Citizens receive better service from integrated government business processes.
    Data will be more accurate, complete, and timely. Working with government will be
    more convenient, for example, when citizens only need to go to a single government
    agency to update their address instead of multiple government agencies
•   Brings the business side into the IT improvement process
•   Better business side control over data, privacy and project development
•   Faster identification and implementation of business solutions. Data governance
    methodically discovers the gaps in how IT services business and shortens the time
    from problem discovery to solution. Data governance shows the business side how
    to find their voice in collaborative problem solving.
•   Improved business decisions due to accurate data from the recognized source of
    Benefits of data governance continued
•   Increased user business side trust in data stored within the organization's databases
•   Helps meet the enterprise’s business goals including adaptation to changing
    regulatory and other environments
•   Eliminates data duplication as a result of data governance process
•   More accurate, consistent, complete, accessible and up-to-date data
•   Fraud detection is facilitated because all data field names across the department and
    state are standardized
•   Placing all data related requests in one place allows patterns to be identified
•   Clear documentation of the lack of integration may provide business managers with
    better new project proposals
•   Ease of business process refinement due to standardization of data components
•   Opportunities for harmonizing and standardizing business terms because
    stakeholders are brought together in a collective review process. For example, if
    identical meaning terms were "cost allocation" and "distributed cost", stakeholders
    could agree to standardize on one of the terms and remove the other from business
    documents such as contracts and agreements
    Benefits of data governance continued

•   Business Intelligence. Data warehouse creation simplified through standardization of
    business data
•   Better programming code due to correctly organized data
•   Agility in responding to new opportunities
•   Stops business system decay. Keeps all systems tuned to organization’s mission
    and to each other so that no new system re-writes are ever necessary.
How has data governance
  worked in real life?
UMass Boston
    Transformational to business side
•   Projects were completed much faster
•   Project quality was much higher
•   Greater programmer collaboration
•   Greater business side collaboration

Productivity went through the roof
  Solutions in all areas of business
• Inventory
• Finance
• Project management
• Etc.
Many unexpected business
 benefits were revealed
    Unlocked data allowed more
opportunities for innovation and agility
Clients were extremely satisfied
Clients were extremely satisfied

  •   New business advantages

  •   Data was prescient

  •   Easier to get reports

  •   Reduced workload

  •   Reduced errors
Success factors:

(1) Continual contact with clients

(2) Modeling data to translate data
model into client solutions
          UMass Boston

Each time a new software system was
installed, it was completely and totally
integrated, not just partly, but with every
possible data connection fully
implemented. Every table and field was
examined for enterprise integration.
Continually tuning the organization
    UMass Boston

One of the first data-integrated
 organizations in the country
UMass Boston data governance

 Built process improvement into the DNA of
 the organization
UMass Boston model of continual tuning
for enterprise-wide integration
Where do we go from here?
(1) Can you name any software product that
    has a tuning process (when one part
    changes or has new components added
    or a different vendor adds something to
    it, all parts are evaluated)?

(2) What is the current process for tuning
    this organization’s data?
Personnel System
         Personnel System
Contenders in the project management

•   Teams from many state departments
•   State Personnel Board
•   Private Industry
•   Single individual
    Methodology comparisons
•   Teams from many state departments - main
    tool: waterfall - failed
•   State Personnel Board - main tool: waterfall
•   Private Industry – main tool: waterfall - failed
•   Single individual – main tool: (1) working daily
    with client to understand requirements (2)
    modifying data design daily to translate
    requirements into best data model
       Comparison of old school
        methodology and data
       governance performance
•   Time
•   Scope
•   Budget
•   Quality
•   Risk
•   Client satisfaction
(a) One data governance method developer
  completed the personnel system in two
(b) Waterfall - The only other contestant that
  finished the project was the centralized
  state department team that had six
  programmers working on it for twenty
(a) Data governance method developer
  completed a fully automated personnel
(b) The centralized state department team
  system was not fully automated.
(a) Data governance - $50,000 a year for
  one developer's salary for two years.
(b) Waterfall - difficult to estimate, but at
  least ten times as much.
(c) Private industry - $500,000, but project
(a) Data governance - A quality project was
  delivered without a single flaw.
(b) Waterfall - difficult to use.
(a) Data governance - no risk because client
  tested and approved every new feature
  daily. Financial expenditure was minimal.
(b) Waterfall - very risky as all projects
  except one failed, wasting large sums of
  taxpayer money.
           Client satisfaction
(a) Data governance - clients were
  extremely satisfied.
(b) Waterfall - all projects failed, except the
  centralized state app, which all clients
  disliked so much, they tried to build their
What was the most important criteria item in the
 personnel system project?

1. Time
2. Scope
3. Budget
4. Quality
5. Risk
6. Client satisfaction
                             Project results
    No client requirements were left
Identical requirements
•   Individual state teams
•   Private industry
•   A state department

What was the project manager best at?

    Client requirements management
•    Understanding client needs
•    Identifying known and unknown client
•    Translating business functions into data
     models that fulfill client requirements
What happens to projects after they
        are completed?
         Success factors
•   Continual contact with clients
•   Modeling data to translate data model
    into client solutions.
    Enterprise system decay
• Change is inevitable. As new components
  are added or change, the IT systems
  across the enterprise begin to slip out of
  alignment with each other.
• Examples are (1) the statewide
  procurement system (2) library book file
  before it was connected to the centralized
Even systems that have recently been rewritten
from scratch or purchased new begin to
disintegrate immediately if small changes are not
evaluated for integration opportunities
     What prevents enterprise
         system decay?
• Whenever there are local changes,
  enterprise-wide data reviews must be
• Change management including data
  governance reviews keep it in tune.
     Continually tuning a whole
• To do this, the Data Governance Council must sit with or
  have access to several critical gateways, such as the
  Change Management Board, FSRs, etc.
• Data Governance Council can then review changes with
  an enterprise-wide perspective.
• It would use the data checklist to look for data
  sharability, harmonization and other opportunities.

If enterprise data were always kept fully
normalized and updated for business rule
changes, would any system re-writes or
replacement purchases be necessary?
       Enormous value of data
• Add up the cost of small, medium and
  largest system rewrites, unnecessary
  maintenance, unnecessary labor and lost
  functionality to see the true value of
  keeping data models fine-tuned.
• Newness of IT equipment is not relevant.
Suggested PMO role in data governance
Ensure that data governance is applied to each project

• Make sure there's an enterprise data modeler looking at
  enterprise-wide and statewide integration opportunities,
  not just a data modeler.
• Ensure that data considerations are reviewed at the
  earliest stages of projects, e.g., conception phase
• Data governance deliverables to PM for each project:
  (1) Data architect initial review of overall data
  interoperability opportunities
  (2) Certification that data is correctly modeled
  (3) Change management comments (comments on
  changes submitted to project’s change management
Additional PMO opportunities
• PMO is a good fit as a member of the Data Governance
  Council. Data Governance discussions with PMO would
  benefit both the Data Governance Council and PMO, as
  PMO would become aware of business opportunities
• Critical gateway notifier
• Change management partner
• Cultural change ambassador
       Concluding thoughts
 The natural trend is to stovepipe
 Data governance reverses that trend
         Data governance summary
• It’s important
• It’s urgent – timeliness is critical
• It significantly affects all facets of the
  organization. UMass Boston and Dept. of
  Insurance successes were transformational to
  those agencies. Data governance
  revolutionized the business side.
• It’s a new discipline for improving BP
• Implementation is simple. Just a checklist.
• It continually tunes IT to business needs

To top