Data Governance A common thread runs through in a vast number of business problems that most problem solvers cannot see Data that is not designed to be interoperable Data design is the heart and soul of how IT enables government to fulfill its mission What is badly designed data? One type of bad design is information that is locked into data islands Data on one island cannot talk to data on another island Another term to describe locked data is “siloed” data or “stovepiped” data Example of how data models affect business processes A programmer is hired to design a system for a new library For convenience of illustration, let's say that patrons can be uniquely identified by first name Problems •Redundant address •Change of address difficulty •Redundant book author problems Data Problems This example shows exactly how business processes are affected by poor data design Bad data design forces business side to work harder than necessary and without good information New layers of inefficient business processes form to adapt to the poor data design For example, a business process could exist where librarians have to hire an army of data entry staff to do error checking to make sure patron names are correctly spelled Bad data design forces creation of layers of bad business processes Twenty-year problem lock-in Data design plays a large part in controlling programming and business processes Enter Data Governance Data governance cleanly resolves inefficient business processes PATRON FILE Book file only contains info about books BOOK FILE Finally, we have a concept that handles the transaction of a patron checking out a book INTERSECTION FILE computer screen display of what clients see Advantages: • Only have to type in the data once • Address changes handled cleanly. Old address is updated for all current books checked out • Automatic error checking on patron or book name Data governance improves service to the business side Minor changes in data design translate into huge business improvements throughout the organization Client satisfaction is increased Nation-wide interoperability A centralized government database from the Library of Congress of all books can be connected to so that library staff doesn’t have to type in book information such as author name, title, etc. The business side has benefited from even more savings in costs and work The data islands are now connected Imagine the work savings from not having to type in new book information such as title and author name This is how data governance brings cost savings Additional improvements are in time, quality, and organizational agility These are not one-time savings, but ongoing savings that leverage each other Agility • Converting to the centralized national book database was easier because the data was already modeled correctly. • They just replaced their book file with the national centralized one. • The error checking and streamlined book check-out process were already in place. Modeling data guarantees business agility Data design is the heart and soul of how IT enables government to fulfill its mission The library system is an example of how tightly data design is coupled to business processes Where is bad data design reflected in the real world? Business clients, e.g., the librarians, can’t visualize that data design caused the various problems Clients see the problem camouflaged as a lack of functionality in many areas their business processes Dept. of Veterans Affairs QUIZ • How did it happen? • What was missing? Answer: a comprehensive way looking at the interconnection of problems and prioritizing them in coordination with the enterprise data vision Do you see the correlation to the centralized remote book file for the library? FBI FBI Director Robert S. Mueller, III FBI had competent vendors and project managers install their system • On time • On budget • On scope. • Good data modelers QUIZ: What was missing? Answer: a comprehensive way looking at the interconnection of problems and prioritizing them in coordination with the enterprise data vision • Hidden requirements were not identified. • No organizational structure existed to ensure enterprise-wide interoperability. • A process that continually takes the enterprise-wide view of how to implement change. They just added systems to the rest like a manager of a fleet of cars adds a new car. They didn’t check for interoperability. The problem was repeated over and over again for each new FBI project. Data governance was missing. New way of looking at the enterprise 1. Connections 2. Potential connections not yet implemented Using this new method to view an organization, how pervasive is this problem? • The problem is huge and expensive. E.g., Fi$cal, 21st Century projects were designed to handle lack of comprehensive integration. • When someone leaves their organization example. • We’re basically talking about everything. A interpreter is needed that identifies business problems stemming from bad data design and then designs the data models that fix the problems at an enterprise-wide level Data governance Data governance is the practice of organizing and implementing policies, procedures and standards that maximizes data access and interoperability for the business mission Why is data governance needed? • Business needs drive data governance This is what interoperability looks like both to business side and the IT side Each link represents a business value where data can reach the business people that need it But beyond the requirements that are represented in the links, many more business requirements are invisible to all except the enterprise data modeler Requirements gathers cannot find them What integration opportunities look like to an enterprise data modeler Currently, much data is stovepiped Many connections are missing Each potential link between data that is not implemented represents a loss of business value QUIZ How do stovepiped systems get built? Forces that create data stovepipes • Contractors and vendors • Programmers • Managers • COTS • Budget structures • Security concerns • Methodologies: SDLC, PMBOK, WATERFALL, AGILE, SPIRAL • Change In sum, an absence of data governance Organizations mistakenly think installing new applications are like replacing one of the cars in their fleet It’s broken, so….replace it… QUIZ What is a better metaphor for representing a new software system than replacing a car? A component of a Neural Network It is important to add to the project team, a person that can check for enterprise interoperability opportunities This is a key method regarding how to look at new projects The neural network will ensure that one hand will know what the other hand is doing A small sampling what types (part of a much larger list) of improvements the links represent • Eliminates manual operations, e.g., when redundant tables or fields are eliminated • Business side has the data it needs when it needs it • Organizational agility Closer IT alignment to organization’s mission. Example of one of the links at the California Dept. of Consumer Affairs Calif. Secretary of State website Corporation data interoperability was an invisible business requirement Because data interoperability problems are invisible, business processes are unnecessarily limited throughout the enterprise due to undiscovered data integration What is data governance’s goal? Bring together the business side, with the people that can see the invisible business requirements in order to build the data structures that will give business the information it needs when it needs it Why is the need to implement data governance urgent? The design phase for projects is relatively short The opportunity to correct it at the right stage consisted of only a tiny, fleeting time window So why is lack of data governance urgent? Project development is currently going on in many areas without the benefit of data governance, which could result in permanent stove-piping of data. Result: Many stovepiped systems. QUIZ How many years can bad data design affect a system? Inefficient business process may get locked in for 20 years or more Twenty-year problem lock-in How does data governance work? The data governance process acts as a central planning center to coordinate data design across organizations It builds the enterprise-wide blueprint that guides all IT development towards data interoperability Simple process!!! Data Governance Council reviews changes or new development against a checklist The governance council reviews all new software development to see if it could be shared enterprise-wide New table or field checklist • Correct data modeling (3rd normal form), correct keys. (Data Architect, IT) • Cross agency data and process sharing opportunities, including SOA and Web Services (Data Architect, IT and business side) [Image: waiting clocks: text: “Clients don’t have to wait until they request data sharing. Comprehensive data sharing opportunities check is done at the outset.” • Business Intelligence – data mart/data warehouse opportunities. (Business side) • Metrics generation opportunities - Can this field or table create useful metrics or appear on a DCA dashboard? Customers could include boards, licensees, the public, finance, governor's office and the legislature (business side) • KPI - Key performance indicators opportunities. Are there opportunities to use the field/table to measure performance? (Data Architect, IT and business side) • Is data subject to legislative oversight or mandates? E.g., Health Insurance Portability and Accountability Act (HIPAA), California Database Breach Act (California SB 1386), http://www.itl.nist.gov/fipspubs/geninfo.htm, FIPS, HSPD-12. Create table of federal, state and departmental regulatory mandates or voluntary guidelines that reviewers check data against (Data Architect, business side) Checklist continued… • Are there opportunities from making this available to a broader audience? Customers that are not immediately evident could include boards, licensees, the public, enforcement, DOJ, finance, governor's office and the legislature (Data Architect, IT and business side) • Can this data be replaced by a better source of data elsewhere or replace other data? Can whole tables be eliminated by consolidation and sharing? (Data Architect, IT and business side) • Prioritize super connector fields and super connector tables. (Super connector fields are those that cross agency boundaries. Super connector tables are the most important tables that can be shared. These must be listed in a transparent, centralized database and reviewed for use whenever a new system is designed. Vendor created systems must be reviewed for table/data sharing and naming conventions.) If this is a new super-connecter, then it should be transparently registered to the repository so that other strategic planners can see it. (Data Architect, IT and business side) • Can it be used to validate data or does it need validation performed on it? (Data Architect, IT and business side) • Data harmonization problems or opportunities. (Data Architect, IT and business side) • Standards evaluation – Are there standards to be adhered to or created? Data standards, business standards, naming conventions, etc. For example, every state agency could have the same standard for this field: Corporation_Tax_ID_Number 40 characters – alphanumeric. Does NIEM. gov already have a standard name for this field? (Data Architect, IT and business side). Example, CAS Class field. Checklist continued… • Alignment to organizational mission. Strategic planning problems or opportunities. (Data Architect, IT and business side) • Enterprise Architecture planning. How does it align with to the “To Be” architecture? • Impact on other systems. Entered into Architect’s dependency database (what systems does it impact or is it impacted by) (Data Architect, IT and business side) • Metadata opportunities (business side) • Risk (Data Architect, IT and business side) • Security. Should it be encrypted? What controls should be applied (Data Architect, IT, business side and ISO) • Should client be given control of the data? Would a data steward be useful for this data? (Data Architect, IT and business side) • Backup considerations - how often. How does it get refreshed when there is a crash? When should it be purged? (IT and business side) • Can data quality be improved? Is data cleansing applicable? (IT and business side) • Quality management – are clients satisfied? Is quality management and continual process improvement built into this system? (Data Architect, IT and business side) • Automated duplicate detection (IT) Checklist continued… • Timeliness. Is there value to the organization if the data is refreshed sooner or by other ways? (IT and business side) • Is the data coming from the best sources (lineage; most reliable, timely)? (IT and business side) • Enterprise architect’s calendar scheduled for periodic data design review every two years • Priority on architect’s data design inventory. (Data Architect) • Optimization by combining multiple projects (past, future or ongoing projects). (Data Architect, business side, IT) • Review for entry into a table of future opportunities and linked to a calendar of related opportunities or future change events. For example, if a related component was scheduled for updating, that would trigger an automatic reminder to review opportunities for this component. (Data Architect) • Audit policy – should the field or table be have its edit or use history recorded (IT and business side) • Error management (IT and business side) Generally, client requests for government interoperability arrive inconsistently as clients struggle to understand how to improve their systems The above checklist ensures that a whole series of government improvement opportunities are checked for at the precise movement when it’s most important Clients don’t have to wait until they request data sharing. Comprehensive data sharing and all other opportunities checks are made at the outset. QUIZ What it would it look like 10 years from now if every government agency used this checklist for every new project? QUIZ Which one of the check list items was used to discover the SOS opportunity? • Correct data modeling (3rd normal form), correct keys. (Data Architect, IT) • Cross agency data and process sharing opportunities, including SOA and Web Services (Data Architect, IT and business side) • Business Intelligence – data mart/data warehouse opportunities. (Business side) To handle the complexity of data, a simplified super-connector check list would assist the Data Governance Council identify data sharing opportunities Tables and fields that have the most intradepartmental and statewide connectivity. Examples: – Corporation Number – License type and license number – SSN – Address (including apartment number) – Criminal case number – Civil case number – Agency code Data governance concept is simple Whenever there’s change we ask a question: Can this be shared in or outside of our organization? Data Governance flow chart Sample integration priority list • Licensing data model changed to make individual unique identifier (QUIZ: how was this identified?) • Remove status code constraints from programming code and move them to table-based system (Where did this idea come from? Data modeler) • Enforcement measurement fields (QUIZ: Where did this come from?) Without data governance, how do data improvement opportunities traditionally become known to IT? Answer • Business client submits a ticket for a problem. This involves delay. • IT manager or OCIO recognizes a pattern. This involves delay. • Business managers recognize a pattern. E.g., LUG and EUG (Board user groups) collectively discover a problem. This involves delay. • Vendor products and recommendations. This involves delay. All of the current methods involve delays and do not provide a consistent and continual process for identifying and addressing problems enterprise-wide How do new opportunities come to the Data Governance Council’s attention? • Project conception • PMO • SDLC • FSR • SPR • PIER • RFP • Change Management Board • IT Governance • PIT (Process Improvement Team) • Strategic plan • Executive strategic discussions • BCP • ITPP - Information Technology Procurement Plan • PSP - Proposal Solicitation Package • Table and field creation process (DBA, programmer, etc.) • List of business-side data related requests • Informal business projects, such as potentially sharable spread sheet data • Programmers or business clients Data Governance shortens the time that it takes to determine business clients need a data change Otherwise, IT waits until business side submits a ticket for a specific problem. They don’t know they have a data modeling problem or don’t realize that their business process can change through data design. A comprehensive array of discovery points above speed up identification of improvement opportunities QUIZ At what discovery point was the SOS opportunity found? (Hint…) • Project conception • PMO • SDLC • FSR • SPR • PIER • RFP • Change Management Board • IT Governance • PIT (Process Improvement Team) • Strategic plan • Executive strategic discussions • BCP • ITPP - Information Technology Procurement Plan • PSP - Proposal Solicitation Package • Table and field creation process (DBA, programmer, etc.) • List of business-side data related requests • Informal business projects, such as potentially sharable spread sheet data • Programmers or business clients Was this too late? When is the best time to discover data opportunities? How can data governance be implemented? Stage One: To handle urgent problems. A quick Stage One with a short time line is envisioned without lengthy discussions DBAs simply email Data Architect any planned schema updates • Simple, cheap and effective. • Reasoning: Project development is currently going on in many areas without the benefit of data governance, which could result in permanent stove-piping of DCA data. • We want to catch any emergency problems in the bud. Stage Two Volunteers from business and IT form an initial, first version of Data Governance Council. The Data Governance Council designs itself and processes are to be improved as we collectively gain more experience. Together, business and IT build the data integration vision • E.g., data warehouse • Enterprise connectivity • Etc. What factors make data governance successful? Data governance is between 80 and 95 percent communication. The most important factor in most successful data governance programs is communication Clearly, data governance is not a typical IT project! How effective is data governance? • Very cost effective • Vast scope of business processes improved • Money savings example: Fi$cal All areas of the organization are improved Data Governance advances the efficiency, cost savings and agility for every service • PMO • Process Improvement • Ticket system • IT Governance. Data governance will help every project become more successful wherever it’s included What are the benefits of data governance? Each time there is a single integration improvement, it removes roadblocks to the organization's mission. Data silos become accessible, clients' problems are reduced, maintenance problems are reduced and connectivity opportunities open up across departments. This incremental method is also the least expensive approach. Benefits of data governance • Greater department-wide and statewide interoperability • Citizens receive better service from integrated government business processes. Data will be more accurate, complete, and timely. Working with government will be more convenient, for example, when citizens only need to go to a single government agency to update their address instead of multiple government agencies • Brings the business side into the IT improvement process • Better business side control over data, privacy and project development • Faster identification and implementation of business solutions. Data governance methodically discovers the gaps in how IT services business and shortens the time from problem discovery to solution. Data governance shows the business side how to find their voice in collaborative problem solving. • Improved business decisions due to accurate data from the recognized source of record Benefits of data governance continued • Increased user business side trust in data stored within the organization's databases • Helps meet the enterprise’s business goals including adaptation to changing regulatory and other environments • Eliminates data duplication as a result of data governance process • More accurate, consistent, complete, accessible and up-to-date data • Fraud detection is facilitated because all data field names across the department and state are standardized • Placing all data related requests in one place allows patterns to be identified • Clear documentation of the lack of integration may provide business managers with better new project proposals • Ease of business process refinement due to standardization of data components • Opportunities for harmonizing and standardizing business terms because stakeholders are brought together in a collective review process. For example, if identical meaning terms were "cost allocation" and "distributed cost", stakeholders could agree to standardize on one of the terms and remove the other from business documents such as contracts and agreements Benefits of data governance continued • Business Intelligence. Data warehouse creation simplified through standardization of business data • Better programming code due to correctly organized data • Agility in responding to new opportunities • Stops business system decay. Keeps all systems tuned to organization’s mission and to each other so that no new system re-writes are ever necessary. How has data governance worked in real life? UMass Boston Transformational to business side • Projects were completed much faster • Project quality was much higher • Greater programmer collaboration • Greater business side collaboration Productivity went through the roof Solutions in all areas of business • Inventory • Finance • Project management • Etc. Many unexpected business benefits were revealed Unlocked data allowed more opportunities for innovation and agility Clients were extremely satisfied Clients were extremely satisfied • New business advantages • Data was prescient • Easier to get reports • Reduced workload • Reduced errors Success factors: (1) Continual contact with clients (2) Modeling data to translate data model into client solutions UMass Boston Each time a new software system was installed, it was completely and totally integrated, not just partly, but with every possible data connection fully implemented. Every table and field was examined for enterprise integration. Continually tuning the organization UMass Boston One of the first data-integrated organizations in the country UMass Boston data governance Built process improvement into the DNA of the organization Vision UMass Boston model of continual tuning for enterprise-wide integration Where do we go from here? QUIZ (1) Can you name any software product that has a tuning process (when one part changes or has new components added or a different vendor adds something to it, all parts are evaluated)? (2) What is the current process for tuning this organization’s data? Personnel System Personnel System Contenders in the project management competition: • Teams from many state departments • State Personnel Board • Private Industry • Single individual Methodology comparisons • Teams from many state departments - main tool: waterfall - failed • State Personnel Board - main tool: waterfall • Private Industry – main tool: waterfall - failed • Single individual – main tool: (1) working daily with client to understand requirements (2) modifying data design daily to translate requirements into best data model Comparison of old school methodology and data governance performance • Time • Scope • Budget • Quality • Risk • Client satisfaction Time (a) One data governance method developer completed the personnel system in two years. (b) Waterfall - The only other contestant that finished the project was the centralized state department team that had six programmers working on it for twenty years. Scope (a) Data governance method developer completed a fully automated personnel system. (b) The centralized state department team system was not fully automated. Budget (a) Data governance - $50,000 a year for one developer's salary for two years. (b) Waterfall - difficult to estimate, but at least ten times as much. (c) Private industry - $500,000, but project failed. Quality (a) Data governance - A quality project was delivered without a single flaw. (b) Waterfall - difficult to use. Risk (a) Data governance - no risk because client tested and approved every new feature daily. Financial expenditure was minimal. (b) Waterfall - very risky as all projects except one failed, wasting large sums of taxpayer money. Client satisfaction (a) Data governance - clients were extremely satisfied. (b) Waterfall - all projects failed, except the centralized state app, which all clients disliked so much, they tried to build their own. QUIZ What was the most important criteria item in the personnel system project? 1. Time 2. Scope 3. Budget 4. Quality 5. Risk 6. Client satisfaction Project results No client requirements were left uncompleted Identical requirements • Individual state teams • Private industry • A state department QUIZ What was the project manager best at? Answer: Client requirements management • Understanding client needs • Identifying known and unknown client requirements • Translating business functions into data models that fulfill client requirements Confidence What happens to projects after they are completed? Success factors • Continual contact with clients • Modeling data to translate data model into client solutions. Enterprise system decay • Change is inevitable. As new components are added or change, the IT systems across the enterprise begin to slip out of alignment with each other. • Examples are (1) the statewide procurement system (2) library book file before it was connected to the centralized database. Even systems that have recently been rewritten from scratch or purchased new begin to disintegrate immediately if small changes are not evaluated for integration opportunities What prevents enterprise system decay? • Whenever there are local changes, enterprise-wide data reviews must be made. • Change management including data governance reviews keep it in tune. Continually tuning a whole organization • To do this, the Data Governance Council must sit with or have access to several critical gateways, such as the Change Management Board, FSRs, etc. • Data Governance Council can then review changes with an enterprise-wide perspective. • It would use the data checklist to look for data sharability, harmonization and other opportunities. QUIZ If enterprise data were always kept fully normalized and updated for business rule changes, would any system re-writes or replacement purchases be necessary? Enormous value of data governance • Add up the cost of small, medium and largest system rewrites, unnecessary maintenance, unnecessary labor and lost functionality to see the true value of keeping data models fine-tuned. • Newness of IT equipment is not relevant. Suggested PMO role in data governance Ensure that data governance is applied to each project • Make sure there's an enterprise data modeler looking at enterprise-wide and statewide integration opportunities, not just a data modeler. • Ensure that data considerations are reviewed at the earliest stages of projects, e.g., conception phase • Data governance deliverables to PM for each project: (1) Data architect initial review of overall data interoperability opportunities (2) Certification that data is correctly modeled (3) Change management comments (comments on changes submitted to project’s change management process) Additional PMO opportunities • PMO is a good fit as a member of the Data Governance Council. Data Governance discussions with PMO would benefit both the Data Governance Council and PMO, as PMO would become aware of business opportunities • Critical gateway notifier • Change management partner • Cultural change ambassador Concluding thoughts The natural trend is to stovepipe Data governance reverses that trend Data governance summary • It’s important • It’s urgent – timeliness is critical • It significantly affects all facets of the organization. UMass Boston and Dept. of Insurance successes were transformational to those agencies. Data governance revolutionized the business side. • It’s a new discipline for improving BP • Implementation is simple. Just a checklist. • It continually tunes IT to business needs Questions…Feedback?
Pages to are hidden for
"Data Governance"Please download to view full document