MIT Enterprise Architecture Guide - Download as PowerPoint

							                                                                                                                                                                                                                          Future State | Enterprise Integration Options

       Enterprise Integration Options
       Several options exist for integrating applications within the enterprise. This section gives a brief outline and comparison of                                        Option 4: ETL (Extract, Transform, Load) Tools
       the major options.                                                                                                                                                    Definition: A standard set of tools and processes used to extract, transform and load large volumes of data between
                                                                                                                                                                             systems. Very useful in populating a data warehouse
       Option 1: Custom Point-to-Point Integrations                                                                                                                          Attributes:
       Definition: A direct point-to-point link is created between applications for each business function                                                                     • Provides tools for data cleansing; correcting misspellings, resolving conflicts (city & zip code incompatibilities), missing
       Attributes:                                                                                                                                                                elements, parsing elements
         • Designed and implemented solely for the purpose of directly integrating two specific systems
                                                                                                                                                                               • Can combine data sources: Matching on key values, fuzzy matches on non-key attributes, textual comparisons to
         • Custom code for                                                                                                                                                        reference tables
              • Data extraction                                                                                                                                                • De-duplicate processing: Identifying and eliminating duplicates
              • Business rule processing
                                                                                                                                                                               • Can create surrogate keys: Operational systems and the data warehouse have different assumptions and data
              • Data loading                                                                                                                                                      requirements thus the data warehouse requires its own set of primary keys
         • Custom data format
                                                                                                                                                                               • Create aggregates to boost performance of common queries in data warehouses and data marts
         • Inconsistent transmission protocol
                                                                                                                                                                               • Loading and indexing: For large data warehouses specialized bulk loading processes are required
         • Used for both real time and batch integrations
       Advantages:                                                                                                                                                           Advantages:
         • No need to invest in expensive tools up front                                                                                                                      • Extremely efficient for moving large volumes of data in short timeframes
         • No need for developers to learn new skills and packages                                                                                                            • Applies consistent transformations
         • No extended time frame for developing and deploying enterprise integration strategy                                                                                • Can provide or integrate with meta-data for the enterprise data model

                                                                                                                                                                             Comparison of Options
       Option 2: Messaging or EAI (Enterprise Application Integration) Tools
       Definition: Source systems “publish” enterprise messages to a common bus; application “subscribe” to relevant messages
       and act on them                                                                                                                                                                         EAI                            Point to point               Web Services                       ETL
       Attributes:
         • “Wraps” each application
                                                                                                                                                                                   • Publish/Subscribe               • Custom code for each         • Standards based            • Suitable for large
         • Acts as a broker between applications
                                                                                                                                                                                     mechanism                         integration need               integration                  volumes of data




                                                                                                                                                               Concept
         • Provides near real-time, guaranteed, once-only delivery
                                                                                                                                                                                   • Most suitable for real          • Suitable for complex         • Most suitable for inter-   • Generally used to move
         • Stores and forwards messages                                                                                                                                              time data needs                   integration needs              organization integration     data between two or
         • Provides an environment in which to define rules                                                                                                                        • Loosely coupled                 • Tightly coupled              • Loosely coupled              more databases
       Advantages:
         • Systems are integrated but not coupled
                                                                                                                                                                                   • Reliability (guaranteed         • Familiar technologies and    • Standards based            • Metadata driven
         • Business rules are centralized in the message broker and transformation engine
                                                                                                                                                                                     delivery)                         processes                      integration                  approach




                                                                                                                                                               Strengths
         • Allows for near real-time integrations which reduced latency                                                                                                            • Enables real-time               • Many point to point          • High degree of reuse       • GUI tools for most tasks
         • Solves the n2 problem; as the number of systems increases, the integration effort expands linearly                                                                        business decisions                integrations already exist   • Wide tool support            (little coding)
                                                                                                                                                                                   • Out of box adapters for         • No major up front              including open source      • Extremely efficient for
                                                                                                                                                                                     many enterprise systems           investment required          • Low up front investment      large data volumes
       Option 3: Web Services
       Definition: Functionality to be integrated is exposed via XML on an open protocol such as SOAP. Other systems can
       consume this service if needed. Inputs and outputs to the web services are XML                                                                                              • High upfront cost         •       Costly over time             • Lack of transaction        • High upfront costs



                                                                                                                                                               Weaknesses
       Attributes:                                                                                                                                                                 • Relatively complex design •       Tight coupling                 support                    • Complexity of tool
         • Common language of communication across heterogeneous systems                                                                                                             patterns                  •       Scalability issues           • Not a publishing model     • Batch oriented
         • Based on standard Internet technologies                                                                                                                                                             •       Opportunities for reuse      • Less established
         • Self describing and advertising                                                                                                                                                                             are slim                       technology
         • Supports dynamic discovery & integration
         • Services fit within an overall architectural model                                                                                                                      • Real time data is               • Should be rarely used        • Integration model is       • In conjunction with a
                                                                                                                                                               When to Use



         • Widely supported by major vendors                                                                                                                                         important                       • When defined enterprise        request/reply                data warehouse
       Advantages:                                                                                                                                                                 • High volume, low                  strategy cannot work         • Real time requirements
         • Solves problems similar to those EAI solves, as well as                                                                                                                   footprint data exchange         • Proto typing                 • High volume, moderate
         • Need for expensive integration tools                                                                                                                                    • Many consumers of the                                            data
         • Use of proprietary integration platforms                                                                                                                                  same data
Version 0.1 – August – September 2004                                                                                                       Prepared by Sapient for MIT                                                                                                                                 Page 1
                                                                       This document represents a snapshot of an evolving set of documents. For information on further iterations, please visit: http://istwiki.mit.edu/istwiki/ItagFrontPage