Docstoc

Company Overview

Document Sample
Company Overview Powered By Docstoc
					Delivering AMI with




                                                                  Partners in Technology
StreamInsight and
BizTalk Server
Craig Butler
Center of Excellence Leader, Application Integration
    follow me - @scbutler1
            Deliver Solutions, Deliver Careers, Deliver Results
Company Overview
    Regional IT professional services firm




                                                 Partners in Technology
•
•   Established in 1993
•   Five offices in the Southeast
•   190+ employees
•   200+ clients in various industry verticals
                         Expanding Presence




Partners in Technology
Company Foundation
• Rigorous and disciplined engineering approach




                                                  Partners in Technology
• Relentless focus on quality
• Intensive client interaction
• State-of-the-art technology
• Continuous staff and
  process improvement
Session Objectives and Takeaways
• Session objective(s):




                                                                    Partners in Technology
  • Discuss the challenges presented by an AMI scenario
  • Identify the various components of StreamInsight and how they
    are used within the solution
  • Demonstrate StreamInsight and BizTalk Server being used
    together

• Takeaways:
  • Look for and go after AMI and similar workloads
  • The Microsoft stack is very capable of solving these problems
  • Provide community with reference architecture
Applicable to other Scenarios
    Statistical Process Control




                                                               Partners in Technology
•
•   Financial Services Solutions
•   Supply Chain Solutions
•   Monitoring
•   Web Click Through
•   Lots of smart devices generating useful data.
    Companies want to be able to collect it, do stuff to it,
    and make sense of it before it all becomes stale.
We are really showing you how to….




                                     Partners in Technology
What is AMI?
• Automated Meter Infrastructure




                                                           Partners in Technology
• Allows power companies to…
  • Receive events for outages, tampers, high temp, etc.
  • Query the meter to receive usage information
  • In some cases, control household appliances

• Critical Component of Smart Grid Initiatives
What is Smart Grid?




                                                   Partners in Technology
• Smart grid is a type of electrical grid which
  attempts to predict and intelligently respond
  to the behavior and actions of all electric
  power users connected to it - suppliers,
  consumers and those that do both – in order
  to efficiently deliver reliable, economic, and
  sustainable electricity services
Future Vision of the Energy Ecosystem
                                                                                             Tomorrow‘s Weather: Sunny, windy, and warm.
                     Hydro




                                                                                                                                                                                                                                                       Partners in Technology
                  Electric                                                  Solar
                                                                                              Wind                                          Nuclear
                                                                                                                    Wind
                                                                                                                                                                        Nuclear &
                                                                                                                                                                       Desalinization
                                                                                                                                                                          Plant                                                 Offshore Wind
                                                                                    Solar

   Hydro Power
                                                                                                                                                                                                   Low Emission
                                                                                                                                                                                                    Power Plant                          Tidal Buoys

                                                                                                                                         Biomass             Offices
                                                                                            Residential
                                                                                               Control
                                                                                                                                                                                                           Sub Station
                                                                                              Solar
                                                                                               Center
                                                                                                                                                   Commercial Offices &
 Vehicle                                                                                                                                              Storefronts

                                                   Microgrid
Recharging                                                                                                                                                                                                        Microgrid


                                                                                                                                                   High Consumption
                                                                                                         Photovoltaic's
                                                                                                                                                    (Manufacturing)                     Power Substation


           Parking Facility with
          PHEV Recharge Outlets
                                                                              Electrical                                                                       H2 Filling Station
                                                                                                                                                                                                                   Manufacturing Plant
                                                                                                                                                                                                                   (High Consumption)

                                                                              Storage                                        Microgrid




                                                               Fuel Cells
                                     Electricity                                                                   Thermal
                                   Micro Storage                                                                                                                                                                                          SMES
                                                                                                                   Storage                              Compressed
                                                                                                                                                        H2 Storage
                                                                                                                                                                                           CO2
                                                                                                                                                                                         Storage
 2003 Power Outage
• The report states that a




                                       Partners in Technology
  generating plant in Eastlake, Ohio
  (a suburb of Cleveland) went
  offline amid high electrical
  demand, putting a strain on high-
  voltage power lines (located in a
  distant rural setting) which later
  went out of service when they
  came in contact with "overgrown
  trees". The cascading effect that
  resulted ultimately forced the
  shutdown of more than 100
  power plants.
  Smart Grid Brings a Data Tsunami!
1,000              Historical Growth




                                                                        Partners in Technology
 900

 800

 700

 600
    35% annual growth
                                                                 897
 500

 400
                                                          641
 300
                                       434
 200
         271       323
 100

   0
         2005      2006                2007               2008   2009

                          Total TB of allocated storage




The data tsunami is coming -- and Utilities
are totally unprepared to handle the
deluge! – Becky Blalock, SVP & CIO
Southern Company
 The Opportunity




                                                            Partners in Technology
• The US Government is providing 4.5 billion to modernize
  the electric power grid.
• Massive Data Storage and Reporting Requirements
• Complex Integration Requirements
• Most Customers already own SQL, .NET, Windows, VS
About Southern Company
• Based in Atlanta, GA




                                    Partners in Technology
• Generates power for 4.3
  million customer in the
  southeastern United States.
• Owns electric utilities in four
  states (AL, MS, GA, FL)
• Competitive power generation.
• Fiber optic and wireless
  communications.
• Listed 165th on Fortune 500.
                         AMI Architecture




Partners in Technology
Solution Overview
• Goal




                                                            Partners in Technology
  •   Create Automated Meter Reading (AMI) Infrastructure
  •   Use as reference architecture
  •   Use for demos
  •   Make available to community
• Technologies Used
  •   Windows
  •   SQL Server
  •   BizTalk Server
  •   .NET
High Level Solution Diagram




                              Partners in Technology
Why StreamInsight




                                                     Partners in Technology
• Need to process potentially 10’s of thousands of
  messages a second
• Need multiple hosting options
• Need a robust query engine
• Need to be able to reuse and compose streams
      StreamInsight Platform
                                           StreamInsight




                                                                                                             Partners in Technology
                                      Application Development

  Event sources              StreamInsight Application at Runtime                Event targets
                            Input                                    Output
    Devices, Sensors       Adapters        StreamInsight Engine     Adapters   Pagers &
                                                                               Monitoring devices

                                         Standing Queries                                  KPI Dashboards,
                                                                                           SharePoint UI
     Web servers
                                           Query            Query
                                           Logic            Logic

                                                                                        Trading stations
Event stores & Databases
                                                            Query
                                                            Logic

Stock ticker, news feeds                                                       Event stores & Databases
Scenarios for Event-Driven Applications
 Latency
Months




                                                                                                   Partners in Technology
Days                                                                   CEP Target Scenarios

                    Relational Database Applications
hours                                                        Operational Analytics Applications,
                                                                     e.g., Logistics, etc.
Minutes                                       Data Warehousing
                                                Applications
Seconds
                                                               Web Analytics Applications

100 ms         Monitoring                 Manufacturing Applications     Financial trading
               Applications                                              Applications
< 1ms


           0            10          100          1000         10000      100000       ~1million

                                Aggregate Data Rate (Events/sec.)
                                                                                                                            20
Practical Example
How many red




                    Partners in Technology
cars are in the
parking lot
behind your
office?
Practical Example (cont.)
Physical Approach                     Relational Database




                                                                   Partners in Technology
• Walk out to the parking lot,        • Connect to the target
  with a clipboard and a pen.           database
• Walk through the parking lot,       • Execute some SQL
  and look at each vehicle              command such as SELECT
• If the vehicle is a passenger car     COUNT(*) FROM ParkingLot
  and is red, tick off on my            WHERE type = ‘AUTO’ AND
  clipboard.                            color = ‘RED’
• Count up all of the ticks, head
  back into the office.
Example Refined
How many cars




                   Partners in Technology
have passed your
office on the
highway in the
last 20 minutes?
Example Refined (cont.)
Physical Approach                   Relational Database




                                                                   Partners in Technology
• Pull all of the cars off of the   • Load the incoming car
  highway into a holding lot.         records into the database.
• Leave each car there for at       • Continuously execute some
  least twenty minutes, and keep      SQL command such as
  track of who is allowed to          SELECT COUNT(*) FROM
  leave.                              ParkingLot WHERE type =
• Wander around counting up all       ‘AUTO’ AND color = ‘RED’
  of the cars every time one        • Have a separate job that
  enters or leaves.                   periodically evicts cars
                                      (records) from the
                                      database.
Example Refined(cont.)
Physical Approach                  StreamInsight Approach




                                                                   Partners in Technology
• Stand by the side of the road,   • Set up a continuously
  counting cars as they go past.     running standing query
                                     along the lines of the LINQ
                                     statement below.
                                   • Use the StreamInsight
                                     engine to connect this
                                     query to an incoming
                                     stream of Vehicle
                                     observation events.
High Level Solution Diagram




                              Partners in Technology
StreamInsight Solution Diagram




                                                        Partners in Technology
                          Outage Query
               An outage event ‘O’ without a tamper
               event ‘T’ within 15 seconds is a valid
               outage event.

               Outage events ‘O’ followed by a tamper
               event ‘T’ within 15 seconds should be
               ignored.
StreamInsight Demo Overview




                                             Partners in Technology
• Create the host instance
• Create the Management Service (optional)
• Create the Stream
  • Create and Configure the Stream Input
  • Create the query
  • Create and Configure the Stream Output
• Start the Stream
Partners in Technology
StreamInsight Advantages and Challenges
• Advantages




                                          Partners in Technology
  •   Low Latency High Volume
  •   Can Compose Streams
  •   Leverages Linq
  •   Can be self hosted or embedded

• Challenges
  •   Queries can be complex
  •   Have to custom code adapters
  •   High availability is not OOB
  •   DR is not OOB
Why BizTalk Server?
    Adaptability




                                              Partners in Technology
•
•   Guaranteed delivery
•   ESB pattern to dynamically route events
•   Manageability and Scalability
•   Enterprise host
•   Process Management
Messaging for Your Distributed Environment




                                                         Partners in Technology
   Message




                Message       Message




                                               Message
                                Host




                                        Host
Reuse IT Assets With BizTalk Server




                                                                                                                    Partners in Technology
                                                   Effective Process
                                                   Implementation
                   Modifiable
                  Business Rules

                                                                                                        Mainframe
   Scalable                           Message                               Message


 Event Driven                                             Business Rules                Heterogeneous
                                                                                         Connectivity
                                                              Engine



                                                          Orchestration

                                    Receive Port                            Send Port                    Service
                                    Receive
                                    Adapter
                                                                              Send
                                                                             Adapter


                                                               Host

                                     Receive                                  Send


                                                                                                         Line of
                                     Pipeline                                Pipeline




                                                                                                        Business
                                        Host                                   Host




                                                          MessageBox
                                                         Persistent Store




                Business Activity                                                        Exception
                  Monitoring                                                            Management
   ESB Toolkit Solutions for the Challenges
Challenges:                             ESB Solutions:




                                                                                   Partners in Technology
• Location changes                      • Dynamic Endpoint Resolution
• Sequencing changes                    • Itinerary-based Routing
• Semantics differences                 • Dynamic Transformation
• Transport disparities                 • Protocol Mediation
• Interaction model incompatibilities   • Itineraries-based Mediation/Processing
• Quality of protection                 • BizTalk Security Model & SSO
• Error recovery                        • Exception Framework & Portal
• Monitoring & visibility               • BAM Integration
• Quality of service                    • Exception Management Portal
BizTalk ESB Core
• Itineraries




                      Partners in Technology
• Resolvers
• Adapter Providers
High Level Solution Diagram




                              Partners in Technology
BizTalk Solution Diagram




                           Partners in Technology
High Level Solution Diagram




                              Partners in Technology
SQL Solution Diagram




                                                    Partners in Technology
     BizTalk ESB               BizTalk ESB
Process Event Itinerary   Process Event Itinerary
    BizTalk Demo Overview




                                Partners in Technology
                      O     C


E        E

             O
Partners in Technology
BizTalk \ ESB Advantages and Challenges
• Advantages




                                                             Partners in Technology
  •   Publish and Subscribe Engine
  •   Adapters
  •   Mapping
  •   Itineraries
  •   Dynamic Resolution

• Challenges
  • Itineraries only move messages from point A to point B
  • ESB Documentation (Knowing where to do what….)
  • Debugging
Visualizing Meter Data
• Let’s now focus on what to do with this data




                                                 Partners in Technology
• Converting Data into Information
• Consuming information thru various mediums
  • Desktop
  • Web-based Applications
  • Mobile Devices
High Level Solution Diagram




                              Partners in Technology
SQL Solution Diagram




                                 Partners in Technology
        Entity Framework Model
                         Demo Overview




Partners in Technology
Recap - High Level Solution Diagram
                                 Repository for       Outage
                                 Meter Data and    Visualization




                                                                     Partners in Technology
    Pass-through all             Simulated LOB
 events to SQL, identify              App
 and submit outages to
      service bus




                                                   Simulated LOB
  Submit Meter                                    (Mark outages as
   Events with                                        resolved)
 Specific Timing
to Drive Queries           Manage Outage
                           Process and LOB
                           Communication
Always more to do…




                                                     Partners in Technology
•   Showcase Power Pivot
•   Showcase SSRS
•   Showcase Performance Point
•   Load testing and performance tuning
•   Incorporate the cloud
•   Incorporate the CIM (Common Information Model)
In Review: Session Objectives and Takeaways
• Session objective(s):




                                                                    Partners in Technology
  • Discuss the challenges presented by an AMI scenario
  • Identify the various components of StreamInsight and how they
    are used within the solution
  • Demonstrate StreamInsight and BizTalk server being used
    together
• Takeaways:
  • Look for and go after AMI and similar workloads
  • If your customer is a utility go after Smart Grid
  • MSFT stack is very capable of solving these problems
                  Questions & Answers




Partners in Technology

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:4
posted:12/18/2011
language:
pages:50