Layout

							SonicMQ for LDIWG


Kris Kostro, Francesco Calderini
             AB/CO
             Layout
SonicMQ in CMW
JMS
Characteristics of SonicMQ
Connectivity

Does it fulfill the LDIWG requirements?
How could LDIWG be implemented with
SonicMQ
            SonicMQ in CMW
1999 requirement capture for CMW, product studies,
recognized need for MOM
May 2000 Preference for JMS, product evaluations
Sept. 2000 presentation by by Mitchell Horovitz, the
Technical Product Manager of SonicMQ
February 2001 SonicMQ has been selected as the
messaging product for the CMW project. Purchased 4
licences.
August 2001 First real use for timing events delivery
What is JMS?
         Java Message Service
    Industry-standard messaging specification
      Developed by Sun and other leading vendors
      Required component in J2EE 1.3
    Common set of APIs and semantics
      Publish and subscribe
      Point-to-point
    Message delivery Quality of Service (QoS)
      Guaranteed
      Once-and-only-once
      At-most-once
    Message content (properties), message
    types, filtering, etc. well defined
    Deployment architecture not addressed
SonicMQ JMS implementation
Hierarchical namespace (topics)
XML support on top of text message
(for DOM2, JAXP, SAX)
Multiple levels of Quality of Service
Connectivity beyond JMS
              SonicMQ
JMS implementation (in Java)
Market leader for JMS
Hub-and-spoke architecture
Scalable (clusters, load balancing, …)
High availability (parallel servers,
queues, topics, persistent topics, etc)
Security (SSL, certificates, HTTP
tunneling, firewalls etc.)
Integrating SonicMQ with
     C/C++ Clients
          Connectivity
HTTP
C, C++
COM
FTP
SNMP
Bridges to proprietary systems (MMQ)
Bridges to other JMS systems
How can SonicMQ fulfill the
  LDIWG requirements?
       Enterprise Messaging
Requires…
               Reliability
             Scalability
            Availability
        Security
       Connectivity        …and Sonic delivers!
          LDIWG requires
Availability (2, 3)
  SonicMQ is typically used in areas which
  much higher availability requirements
  Intrinsically high availability architecture.
  Brokers can be set up as redundant, can
  be clustered or partitioned into
  independent domains
                                                          Availability

     Connection Management
Removing Bottlenecks and Points of Failure
   Distribution of client connections across
   cluster
     Connection-time load balancing
        One or more brokers from list used as initial connection
        points
        Clients redirected to other brokers via weighted round-
        robin algorithm
   High availability of server connections
     Connect time failover
        Clients connect to 1st available broker in list
     Subsequent failover
        Reconnect on notification of connection loss
          LDIWG requires
Reliability (4, 5)
  Publisher down -> watchdog mechanism
  (outside of SonicMQ)
  Control frequency -> No, should be
  assured by the domain, authentication
  possible
  Guaranteed delivery possible
                                                 Reliability

          Guaranteed Delivery
Handles Failure at Any Point in Lifecycle
   Messages delivered even in the most
   challenging situations
      Persistent messages are stored and flushed to disk
      before being acknowledged
      Patent-pending storage mechanism offers
      reliability and high performance
   Dead Message Queue
   Includes large message support and client-
   side persistence
   Supports local or distributed (2-phase)
   transactions
      Reliable groups of actions
    LDIWG requires (cont.)
Synchronisation (6)
  Timestamp is part of JMS (ms), in LHC all
  data will be time-stamped at the source
    LDIWG requires (cont.)
Latency (7)
  Latency depends on configuration and
  required throughput
Performance (8)
  Guarantee of delivery (different levels)
  Throughput depends on configuration and
  QoS
  Extensive performance figures available
                                                              Scalability

                            Built to Scale
   Multi-threaded Broker
        Architected for high capacity*
             Connections > 2000
             Destinations > 80,000
             Messages > 8,000/s (persistent)
             Latency < 10ms
        Actual figures depend on environment
   Single Cluster
        Required when single broker capacity is exceeded
        Multiple brokers in cluster act as single virtual broker
   Communities of Clusters
        Linked via Dynamic Routing Architecture™ (DRA)
*Tested on 4-way 550MHz Windows NT Server (1K message size)
                                                           Scalability

        Expanding the Cluster
Achieve Linear Scalability
   No limits on cluster size
     Current deployments up             Head Office
     to 64 brokers              Business                    Business

   Clients are load-           Application                 Application


   balanced across the               Broker Cluster               Business
                                                                 Application

   cluster                         Broker         Broker
                                                                Business
   Messages routed                           Broker
                                                               Application


   through inter-broker
   connections where
   necessary
    LDIWG requires (cont.)
Adaptability (9)
  Fully dynamic configuration
Protocol features
  (10) JMS is an industry standard
  (11) Supports several platforms (see list)
  (12) On change semantics must be assured
  by the producer
    LDIWG requires (cont.)
Protocol features (cont.)
  (13) Grouping -> performance issue
  (14) Last published value ->posibility of
  persistence with timeout. Request for last
  value can be implemented outside of
  SonicMQ
   (15) JNDI can be used as naming and
  directory service
  (16) Hierarchical namespace with wildcards
         LDIWG constrains
Constrains
  (17) Can do much better than 10 messages/s but
  it is really scalability issue
  (18) Can do much better for latency, again
  scalability issue
  (19) No known blocking problems
  (20) No flow control inside SonicMQ (domain
  responsibility)
  (21) User-based identification and topic-level
  authorization via ACL
  (23) Administration utilities available
                                                            Security

      Comprehensive Security
Flexible Deployment Options
  Encryption
    Built-in payload encryption
       Secure messaging without performance cost of SSL
    Channel encryption
       SSL with up to 168-bit keys
  Authentication
    Built-in username/password authentication
    Supports digital certificates for user authentication
  Authorization
    Specify rights of authenticated users
    Exploit destination hierarchies and groups of users to
    simplify administration
SonicMQ Explorer
  How could LDIWG be
implemented with SonicMQ
    How could LDIWG be
  implemented with SonicMQ
Hierarchical namespace to deal
with “private” domain naming
XML as common, self-describing data
format
Bridge for each domain (SonicMQ
Bridges technology?)
Common administration
                 Example topics hierarchy
        Common root CERN
        Partitioned into administration domains (PS, LHC, ST, Alarms, CMS ..)
        Every domain defines it’s own hierarchy
        All accelerator domains follow the class/device hierarchy
                                                                            Root
                                     CERN


            PS               SPS                CMS               ST        Domain


                                                                            Class
                  Magnet                  BPM                    Class N


                                                                            Device
        Magnet1                 Magnet2               Device instance N


                                                                            Property
   Status                  Current



Can subscribe to status of all magnets: CERN.SPS.Magnet.*.Status
    How could LDIWG be
  implemented with SonicMQ
Hierarchical namespace to deal with
“private” domain naming
XML as common, self-describing
data format
Bridge for each domain (SonicMQ
Bridges technology?)
Common administration
   XML as common, self-
   describing data format
Support within SonicMQ
Used for LHC logging following String 2
experience
Will probably be used in other domains
(post-mortem, alarms)
Self-describing data, no need to
negotiate between domains, Web
support
    How could LDIWG be
  implemented with SonicMQ
Hierarchical namespace to deal with
“private” domain naming
XML as common, self-describing data
format
Bridge for each domain (SonicMQ
Bridges technology?)
Common administration
   Bridge for each domain
CMW – has been done in the past -
easy
PVSS – has been done (in one
direction)
DIM – easy to do as there are similar
concepts (namespace) and there is
JAVA API
Smartsockets
  SonicMQ Domain Gateway

SonicMQ
 Client                                   CMW
                                          Server

          SonicMQ
           Server         CMW Agent

           Topic               Listener


SonicMQ                                   CMW
 Client
                                          Server




                    Controls
                      DB
            SonicMQ Bridges
Connectivity to Internet services and
messaging systems
 Bi-directional message forwarding
   Between messaging systems
   Across other Internet services
 Transparent to client applications
 Mappings held in XML configuration file
 Administration through SonicMQ
 Common exception handling and logging
                    SonicMQ Bridges
                                       XML
SonicMQ                               Mapping
 Client                  SonicMQ       Files
                          Bridge
          SonicMQ
           Server

           Topic

SonicMQ                  SonicMQ 
 Client                   MQSeries
                          Mapping
                                                   MQSeries
                          MQSeries                 Client
                          SonicMQ       MQSeries
                          Mapping        Broker

                                          Queue

                                                   Mqseries
                                                    Client
                    SonicMQ Bridges
                                       XML
SonicMQ                               Mapping
 Client                  SonicMQ       Files
                          Bridge
          SonicMQ
           Server

           Topic

SonicMQ
 Client


                                                     CMW
                                                     Server
                           CMW
                          SonicMQ
                          Mapping      CMW Agent


                                          Listener

                                                     CMW

                                                     Server
  Reasons to use SonicMQ
Solid industrial product by market leader
Implements standard, hence certain product
independence
Scalable: Start with one broker and expand
later if needed
Inexpensive
Good connectivity (some non-standard)
Fulfills LDIWG requirements and more
Easy to implement LDIWG