SMF Service Management Facility

Document Sample
SMF Service Management Facility Powered By Docstoc
					SMF: Service Management

Liane Praza
Sun Microsystems
• SMF basics
• SMF service development

                  Sun Proprietary/Confidential: Internal Use Only
SMF basics: svc.startd
• A new system daemon, svc.startd, has taken over most
  of init's responsibilities in starting system services
• init still uses inittab, and /etc/rc?.d scripts are still run
• svc.startd can automatically restart services
  > If sendmail is enabled, then it is
     > started at boot
     > restarted if it dies (even if killed)
  > sendmail      may be disabled by a single command
      > stopped
      > not started at boot
      > not started after patch or upgrade

                          Sun Proprietary/Confidential: Internal Use Only
Service states
• SMF keeps a state for each service
  > uninitialized    has not been evaluated yet
  > disabled         service is disabled, not running
  > offline          enabled, waiting for dependencies
  > online           enabled and running
  > degraded         running below full performance
  > maintenance      service problem occurred

                    Sun Proprietary/Confidential: Internal Use Only
Service dependencies
• Services may declare dependencies on each other
• svc.startd starts services in dependency order
   > Parallel service startup → faster boot
• If a service encounters a hardware error, only the service
  and services which depend on it are restarted (not the
  whole system)
• If a service doesn't start for any reason, services with hard
  dependencies on it will stay in the offline state and won't
  be started
   > A new command answers “What services is service X
      waiting for?”
                       Sun Proprietary/Confidential: Internal Use Only
Service methods
• svc.startd uses a service's methods to manipulate it
  > start & stop methods start & stop service
  > refresh method (optional) instructs service to reread
• Methods can be scripts, binaries, or keywords:
  > :true to do nothing
  > :kill [-sig] to send a signal to the service's processes

• Methods can specify context
  > user, group, privileges
  > resource management settings

                       Sun Proprietary/Confidential: Internal Use Only
SMF configuration
• Service meta-configuration
  (enabled status, state,                                          svc.startd

  dependencies, methods, etc.) is
  kept in the repository                                                            SMF tools

• The repository is controlled by
  svc.configd, another new
• The repository is (currently)
  stored in
  /etc/svc/repository.db                                                  repository.db

                        Sun Proprietary/Confidential: Internal Use Only
Service instances
• Services are represented as
  instance nodes which are         repository
  children of service nodes
• Both service nodes and instance       service     properties
  nodes can have properties
• If an instance doesn't have              instance    properties
  property X, the service's
  property X is used                       instance    properties
• Allows similar services to share
   > For example, two copies of
      the same webserver on
      different ports                   service

                         Sun Proprietary/Confidential: Internal Use Only
Service names: FMRIs
• Services are named by Fault Management Resource
  Identifiers, or FMRIs
  > URI syntax
        service name                                                     instance name
• Note that while the service name usually contains slashes,
  there are no service directories! The namespace is flat.
• Commands accept abbreviations (system/cron, cron)
  and glob patterns

                       Sun Proprietary/Confidential: Internal Use Only
Property structure
• Properties have a type (string, service
  integer, etc.) and zero or more
  values                             property group
• Properties are grouped in               property  value
  named property groups, which                       value
  have a type tag (e.g.,
  “framework”, “dependency”)              property  value
• If a service's
  general/enabled property           property group
  has a single true value, then
  the service is enabled                  property  value

                                                                     property   value
                        Sun Proprietary/Confidential: Internal Use Only
Property snapshots
• A snapshot of an instance is a
  copy of its properties and the                       instance                    property group
  properties of its service node
• Snapshots are automatically
  taken at key times                                                    instance    property group
  > After successful start
  > Before upgrade                                                       service
                                                                                    property group
• A service's properties can be
  rolled back to a snapshot                                     snapshot

• SMF components usually use                                    snapshot
  the “running” snapshot so
  changes can be committed by
  updating that snapshot
                             Sun Proprietary/Confidential: Internal Use Only
Repository use and access
• Services may store their own configuration in the repository
  > Avoids writing a parser
  > Benefits from snapshots
  > Use libscf(3lib) or svcprop(1)
  > Example: rpc/bind's config/verbose_logging property
• All users may read repository properties
• Write privileges may be delegated via RBAC authorizations
  > Can delegate administration (start, stop, etc.) of
     individual services
  > See smf_security(5)

                       Sun Proprietary/Confidential: Internal Use Only
Service manifests
• A service manifest is an XML file which describes
  dependencies, methods, service-specific properties
• Manifests are delivered into /var/svc/manifest
• During startup, new manifests in /var/svc/manifest
  and old manifests which have changed are loaded into
  the repository with the svccfg(1M) command
• Manifests in /var/svc/manifest should not be
  customized; customizations should be made with
  svccfg(1M), etc., and they will be preserved across
  patch & upgrade
                    Sun Proprietary/Confidential: Internal Use Only
Commands: svcs(1)
• Without arguments, lists state, state-time, and FMRI of
  services that are enabled; with -a, lists all services
• -x explains errors & states
• Show dependencies (-d) and dependents (-D)
• Show member processes (-p), additional details (-v/-l)
            $ svcs
            STATE          STIME            FMRI
            online         18:18:30         svc:/network/http:apache2
            online         18:18:29         svc:/network/smtp:sendmail
            $ svcs -p sendmail
            STATE          STIME            FMRI
            online         18:18:29         svc:/network/smtp:sendmail
                           18:18:29           100180 sendmail
                           18:18:29           100181 sendmail
            $ svcs -d sendmail
            STATE          STIME    FMRI
            online         18:17:44 svc:/system/identity:domain
            online         18:17:52 svc:/network/service:default

                          Sun Proprietary/Confidential: Internal Use Only
Commands: svcadm(1M)
• svcadm manipulates services
  > svcadm enable    enables services, services start when
      dependencies are ready
  >   svcadm disable disables services
  >   svcadm restart stops and starts services
  >   svcadm refresh commits the current properties to the running
      snapshot and runs the refresh method
  >   svcadm clear signals that a service in maintenance has been fixed

• These commands are asynchronous: they issue commands to
  svc.startd and return immediately
• With -s, enable & disable wait until completion
• With -t, enable & disable are temporary until next boot

                          Sun Proprietary/Confidential: Internal Use Only
Commands: svccfg(1M)
• Interactive access to properties and snapshots
• svccfg import creates services from a service
   # svccfg
   svc:> select network/http:apache2
   svc:/network/http:apache2> listprop
   general                  framework
   general/enabled          boolean false
   start                    method
   start/exec               astring "/lib/svc/method/http-apache2 start"
   start/timeout_seconds    count    60
   start/type               astring method
   svc:/network/http:apache> editprop
   [ $EDITOR launches on a temporary file containing property settings ]
   svc:/network/http:apache2> exit
   # svcadm refresh apache2         # read latest configuration
   # svcadm restart apache2         # restart with latest configuration

                            Sun Proprietary/Confidential: Internal Use Only
Commands: svcprop(1)
• List properties of services and instances
• Fetch individual properties for use in scripts
       $ svcprop network/http:apache2
       physical/entities fmri svc:/network/physical:default
       physical/grouping astring optional_all
       physical/restart_on astring error
       physical/type astring service
       start/exec astring /lib/svc/method/http-apache2\ start
       start/timeout_seconds count 60
       start/type astring method
       stop/exec astring /lib/svc/method/http-apache2\ stop
       stop/timeout_seconds count 60
       stop/type astring method
       restarter/auxiliary_state astring none
       restarter/next_state astring none
       restarter/state astring disabled
       restarter/state_timestamp time 1102030556.737590000
       $ svcprop -p enabled network/http:apache2

                         Sun Proprietary/Confidential: Internal Use Only
Delegated Restarters
• Not all services fit svc.startd's service model
• SMF allows a service to be a delegated restarter for other
  > Start, stop, and restart services
  > Set service states
  > svc.startd    still handles enabledness & dependencies, though
• inetd is currently the only delegated restarter
  > Methods are called inetd_start, inetd_stop, etc.
  > Services come online when inetd starts listening for them
  > The repository is used for configuration instead of inetd.conf
     (more on next slide)
• More delegated restarters will come
                            Sun Proprietary/Confidential: Internal Use Only
/etc/inetd.conf & inetadm(1M)
• inetd.conf is no longer the primary configuration
• Most Solaris inet services have been converted
• Entries in inetd.conf are automatically converted
  during install & upgrade by inetconv(1M)
• inetd(1M) will issue a warning message if it discovers an
  unconverted entry in /etc/inetd.conf
• Administrators can run inetconv again at any time
• inetadm(1M) can be used to modify inetd-specific

                      Sun Proprietary/Confidential: Internal Use Only
• SMF basics
• SMF service development

                  Sun Proprietary/Confidential: Internal Use Only
Service Development: Benefits
• Services appear with SMF FMRIs
  > Visible using standard Solaris tools; your service
    appears in administrative heads-up displays
  > Manageable using standard Solaris tools; admin can
    leverage existing knowledge to use your service
  > New generic tools developed will automatically see your
• Built-in restart due to administrative error, software,
  or hardware fault
• Participation in future software diagnosis
                       Sun Proprietary/Confidential: Internal Use Only
Service Development: Tasks
• An existing Solaris service may be converted
  incrementally, and to different levels
  > Get it working: write a manifest with existing init script
  > Handle error cases: refine methods (init script)
  > Full restartability: If service has multiple processes, split
    them into individual services
  > Customized error/restart handling: avoid service restart if
    fault can be handled internally

                        Sun Proprietary/Confidential: Internal Use Only
Manifest Creation
•   Name your service
•   Identify whether your service may have multiple instances
•   Identify how your service is started/stopped
•   Determine faults to be ignored
•   Identify dependencies
•   Identify dependents
•   Create, if appropriate, a default instance
•   Create template information to describe your service

                        Sun Proprietary/Confidential: Internal Use Only
Example Manifest: utmpd(1M)
   <service name='system/utmp' type='service' version='1'>
       <create_default_instance enabled='true' />
       <single_instance />
       <dependency name='milestone' grouping='require_all'
           restart_on='none' type='service'>
               <service_fmri value='svc:/milestone/sysconfig'/>
       <dependent name='utmpd_multi-user' grouping='optional_all'
               <service_fmri value='svc:/milestone/multi-user'/>
      <exec_method type='method' name='start'
          exec='/lib/svc/method/svc-utmpd' timeout='60' />
      <exec_method type='method' name='stop'
          exec=':kill' timeout='60' />
      <stability value='Unstable' />
              <common_name><loctext xml:lang='C'>
                      utmpx monitoring

                       <manpage title='utmpd' section='1M'
                           manpath='/usr/share/man' />

                         Sun Proprietary/Confidential: Internal Use Only
Method refinement
• On failure, explain the problem to stdout or
  stderr (goes to a log) and exit with a non-0 code
  > If the failure is not transient, return
    from /lib/svc/share/
• On success, don't return until service is ready to
  serve clients
  > Dependent services may be started immediately
• If specified, the refresh method must not cause the
  service to exit

                         Sun Proprietary/Confidential: Internal Use Only
Development: Other Examples
• Manifest DTD is well-documented; read it at
• Explore /var/svc/manifest for similar services
  > system/utmp  is a simple standalone daemon
  >   system/coreadm is a simple configuration service
  >   network/telnet is an inet-managed daemon
• Explore /lib/svc/method for similar methods

                       Sun Proprietary/Confidential: Internal Use Only
• Service failures printed to console, syslog
• Start with svcs -x output, any articles it
• svcs -x will refer to a log file in either:
  > /var/svc/log
  > /etc/svc/volatile

• Use svcadm clear to clear failed services in the maintenance
• Use svccfg to tweak debugging variables:
  > svccfg -s system/foo setenv LD_PRELOAD
  > svccfg -s system/foo setenv UMEM_DEBUG default

                      Sun Proprietary/Confidential: Internal Use Only
• If a single service is broken, make sure you've got the latest
  service config; svcadm refresh <fmri>
• Follow instructions from svcs -x pointer
• Revert to a previous snapshot.
            $ svccfg -s system/cron:default
            svc:/system/cron:default> listsnap
            svc:/system/cron:default> revert start
            svc:/system/cron:default> exit
            $ svcadm refresh cron
            $ svcadm restart cron

                        Sun Proprietary/Confidential: Internal Use Only
Service Packaging
• Use i.manfest and r.manifest from a Sun-
  delivered package (e.g. SUNWcsr)
• Manifests delivered into /var/svc/manifest with
  type “f” and class “manifest”
  > Use /var/svc/manifest/site if the service is
    specific to your site
  > Use another directory if you're an ISV, but remember a
• Methods delivered with your application binaries
  (/opt strongly recommended)
                      Sun Proprietary/Confidential: Internal Use Only
Sun Service Refinements
• Service dependencies reduce complex install-time
  logic and eliminate start-time checks
• Using SMF allows participation in Solaris
  innovations: FMA, resource management, security
• Configuration migration unifies Solaris config mgmt
• Service names must go to the appropriate ARC
• Send manifests and methods to for comment

                    Sun Proprietary/Confidential: Internal Use Only
Developer References
• Manifest development
  > /usr/share/lib/xml/dtd/service_bundle.dtd.1

  > Look in /var/svc/manifest for examples
  > inetconv -i file   to create an empty inetd manifest
  > smf_method(5) – information for writing methods

  > smf_restarter(5) – information for writing restarters

  > Service Developer Introduction available at

                     Sun Proprietary/Confidential: Internal Use Only
Additional Resources
• Discussion and further information at
• Additional quickstart and developer documentation
  available at
• Solaris System Administration Guide has SMF
• smf(5) manpage introduces the facility
• Blogs:
                      Sun Proprietary/Confidential: Internal Use Only
Thank you!

Liane Praza
Sun Microsystems