PUPPET OPEN SOURCE SYSTEM CONFIGURATION MANAGEMENT

Document Sample
PUPPET OPEN SOURCE SYSTEM CONFIGURATION MANAGEMENT Powered By Docstoc
					PUPPET: OPEN SOURCE
             SYSTEM
     CONFIGURATION
        MANAGEMENT

         Jesús Couto Fandiño
                 09/09/2010




                               1
 The System Configuration Management Problem



Tasks:
• Load software and configure machines to provide the required
  functionality
• Reconfigure machines whenever the required functionality changes.
• Reconfigure machines to maintain functionality whenever the
  environment changes - for example, when things break.
Management:
 How to do the task well (cheaper, faster, without errors…)




                                                                      2
Approaches to the problem


• Manual
 – Syadmins log in and do all the tasks required.
 – Problem: Infrastructure keeps growing (number of machines, software
   packages, interrelationships between parts)
 – Does not scale in cost, time, personnel.
  • Specially if conformance to standards & policies have to be mantained and audited


• Golden images
 – Image of “standard systems” are cloned every time
 – Helps with the setup
 – Leaves evolution of the configuration at the manual level




                                                                                        3
Approaches to the problem (II)


• Scripting
 – “For/SSH” loops through servers executing stored steps to implement a task
 – Can be used with golden images
 – Achieves better speed (even more with parallelism)
 – Difficult and opaque: defines the steps, not the state


• CM tools with declarative languages
 – Specification documents what (state), not how (steps).
 – Tools that read specification and figure the steps to achieve the state
 – Speed & parallelism plus:
   • Configuration = Documentation
   • Abstracting complexity
 – Several Open Source tools: cfengine, BCFG2, LCFG, Puppet


                                                                                4
Puppet


• Developed by Puppet Labs (formerly Reductive Labs), founded by
  Puppet creator, Luke Kaines
• Latest version: 2.6
• Works on Unix systems
• Central concepts:
 – Puppet manages resources (users, packages, files, services …)
 – Resources have a type defining configurable parameters (user => uid, gid,
   name, shell, etc.)
 – Types have providers that know how to implement the configuration.




                                                                               5
Puppet


• Architecture




 – Puppetmaster (server) stores configuration (manifest) for all machines (nodes)
 – Puppet processes running on machines connect to puppetmaster
 – Puppetmaster sends compiled configuration (catalog) for the machine
 – Puppet checks current state against catalog, implements changes




                                                                                    6
Puppet


• Facter
 – Small program on puppet nodes that collects facts about system in variables
 # facter
 architecture => x86_64
 facterversion => 1.5.7
 fqdn => longhair.domain.com
 hardwareisa => x86_64
 hardwaremodel => x86_64
 hostname => longhair
…
 – Variables can be used in manifests & templates, as values or in control
   structures.




                                                                                 7
The Puppet Language


• Resources
 – Have a type, a name and a series of value pairs for the configurable parameters
    file {“/etc/passwd”:
            owner => “root”
    }
 – A resource (type & name) is specified only once per node (can be referenced
   later)
 – 30+ built-in types
 – Each type has 1-N providers implementing it
 – New types & providers can be written & incorporated (Ruby)
 – Metaparameters define relationships between resources:
  • Notify/subscribe: changes in A should make B be re-instantiated again
  • Require/before: A has to be defined and instantiated before B




                                                                                 8
The Puppet Language


• Classes & nodes
 – Class : Collection of resources
 class apache {
          package { httpd: ensure => installed }
          service { “httpd”:
                ensure => running,
                require => Package[”httpd”],
          }
 }
 – Node: configuration for a particular managed machine
 – Classes and nodes can
     • Include other classes (copy it inside)
     • Inherit from another class or node.
 – Included/inherited only once.




                                                          9
The Puppet Language


• Definitions
 – Similar to classes (collection of resources)
 – Similar to types (but written using the Puppet DSL)
 – Accept parameters
 – Can be called several times per node
 define config_file($owner = root, $group = root, $mode = 0644,
  $source, $ensure = file) {
      file { $name:
           mode => $mode,
           owner => $owner,
           group => $group,
           ensure => $ensure,
           source => “puppet:///$source”
      }
   }
  config_file { “/etc/vnc.conf”:
          source => “vnc/vnc.conf”,
          mode => “0640”
 }
                                                                  10
The Puppet Language


• Modules
 – Package of related classes, definitions, files, etc in a portable way that can be
   autoloaded by Puppet
• Fileserver
 – Puppetmaster also acts as a fileserver to store “master” files to be sent to the
   clients
 – File content can be ERB templates that get customized using variables (defined
   on the manifest, Facter “facts”, etc)




                                                                                       11
Modeling a small site with Puppet


• Original idea was to model the setup and tasks of CSC eServices group
  on Dupont account.
• Managing pairs of web/app servers (Apache HTTPd, Apache Tomcat)
• Several instances (separate processes) per machine.


               Web Server 1              Web Server 2
                 Apache-Legal              Apache-Legal

                Apache-Dupont             Apache-Dupont
                    Direct                    Direct
                Apache – Portal           Apache – Portal




                                                                          12
Lessons learned from developing the model


• Similar to programming
 – Low level issues (need to localize per operating system)
 – High level design issues (is the design flexible enough? Reusable? Better way
   to partition the problem into classes, definitions, etc?)
• “Operations as code” mantra: using software engineering for operations:
 – Design, code, test cycles (unit testing, regression testing…)
 – Release management
 – Version control, branches, merges
 – Sharing code between people & organizations (Tomcat module was “lifted” from
   Camptocamp -http://www.camptocamp.com/ )




                                                                                   13
Advantages of using Puppet


• Standarization of results due to automation
• Build and rebuild machines becomes easier and quicker.
• Operations as code
• Manifest + Version Control = Change log and audit trail
• Facts + Manifest = kinda of CMDB?




                                                            14
Limitations of Puppet


• Declarative == static
• Limited network-wide model, or no relationship above node
• Example:
 – “Update application” may be:
   • Change webservers to maintenance
   • Stop application servers
   • Stop database servers
   • Update database, start
   • Update application, start
   • Start webservers.
 – No way to model it in inside Puppet (yet); have to change manifests for all steps
   and wait for/force each to be implemented.




                                                                                  15
Puppet as a layer


• Possible solution: Puppet as base layer, then above a imperative tool
• Example paper: “Web Ops 2.0: Achieving Fully Automated Provisioning”
  (http://dev2ops.org/storage/downloads/FullyAutomatedProvisioning_Whit
  epaper.pdf)
• Puppet is used to ensure machines have correct configuration of required
  services, Control Tier coordinates deploys, stops, starts.
• Another option: mcolletive (http://marionette-collective.org/). Has some
  integration with Puppet
 – Can use facts or included puppet classes to define where to do operations
 – Have plugins that let you “work” with Puppet defined resources (services)




                                                                               16
Since the paper…


• Puppet is now 2.6.
 – Mature enough for 1.0 several years ago
 – Several changes from previous stable version
   • From XML-RPC to REST
   • Single binary with different run options
   • Language changes
    – Classes now accept parameters
    – Relationships between resources can be defined outside them
    – Add “run stages”
    – Variables can now contain hashes
    – Can write manifest directly in Ruby
   • Many more…




                                                                    17
Since the paper (II)


• Puppet Forge is now open (http://forge.puppetlabs.com/)
 – Central place to share and find modules
 – puppet-module tool lets you quickly install community modules, or share your
   modules.


• PuppetCamp Europe 2010
 – Held in Ghent, Belgium, May 27&28th
 – Puppet is being used in several critical enviroments
   • Not always in the “standard” configuration – adapting to different business requirements
 – Next PuppetCamp on San Francisco, Oct 7&8th




                                                                                           18
19

				
DOCUMENT INFO