Automating lighttpd installations with Puppet – Part 3

If you’ve followed through the two earlier parts found here and here, you should now have a basic installation of lighttpd installed on your node, together with a nice small Puppet module that you’ve created yourself on your Puppet master.

Let’s see if we can make that module a bit cleaner, shall we?

First, let’s make sure we have all the necessary files. It should look like this in the /etc/puppet/modules/ directory:

lighttpd/files/index.html
lighttpd/files/lighttpd.conf
lighttpd/manifests/init.pp
lighttpd/manifests/install.pp
lighttpd/manifests/indexfile.pp
lighttpd/manifests/service.pp
lighttpd/manifests/config.pp

The reason we’re doing this is because we’d like to break out the different parts of the installation process into it’s own file, for easier management. Instead of one big file (not so big for us right now, but for other services it can grow quite large), we’ll have smaller files that are more targeted to what they should be doing.

Let’s start with the install.pp file, it should look like this:

class lighttpd::install {
    package { 'lighttpd':
        ensure => installed
    }
}

But wait, you might say. Isn’t that exactly what we already had in our old init.pp file? You are correct! However, we’ve added “::install” to the name of the class, a naming convention that gives us easier inclusion later on when we’ll put all this together. This shows us that this class belongs to the main class lighttpd, and it will handle the installation of the package that we have defined. Defining these parts of a class make it easier for us to edit just a small part of a large class, instead of having to crawl through hundreds or thousands of lines of code in one file. The name that comes after the “::” must always correspond to the filename itself, like here; ::install = install.pp.

Easy module management, check!

Now let’s look at the service.pp file:

class lighttpd::service {
    service { 'lighttpd':
        ensure => running,
        enable => true,
        require => Class["lighttpd::install"]
    }
}

I think you can see a pattern evolving here, us making simple sub-classes instead of one big one. Also, you can see that we’ve changed the require statement from being a package to being a class. This means that the class lighttpd::install will always be verified first before moving on to this class.

Let’s look at the next one, indexfile.pp:

class lighttpd::indexfile {
    file { "/var/www/index.html":
        ensure => present,
        owner => 'root',
        group => 'root',
        mode => 0644,
        source => "puppet:///modules/lighttpd/index.html",
        require => Class["lighttpd::install"]
    }
}

For the lighttpd/files/lighttpd.conf file, you can use the following mass of text (standard config file for lighttpd for Ubuntu 12.04, dated september 2012):

server.modules = (
 "mod_access",
 "mod_alias",
 "mod_compress",
 "mod_redirect",
 # "mod_rewrite",
 )

server.document-root = "/var/www"
 server.upload-dirs = ( "/var/cache/lighttpd/uploads" )
 server.errorlog = "/var/log/lighttpd/error.log"
 server.pid-file = "/var/run/lighttpd.pid"
 server.username = "www-data"
 server.groupname = "www-data"

index-file.names = ( "index.php", "index.html",
 "index.htm", "default.htm",
 " index.lighttpd.html" )

url.access-deny = ( "~", ".inc" )

static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )

## Use ipv6 if available
 #include_shell "/usr/share/lighttpd/use-ipv6.pl"

dir-listing.encoding = "utf-8"
 server.dir-listing = "enable"

compress.cache-dir = "/var/cache/lighttpd/compress/"
 compress.filetype = ( "application/x-javascript", "text/css", "text/html", "text/plain" )

include_shell "/usr/share/lighttpd/create-mime.assign.pl"
 include_shell "/usr/share/lighttpd/include-conf-enabled.pl"

Let’s make the config file installation/verification into a sub-class as well. Here’s config.pp:

class lighttpd::config {
    file { "/etc/lighttpd/lighttpd.conf":
        ensure => present,
        owner => 'root',
        group => 'root',
        mode => 0644,
        source => "puppet:///modules/lighttpd/lighttpd.conf",
        require => Class["lighttpd::install"],
        notify => Class["lighttpd::service"]
    } 
}

So pretty much the same thing here, we’re requiring the lighttpd::install class. But we’re also adding something else, a metaparameter (like require) called “notify”. The notify metaparameter will create a notification relationship. This relationship makes it possible for the lighttpd::config class to call onto the lighttpd::service class if something is changed, like a configuration file content change for instance. If the config file is changed, the service should be restarted, and that is accomplished automatically by using this notify metaparameter. Nifty!

Another thing that we get from this is maybe not as visible, but equally if not even more important. By controlling the configuration files from the Puppet master instead of the nodes, we get centralized configuration management. This can easily be built to use Git or other versioning tools as well. Hmm, perhaps another blog post. But the really cool thing about this is the concept of self-healing. If someone, by mistake, changes the configuration file on one of the Puppet nodes then it will automatically be set straight during the next Puppet agent cycle. Now THAT is cool!

So what have we done here? We’ve broken out the installation, the configuration, the service management, the config file, and the standard index file into separate entities, each manageable using small independent files. No massive monoliths of config files to go through, perfect!

So now that we’ve created these, should our node definition include all these classes? Should it include lighttpd::install, then lighttpd::config etc? Heck no! Let’s take a look at what your init.pp file should look like from now on:

class lighttpd {
 include lighttpd::install, lighttpd::config, lighttpd::service, lighttpd::indexfile
}

There we have it. We’ve included all these new sub-classes into the main class, which is already defined on our node so change needed there at all:

node "35553b90abfc012f4380000c293a4170" inherits default {
 include lighttpd
}

So we rearranged an entire class into multiple different files, sorted them with require and notify metaparameters, should we see a change on the node? Let’s find out!

root@node9:~# puppet agent --test --verbose
 info: Retrieving plugin
 info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
 info: Loading facts in /var/lib/puppet/lib/facter/iptables.rb
 info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb
 info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
 info: Caching catalog for 35553b90abfc012f4380000c293a4170
 info: Applying configuration version '1349211009'
 notice: Finished catalog run in 0.75 seconds

What? Nothing? Damn we’re good! That means we did everything by the book, and we’re done configuring that module. But wait! Don’t you want to see what happens if you change the config file just a small bit? Let’s remove the # in front of ”mod_rewrite” in lighttpd.conf, and rerun the puppet agent:

root@node9:~# puppet agent --test --verbose
<snip>
info: Applying configuration version '1349211009'
notice: /Stage[main]/Lighttpd::Config/File[/etc/lighttpd/lighttpd.conf]/content:
--- /etc/lighttpd/lighttpd.conf 2012-10-02 22:56:56.382446433 +0200
+++ /tmp/puppet-file20121002-23240-11rgkxv-0 2012-10-02 22:59:21.450141252 +0200
@@ -3,7 +3,7 @@
 "mod_alias",
 "mod_compress",
 "mod_redirect",
-# "mod_rewrite",
+ "mod_rewrite",
 )

server.document-root = "/var/www"
info: FileBucket adding {md5}615b478c61c85222bcebe9a0a06eb342
info: /Stage[main]/Lighttpd::Config/File[/etc/lighttpd/lighttpd.conf]: Filebucketed /etc/lighttpd/lighttpd.conf to puppet with sum 615b478c61c85222bcebe9a0a06eb342
notice: /Stage[main]/Lighttpd::Config/File[/etc/lighttpd/lighttpd.conf]/content: content changed '{md5}615b478c61c85222bcebe9a0a06eb342' to '{md5}c596a1986a2534025b160458fcd106a5'
info: /Stage[main]/Lighttpd::Config/File[/etc/lighttpd/lighttpd.conf]: Scheduling refresh of Class[Lighttpd::Service]
info: Class[Lighttpd::Service]: Scheduling refresh of Service[lighttpd]
notice: /Stage[main]/Lighttpd::Service/Service[lighttpd]: Triggered 'refresh' from 1 events
notice: Finished catalog run in 1.92 seconds

How cool is that! You get a changelog of the changes being done on your config file, and you also get a notice on the md5sums of the old vs new file. Plus you get a restarted lighttpd service which means that the config file will be in effect immediately.

Well done, you can now pat yourself on the back and enjoy the fruits of your labour, a complete (albeit lacking a bit of documentation, but I’ll leave that up to you, the reader) Puppet module for an automated installation of lighttpd.