The time has come to move further in our Automation series, now that we’re pretty much done with the “physical” provisioning part using Razor.
One of the first things I’d like to share with you is how to get lighttpd up and running on a newly provisioned server, and make the deployment process fully automated using Puppet from Puppetlabs.
First though, a few notes:
- Creating something that automates something else is not an automated process in itself (Quantum computing and smart AI might change that around year 2030). So while the configuration below might look complex to do a simple task, remember that you can reuse the same configuration for many nodes. One configuration for Many installations instead of Many installations with Many (often different) configurations.
- I am using the free and open source verion of Puppet, which means we do not have a dashboard for even easier configuration. Might do a post about that later on.
Last but not least, Puppet focuses on WHAT needs to be done, not HOW. Do your customers care HOW a service gets installed and gets it’s configuration, or do they just care WHAT is installed? Thought so 
Now that that’s out of the way, let’s get started!
For this exercise, we’ll install the lighttpd webserver (awesome httpd alternative to the bulky Apache IMHO) onto one of our newly provisioned nodes. As Razor during OS-provisioning will be giving the node a specific node name which will then be used in the handoff to the puppetmaster, this is the name we’ll be using throughout the tutorial. If you have a smaller test environment, or not using Razor at all, you could also give it a more userfriendly name, say “webserver01″, by just editing the puppet.conf file on the node itself.
The node I’ll be configuring is this one:
root@puppet:~# puppet cert list --all + "35553b90abfc012f4380000c293a4170" (37:13:86:7B:5B:22:BD:B9:91:58:AE:EF:C7:A0:D6:89)
It has a “fancy” node ID called “35553b90abfc012f4380000c293a4170″ . Remember that until a bit later.
When creating an automatic configuration for a puppet node, we need to go through a few stages. First of all, stop the puppet agent service on your node, as we’ll run puppet agent manually.
root@node9:~# /etc/init.d/puppet stop * Stopping puppet agent
Now, there are 4 steps that should always be taken when creating a configuration; Define, Simulate, Enforce and Report.
1. Define
We need to define a configuration that includes everything that the node should do. This could be installing a package, ensure a file exists, or perhaps starting a service. We can do this by editing the site.pp file (which is seen as the common way), or do this by editing the /etc/puppet/manifests/nodes.pp file, just like we did here. I’ll add it to my nodes.pp file as we’re only applying this to one node at the moment. We’ll look at the site.pp file in another post:
#This was added as part of of a previous lab, to ensure that NTP
#was installed and set correctly
node default {
class { 'timezone':
timezone => 'CET',
}
include ntp
}
#Our node inherits the base NTP configuration (inherits default)
#that all nodes should have, and then we add the service
#we want on it
node "35553b90abfc012f4380000c293a4170" inherits default {
package {
'lighttpd':
ensure => installed
}
service {
'lighttpd':
ensure => running,
enable => true,
require => Package['lighttpd']
}
}
Let’s go through that config, bit by bit.
“Package” means we’ll handle an installation package. This usually has a function such as installing a service (like a web server or SQL server) or some functionality (PHP for instance) or a software (like zsh, an alternative to bash). Here we “ensure” that the package “lighttpd” is installed.
“Service” means we’ll make sure we have a service tied to this configuration. We ensure the service is running, we’re enabling it on boot, and for this service to function we require the “Package” that we’ve defined.
Pretty easy to read and follow, right?
2 & 3: Simulate & Enforce
Let’s simulate applying this configuration on our node. Running the puppet agent in “noop” mode means that no changes will be made:
root@node9:~# puppet agent --noop --test info: Retrieving plugin info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb info: Loading facts in /var/lib/puppet/lib/facter/iptables.rb info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb info: Caching catalog for 35553b90abfc012f4380000c293a4170 info: Applying configuration version '1344867861' notice: /Stage[main]//Node[35553b90abfc012f4380000c293a4170]/Package[lighttpd]/ensure: current_value purged, should be present (noop) err: /Stage[main]//Node[35553b90abfc012f4380000c293a4170]/Service[lighttpd]: Could not evaluate: Could not find init script or upstart conf file for 'lighttpd' notice: Node[35553b90abfc012f4380000c293a4170]: Would have triggered 'refresh' from 1 events notice: Class[Main]: Would have triggered 'refresh' from 1 events notice: Stage[main]: Would have triggered 'refresh' from 1 events notice: Finished catalog run in 0.27 seconds
We see that it would set the package value for lighttpd to present (by choosing the package and making sure it gets installed properly), and it fails on making sure the service is running as the software isn’t installed. Looks good so far, let’s run the agent again but without the “noop” mode to enforce the changes to be made:
root@node9:~# puppet agent --test --verbose info: Retrieving plugin info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb info: Loading facts in /var/lib/puppet/lib/facter/iptables.rb info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb info: Caching catalog for 35553b90abfc012f4380000c293a4170 info: Applying configuration version '1344867861' notice: /Stage[main]//Node[35553b90abfc012f4380000c293a4170]/Package[lighttpd]/ensure: ensure changed 'purged' to 'present' notice: Finished catalog run in 6.67 seconds
We can now see that the configuration has been run and successfully completed, let’s see if lighttpd is running:
root@node9:~# ps ax | grep light 8635 ? S 0:00 /usr/sbin/lighttpd -f /etc/lighttpd/lighttpd.conf
It is, HOORAY! How cool is that?
You should now be able to browse to the IP address of that webserver and see something similar to this:
Before we go into configuring this service, I want you to know how to remove the package and service as well. To do this, edit your your nodes.pp file into this:
node "35553b90abfc012f4380000c293a4170" inherits default {
package {
'lighttpd':
ensure => purged
}
service {
'lighttpd':
ensure => stopped,
enable => false,
require => Package['lighttpd']
}
}
Run the puppet agent in “noop” mode on the node:
root@node9:~# puppet agent --noop --test info: Retrieving plugin info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb info: Loading facts in /var/lib/puppet/lib/facter/iptables.rb info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb info: Caching catalog for 35553b90abfc012f4380000c293a4170 info: Applying configuration version '1344869159' notice: /Stage[main]//Node[35553b90abfc012f4380000c293a4170]/Package[lighttpd]/ensure: current_value 1.4.28-2ubuntu4, should be purged (noop) notice: /Stage[main]//Node[35553b90abfc012f4380000c293a4170]/Service[lighttpd]/ensure: current_value running, should be stopped (noop) notice: Node[35553b90abfc012f4380000c293a4170]: Would have triggered 'refresh' from 2 events notice: Class[Main]: Would have triggered 'refresh' from 1 events notice: Stage[main]: Would have triggered 'refresh' from 1 events notice: Finished catalog run in 0.40 seconds
You can see that the package will be purged (both binaries and configuration files removed). If you want, you can apply the new settings:
root@node9:~# puppet agent --test --verbose info: Retrieving plugin info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb info: Loading facts in /var/lib/puppet/lib/facter/iptables.rb info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb info: Caching catalog for 35553b90abfc012f4380000c293a4170 info: Applying configuration version '1344869159' notice: /Stage[main]//Node[35553b90abfc012f4380000c293a4170]/Package[lighttpd]/ensure: ensure changed '1.4.28-2ubuntu4' to 'purged' err: /Stage[main]//Node[35553b90abfc012f4380000c293a4170]/Service[lighttpd]: Could not evaluate: Could not find init script or upstart conf file for 'lighttpd' notice: Finished catalog run in 3.37 seconds
And we can now verify that the package is removed, the service isn’t running, and no config files are left.
Revert back your changes so that the package IS installed and the service IS running. Let’s continue.
4. Report
Puppet does have some nifty reporting tools, one of the foremost tools is of course the dashboard which we won’t use right now…
For now I just want to show you one easy way of getting the important information into your puppetmaster’s logs instead of having to go through each agent’s logs.
First, edit your /etc/default/puppet file on your node to include the “–report” option (which will make it automatically report everything back to the master everytime the puppet agent service is started) like this:
# Defaults for puppet - sourced by /etc/init.d/puppet # Start puppet on boot? START=yes # Startup options DAEMON_OPTS="--report"
Then, on your puppetmaster, change your puppet.conf file to include the following:
reports=log
Restart your puppetmaster service:
root@puppet:~# /etc/init.d/puppetmaster restart * Restarting puppet master [ OK ]
Now let’s do a small change in our configuration, change the lighttpd from “running” to “stopped” in the /etc/puppet/manifests/nodes.pp file:
service {
'lighttpd':
ensure => stopped,
enable => true,
require => Package['lighttpd']
}
Now, run the puppet agent on your node:
root@node9:~# puppet agent --test --verbose --report info: Retrieving plugin info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb info: Loading facts in /var/lib/puppet/lib/facter/iptables.rb info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb info: Caching catalog for 35553b90abfc012f4380000c293a4170 info: Applying configuration version '1344870208' notice: /Stage[main]//Node[35553b90abfc012f4380000c293a4170]/Service[lighttpd]/ensure: ensure changed 'running' to 'stopped' notice: Finished catalog run in 0.48 seconds
We see that the service is stopped. Perfect. Let’s have a look at the syslog on the puppetmaster:
root@puppet:~# tail /var/log/syslog Aug 13 17:03:28 puppet puppet-master[2040]: Compiled catalog for 35553b90abfc012f4380000c293a4170 in environment production in 0.07 seconds Aug 13 17:03:29 puppet puppet-master[2040]: (//35553b90abfc012f4380000c293a4170//Stage[main]//Node[35553b90abfc012f4380000c293a4170]/Service[lighttpd]/ensure) ensure changed 'running' to 'stopped' Aug 13 17:03:29 puppet puppet-master[2040]: (//35553b90abfc012f4380000c293a4170/Puppet) Finished catalog run in 0.48 seconds
AWESOME! Now we can see the configuration changes being made on each node, by just looking at the syslog file on our master. That’s a great start
Now let’s look at what you’ve been doing during this post, in another easy-to-understand picture from Puppetlabs:
The puppet agent send “facts” about itself to the puppetmaster, the puppetmaster answers with the “catalog” which is a compiled list of all aggregated configuration changes that should be done, the puppet agent reports back to the puppetmaster and we can see that report by looking into the syslog file.
Revert the configuration back once again so that the service IS running, and you’ve come to the end of this post. Next time we’ll add a few files to the service, and perhaps do some more cool stuff
