Audience: System Admin, Self taught coder, Computer Scientist
Summary: How to leverage puppet-retrospec to generate your puppet unit test suite
Time Involved: 10 minutes
How many of you remember when the spell checker first came out? It had such a huge impact in everyones life because instantly everyone who used the spell checker appeared as if they had won the international spelling bee. How long did it take you to switch from looking up every single word in a paper dictionary to simply right clicking on a word? Assuming you grew up in the 90’s, we had to use books to spell check, crazy isn’t it. Spell checker turned horrible spellers into artisan letter assemblers overnight. With that said, I consider the spell checker to be the first form of unit testing because the spell checker tested every word of your summer vacation essay against a simple algorithm to ensure the word passed a basic spelling test.
So what does spell checking have to do with configuration management tools like Chef and Puppet? Basically, after you learn some basic CF (Configuration Management) programming you will need to start testing your own code, just like the spell checker. If your new to development which includes anybody using configuration management, unit testing your CF code brings a spotlight to bugs without having to pay much attention. This is especially important because a sysadmin’s workday is bombarded with distractions, especially from your trusty feline sidekick. So lets move forward and review some basic automated testing principles.
When it comes to testing there are two types one must test. You might have seen these before but these types are unit and acceptance/integration testing.
The unit test performs a very simple test to ensure that you do not have syntax errors and your conditional logic works against a set of supplied assumptions. Does your conditional statements work as intended? Do your variables interpolate correctly? Do the functions you use, perform the magic you expected? These are the things you should be asking yourself when building unit tests.
Writing your first unit test
With regards to configuration management code we will first test against compile time bugs because that is where the bulk of your mistakes will be caught and its also the fastest.
Part of the problem with new developers and lack of test code is the amount of time taken away from writing the “real” code. There are plenty of articles on how to write test code, but its often difficult to find something that caters to the absolute beginner and just getting the code setup to run tests is often too much. So questions like, whats a helper?, whats rspec?, mocking, mocha, shoulda, doubles, fixtures, unit test, integration test, a/b test? The problem is that you already don’t know what your doing and to make matters worse, now you have to learn new terminology to test the code that you barely know how to write. The mountain of knowledge needed to run a basic test serves as a barrier to becoming a better programmer and is often what separates a junior from a senior level programmer. Testing should be just as easy as writing the code itself. While I cannot change the fact that good testing practices is a skill in itself. I can at least automate some basic testing patterns and remove the immediate barrier to becoming a better configuration management programmer.
With this in mind I would like to introduce Retrospec. Retrospec is a tool that will generate puppet rspec test code based on the code inside the manifests directory. Its sole purpose is to get your module setup with automated testing, stat! Additionally, Retrospec will actually write some basic test code for you. Yes, you heard me right, it will write tests for you, even while you sip beer. While the generated tests are only basic checks it lays the groundwork for you to create even more advanced test cases and increase your BeerOps time. So lets get started.
The first thing you need to do is install retrospec
gem install puppet-retrospec
We will use a puppet module I wrote for non-root environments as an example. Go ahead and clone this repo so you can follow along.
Now we just need to do some house cleaning to show how retrospec generates files easily. Obviously, you won’t do this in your own module, unless you really want to.
You can run retrospec -h by itself to see what options you have.
1 2 3 4 5 6 7 8 9 10 11 12
Now that we have a module without tests we can use Retrospec to retrofit the module with some basic tests. Many of these basic files are needed for the puppetlabs_spec_helper and rspec-puppet.
Note: If you have set your bundle path to be something other than GEM_HOME you will need to install retrospec in that path.
`bundle exec gem install puppet-retrospec`, otherwise you will get an error.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Bam! Wasn’t that easy?
Now Retrospec isn’t perfect but it did save us several hours of BeerOps time for us by generating all these files. You
should see something similar like below when you run
cat spec/classes/nonrootlib_spec.rb. This gives us a easy place
to start. Keep following and I’ll show you some basic testing patterns to fix some of this generated code. Retrospec
only generates files that you do not already have as noted by the
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
Once you retrospec your module many tests are generated but need to be prepped for testing. Note the tests will fail until you refactor the test code. Since testing puppet code relies heavily on other gems we need to use bundler to download all these dependencies.
cd puppet-nonrootlib# if not already in the directory
gem install bundler# unless bundler is already installed
bundle install# installs all the gems necessary for puppet unit testing ( You should be in the module directory)
bundle exec rake spec_prep# sets up fixtures, not necessary if using rake spec
bundle exec rspec spec/classes/nonrootlib_spec.rb# run your test against a single test file
Normally you would run
bundle exec rake spec but I wanted to just run a single test file so I used rspec directly.
Lets go ahead and open the spec/classes/nonrootlib_spec.rb file and refactor the test code, because out of the box, these tests will fail.
Below is an example of how you mock facts in a rspec-puppet testing environment. I refactored spec/classes/nonrootlib_spec.rb to work by specifying the facts to mock like home_dir and id. Note, these are just mocks so directories and users don’t actually have to exist. I only need to mock the facts that are used in my manfiest code. Go ahead and update your spec/classes/nonrootlib_spec.rb file to match the facts block below.
1 2 3
Note: You can only mock hiera values, facts, params, and functions with rspec-puppet. This is all you really need to influence conditional logic in your code as your will be testing against the catalog. I am only covering facts and params in this article since the other items are considered an advanced topic. Plus your attention span can’t handle much more anyways.
Below is a real example of how you can mock parameters to incluence your conditional logic. It follows the same syntax as mocking facts. You may have noticed that retrospec comments out any parameters with default values. However, I specified each parameter value statically as I consider specifying the parameter values better for long term test maintainability. If a future developer changes the default parameter values in your manifest code some of these test will break, so its good practice to set them in stone here. But your not required to do this which is why they are commented out in the initial test generation. This is where you can mock parameter values and test against different scenarios. Its worth noting that without Retrospec you would had to specify every parameter by hand.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Since the manifest code creates a bunch of directories you can speed up your test creation by using the ruby each iterator and iterate around the resources you are creating by defining an array. This only works because all the resources have the same attributes with the exception of the name. Alternatively, you could statically define each test case as well, but call me lazy. Rspec-puppet which is the testing library required for testing puppet code will query the catalog that puppet generated during the testing process.
When you write a test it should mentally read, “The manifest named nonrootlib when compiled into a catalog is expected to contain the file resource XX with ensure set to directory.”
1 2 3 4 5 6 7 8 9
I have also gone ahead and removed the content line from the service and bash_profile resource because I will discuss verifying content in a future article. So go ahead and replace the contents below in your own spec/classes/nonrootlib_spec.rb file just like below.
1 2 3 4 5 6 7 8 9 10
The finished test code after refactor.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56
At this point your ready to test and should see similar output (I have omitted some deprecation warnings)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Now that you have created your first rspec-puppet test you should be able to start testing your infallibleness on your own module and find out just how perfect your code is. With the help of Retrospec this should be pretty easy. But remember unit testing is not just about testing your code, its about maintaining code integrity long after you have left. Because many times when new folks are added to a team they make a ton of mistakes until they are familiar with the code base. So its important to build a safety net for them with basic unit tests that allows them to test against a feature set outlined in the test code.
There are many things that I did not discuss in this article that are very important but Retrospec automated these things such as .fixtures.yml, spec_helper, Rakefile, Gemfile and others. So stay tuned and I’ll cover the these items in a later article.