(Locally) Testing ansible deployments

I’ve always felt my playbooks undertested. I know about a possible solution of spinning up new OpenStack instances with the ansible nova module, but felt it to be too complex as a good idea to implement. Now I’ve found a quicker way to test your playbooks by using Docker.

In principal, all my test does is:

1. create a docker container
2. create a copy of the current ansible playbook in a temporary directory and mount it as a volume
3. inside the docker container, run the playbook

This is obviously not perfect, since:

• running a playbook locally vs connecting via ssh can be a different beast to test
• can become resource intensive if you want to test different scenarios represented as docker images.

There is possibly more, but for myself in small it is a workable solution so far.

Find the code on github if you’d like to have a look. Improvements welcome!

 

Running Selenium tests without X11

Update: Most likely my scripts for download will lead into a dead end. Please check the github repository posted by one of my commentators: https://github.com/amenk/SelfScripts/blob/master/selenium-headless

Because writing tests with zope.testbrowser can sometimes be a pain, I recently got a hint from Christian Zagrodnick to have a look back on Selenium again. Selenium plays a pre-recorded, or scripted session to test your web-application in a browser: click here, verify if ‘hello world’ is present, click there and so forth. It’s bit like driving with hands off the steering wheel.

Now, I remember having used Selenium before, but

• it was a nuissance to setup, esp to work nicely with a Plone TestCase
• getting it setup in a continous integration environment was quite some work too.

Motivation

With Plone 4, a new testcase component is implemented which made it much, much, much easier to write tests for Plone packages. The gocept.selenium packages wraps Selenium in a nice compatible package and provides test classes for Zope2, Plone and Zope3 tests. The result is, that the setup and maintainance of tests with selenium became much, much easier as well.

The only missing piece was the continious integration (CI) environment. I’m running all tests of the packages I develop at Mooball in Hudson continously. Being also able to run all Selenium tests would be a big, big plus. But without an X server, you can’t run a browser, which is essential to run the tests.

I didn’t like the idea of installing a complete X environment on my CI server, so I searched around and found these articles to run Selenium tests without requiring a complete X server installed:

• Hudson and SeleniumTest Suite
• How To: Run Selenium Tests with Hudson on a Headless Linux Server, Part One–Xvfb

There are more around, but they helped me getting started.

Problem

My main questions were:

1. How to run the selenium server? Start the server for each build or leave it running as a daemon in the background?
2. Run an X server (e.g. Xvfb) or an X server with a VNC server, because Hudson already provides plug-ins for the VNC server?
3. Run the X server per build or leave it running in the background?

My Solution

1. Selenium Server

I configured the selenium server to run as a daemon. Starting/Stopping the server per test would be too resource intensive and perhaps unreliable. So I came up with a small init script. It allows me to start the selenium server at boot time and keep it running.

2. + 3. The X Server

I opted for installing Xvfb and thought first about starting/stopping it per build, but here again, I think it becomes a nuisance and too unreliable if the server for whatever reason doesn’t start up or crashes. I created another init script to keep the Xvfb running. Note though, I don’t disable the access permissions to connect to Xvfb as most of the other blog articles do.

Installation

The installation is actually pretty much straight forward:

1. Install java and selenium-server (I preferred /opt)
2. Configure the init script variables (port, paths, browser, etc). I for one prefer to run firefoxchrome, which works fine for me.
3. Install Xvfb and adjust the init script variables.
4. Start both servers. If you’re running your tests with gocept.selenium, follow the installation instructions on the packages pypi page.

That’s it. Any suggestions and updates are welcome 🙂