Jenkins and PHPUnit

Continuous Integration is the process where code from a developer’s repository is called by an automated system, built and tested against a set of standards that the developer has created in their tests. In this post we look at the steps to create a server that will automate your testing suite – and hopefully make you more productive…

At Loft, we have previously used a number of different tools for continuous integration:

Xinc – to run PHP unit tests and integration tests
PhpUnderControl (CruiseControl) – to run functional tests
PHP Unit Testing framework
Selenium RC Server – standalone server for launching functional tests in web browser

Using all these different tools made things very confusing, and our sysadmin had to learn all these different technologies to try and keep a CI single system together. Some of the main issues with this configuration were:

Xinc – could be tricky to configure, it looks like a PHP application but doesn’t behave like one, it has poor documentation, and to have Git support you have to migrate to the latest version
PhpUnderContol – did a really good job for us over the years, but the project looks half dead and project XML configuration can be quite tricky
Selenium Server – to be able to run tests in a web browser (Linux) we installed full X Server, which is not the best solution from resources point of view: it needs at least 2GB of RAM

To consolidate these systems and their configuration we opted for Jenkins CI which is a continuation of the old Hudson project, available under Creative Commons License, aimed at making CI as easy as possible to implement. The beauty of Jenkins (apart from its single tool structure that contains all CI elements) is that it is a single tool with a pretty, easy GUI that enables easy configuration of a project so that it can be tested.

Another massive benefit is that Jenkins is one of those brilliant tools (like Git) that is so customisable that you can bend it to work in whatever way you need. In this post we shall be looking at how Loft implemented Jenkins into our workflow practices.

So, let’s get started…

While planning the server, we listed what services we would need from it:

Git and SVN abilities – for getting the projects out of our repositories
PHPUnit
Xdebug
Selenium Server
JUnit – for reports

(Not part of the plan but added for fun 🙂 )

Clover reports – code coverage

Once we had the plan the next step was the build…

Jenkins installation

First, we needed a server to set up as our Continuous Integration server (henceforth referred to as CI server). This was to be a Linux Box as it is far easier to work with than a Windows box, as well as more secure and cheaper. A disadvantage is that you can’t run tests in Internet Explorer on this box.

Luckily for us, Jenkins has it’s own repo which can be installed on almost any distribution of Linux that has it’s own package manager (Such as Yum or Apt). For RHEL and similar you can install the repository with:

sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo
sudo rpm --import http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key
sudo yum install jenkins

And for Debian/Ubuntu:

wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo apt-get update
sudo apt-get install jenkins

This will do all the hard work of installing Jenkins to the server that we are working on, so all we have to do (as root) is start the service:

/etc/init.d/jenkins start

This will start Jenkins and the Jetty (Java-based webserver) that it runs on. Like us, you may decide that the Jetty is not fit for your purpose and might look for something like NginX or Apache to handle a proxy pass to the Jetty which will give you some extra security to hide your Jenkins Interface behind.

Most of Jenkins is very configurable from the management GUI. From this web page you can download and install new modules to the server which will allow you to use different tools to work out more information about your code and project. These plugins…. do just that: they plug into the project configuration area, which enables the user to add more build stages to the project and extra tests.

Selenium Server requirements

For functional testing we use Selenium Tests with the Selenium RC server to control requests to a web server which acts as a real user going through the site, clicking on links and filling in forms, all the while taking pictures of problems and assessing that all the correct information is displayed on the page. One problem with these tests is that they require a GUI or some form of desktop to use the web browser. In previous set ups of CI, Loft tended to use a Full Blown Desktop (Gnome or KDE) which made our sysadmin’s life trickier.

In this case we decided to install a different type of GUI on to the server: one that could still be viewed over an encrypted VNC connection and give access to the web browser that Selenium is running, but would also be light enough that it would not impact on the CI potential of the server.

To remedy this we used a combination of:

xvfb – as the X server
x11vnc – acting as the vnc server with the -shared -forever -localhost options set
Firefox – the web browser

The installation of the above allowed us to configure a display on xvfb which acted as a blank x server. On top of this platform we could call our web browser to run the selenium tests.

Configuring the PHP project

To run tests, you need a few extra plugins, which are easy to install through Jenkins plugin manager:

Jenkins Mailer Plugin – to be able to send emails
Git Plugin – to update tests / code
SeleniumRC plugin – if you want to run Selenium tests
Clover PHP Plugin – to be able to publish code coverage
xUnit Plugin – for publishing test results

The project has a few sections which enable you to configure what to run, when to run it, and what to do with results.

1) Update code / Tests

Jenkins has excellent support for VCS – SVN, GIT – you can choose or even combine different repositories to update tested code or tests.

2) Build trigger

You can choose when the build is started. We use the option to Trigger builds remotely for builds triggered by post-commit (receive) hooks from Git.

Here is a sample post-receive hook used in a Git repo:

#!/bin/sh 
PROJECT="/var/www/vhosts/project"
cd $PROJECT || exit
unset GIT_DIR
git pull
#start jenkins
wget http://jenkins:8080/job/project/build?token=sdsdjsdjksdjds787jk -O /dev/null

Or for functional tests we use the Build periodically option – for example to run tests every 4 hours.

3) Build

In our case, just a simple command to run all unit tests:

phpunit --log-junit results/phpunit/phpunit.xml -c tests/phpunit.xml

4) Publishing results

Because we export test results to junit format, we can use JUnit (Java Unit test) Publish to publish results.

5) Notify developers

By publishing results, Jenkins knows if the build was successful or not. If not, you can notify developers – or even point a finger at the person who broke the build!

6) Selenium

If you need to start the Selenium server, you can start it for every build or check that server is running.

7) Code coverage

Code coverage is optional, but it can give you very interesting information: