Sasha

13 Aug, 2014

Headless Browser Testing for multiple Drupal projects

In this post we will talk about how to setup Acceptance testing using Codeception + PhantomJS + Webception.

Sasha Varganov's blog

In this post we will talk about how to setup Acceptance testing using Codeception + PhantomJS + Webception. Codeception is a modern PHP testing framework. Previously we were using Behat but find Codeception much faster to write tests in as we are able to write them directly in PHP and not through the over head of translating Gerkin. Webception is a beautifully clean web front end for Codeception that allows non developers to run tests and review the output. PhantomJS is a fast headless browser that performs the BDD tests.

Why do we need this:

because we want to produce rock-solid software
because we want to be sure everything works fine after every release
because we want to be able to test some specific functionality

So, before you start you should have:

Ubuntu 14.04(64bit)
LEMP or LAMP stack installed
Git
Composer
Positive attitude :-)

Step 1 - Install Headless browser (PhantomJS)

To install Phantomjs we just need to run

sudo apt-get install phantomjs

To run it use this command:

phantomjs --webdriver=4444

We don't want to run this command manually every time before we run tests. We want it to run all the time even after server restart. To achieve that we will use Supervisor:

sudo apt-get install supervisor

To make PhantomJS runing all the time we need to add these lines to /etc/supervisor/supervisord.conf:

[program:phantomjs]
command=phantomjs --webdriver=4444

To start supervisord, run $BINDIR/supervisord(/usr/local/bin/supervisord). The resulting process will daemonize itself and detach from the terminal. It keeps an operations log at $CWD/supervisor.log by default (in our case: /var/log/supervisor).

Because we are using a distribution-packaged version of Supervisor, it is already integrated into the service management infrastructure of your distribution. This means that it will run automatically after system restart, as well as PhantomJS.

If you need more information about Supervisor, you can find it here.

Step 2 - Install Webception

Clone Webception from Github repository:

git clone https://github.com/jayhealey/Webception.git webtests

Create virtual host pointed to the webception folder. Virtual host configuration for nginx:

server {
        listen       80;
        server_name  webtests.loc;
        root /var/www/webtests/public/;
 
        access_log /var/log/nginx/webtests.loc.access.log;
        error_log  /var/log/nginx/webtests.loc.error.log info;
 
        location / {
            index index.php;
            try_files $uri $uri/ /index.php?$args;
        }
 
        location ~ \.php/?(.*)$ {
            fastcgi_connect_timeout 600000s;
            fastcgi_read_timeout 100000s;
            fastcgi_pass unix:/var/run/php5-fpm.sock;
            fastcgi_index  index.php;
            include fastcgi_params;
            fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
        }
}

We want Webception to work with Codeception 2.0.1, so we need to make several corrections to composer.json file:

{
    "name": "jayhealey/webception",
    "description": "Web Interface for running Codeception tests.",
    "keywords": ["codeception", "webception", "testing", "php", "web interface"],
    "homepage" : "https://github.com/jayhealey/Webception",
    "authors": [
        {
            "name": "James Healey",
            "email": "jayhealey [at] gmail [dot] com",
            "homepage": "http://twitter.com/jayhealey",
            "role": "Developer"
        }
    ],
    "license": "MIT",
    "require": {
		"php": ">=5.3.0",
		"slim/slim": "2.*",
		"slim/views": "0.1.*",
		"twig/twig": "~1.13",
		"symfony/yaml": "2.5.*@dev",
		"codeception/codeception": "2.0.1"
	  },
	  "autoload": {
		"psr-0": {
		  "App": ""
		}
	  },
	  "minimum-stability": "dev"
}

Now we need to delete lock file:

rm composer.lock

After that we can install dependencies:

composer install

While installation is going you can get your coffee :-)

Change cache folder permissions after installation:

chmod 777 App/Templates/_cache

Change permissions for default log folder:

chmod 777 App/Tests/_log

Now if you visit webception.loc in your browser you should be able to see this:

As you can see there are a lot of default tests we don't need. Feel free to delete all *Cept.php files form Tests/acceptence folder.

To make sure our installation works we will create our own basic test. First of all we want a WebGuy class file to be writeable, so Codeception will be able to rebuild it according to settings file.

chmod 666 App/Tests/acceptance/WebGuy.php

Change global settings file:

paths:
    tests: App/Tests
    log: App/Tests/_log
    data: App/Tests/_data
    helpers: App/Tests/_helpers
settings:
    bootstrap: _bootstrap.php
    suite_class: \PHPUnit_Framework_TestSuite
    colors: false
    memory_limit: 1024M
    log: true
modules:
    enabled: [WebDriver, WebHelper]
    config:
        WebDriver:
            url: 'http://webscope.co.nz/'
            browser: phantomjs
            window_size: 1024x768
            capabilities:
              unexpectedAlertBehaviour: ‘accept’
              javascriptEnabled: true

Change settings for acceptance tests(App/Tests/acceptence.suite.yml):

class_name: WebGuy

Now we can rebuild WebGuy class. Run

cd vendor/codeception/codeception
php codecept build

It will give you output:

Building Actor classes for suites: web, cli, coverage, unit
WebGuy includes modules: WebDriver, WebHelper
WebGuy.php generated successfully. 72 methods added
CliGuy includes modules: Filesystem, Cli, CliHelper, CodeHelper
CliGuy.php generated successfully. 19 methods added
CoverGuy includes modules: Filesystem, Cli, CliHelper, CoverHelper
CoverGuy.php generated successfully. 19 methods added
CodeGuy includes modules: CodeHelper, EmulateModuleHelper
CodeGuy.php generated successfully. 2 methods added

After that we can create our first test. Go to webtests/App/Tests/acceptance and delete all files(default tests) except WebGuy.php and _bootstrap.php. Create new file in this folder, call it BasicCept.php

$I = new WebGuy($scenario);
$I->amOnPage('/');
$I->resizeWindow(1200,1200);
$I->canSee('We are your Drupal web team');

We don't want to use unit tests or functional tests, so we can disable it. To do that go to App/Config/codeception.php and change these lines:

/*
    |--------------------------------------------------------------------------
    | You get to decide which type of tests get included.
    |--------------------------------------------------------------------------
    */
 
    'tests' => array(
        'acceptance' => TRUE,
        'functional' => FALSE,
        'unit'       => FALSE,
    ),

Open webtests.loc in your browser, you should see something like this:

If your Phantomjs is running just press Start and see test results:

Now we are ready for multidomain installation.

Step 3 - Setup tests for several domains

We want to have several sets of tests and we want to be able to run each set of tests on several domains. First of all we need to create environment folder:

mkdir /var/www/webtests/environment

Inside this folder we will store our sets of tests:

mkdir /var/www/webtests/environment/project1
mkdir /var/www/webtests/environment/project2

We need to change configuration file as well (/App/Config/codeception.php). Below is an example of setup for 2 projects, each project has 1 set of tests but 2 environments/domains. We will specify domains in these *.yml files

'sites' => array(
        'Project 1 - Dev'     => dirname(__FILE__) .'/../../environments/project1/codeception_dev.yml',
        'Project 1 - Staging' => dirname(__FILE__) .'/../../environments/project1/codeception_staging.yml',
        'Project 2 - Dev'     => dirname(__FILE__) .'/../../environments/project2/codeception_dev.yml',
        'Project 2 - Staging' => dirname(__FILE__) .'/../../environments/project2/codeception_staging.yml',
    ),

Now lets talk about configuration for each set of tests. As you can see on the listing above, we have several *.yml files inside project1 and project2 folders. Those files should be created by you and them should include settings like following

paths:
    tests: tests
    log: tests/_log
    data: tests/_data
    helpers: tests/_helpers
settings:
    bootstrap: _bootstrap.php
    suite_class: \PHPUnit_Framework_TestSuite
    colors: true
    memory_limit: 256M
    log: true
modules:
    config:
        WebDriver:
            url: 'http://dev.project1.webscope.co.nz/'
            browser: phantomjs
            window_size: 1024x768
            capabilities:
              unexpectedAlertBehaviour: ‘accept’
              javascriptEnabled: true

You can have as many environments/domains as you need.

Inside every folder(project1, project2) we will have tests folder. The content and the structure of that folder will be like in App/Tests folder. The only difference is we don't need to specify url in acceptance.suite.yml because we have already set it up in /environments/project1/codeception_dev.yml

To make sure our installation works like expected we will create one test for project1 and one more test for project2. Test for project1:

$I = new WebGuy($scenario);
$I->amOnPage('/');
$I->resizeWindow(1200,1200);
$I->canSee('We are your Drupal web team');

This test needs to be placed in /environments/project1/tests/acceptance folder.

One more test:

$I = new WebGuy($scenario);
$I->amOnPage('/');
$I->seeInTitle('Blog');

This one needs to be placed in /environments/project2/tests/acceptance folder.

Done. If you open in bowser webtests.loc you will be able to choose environment to run tests on:

After your selection Webception will load related tests list below automatically. So you can choose which tests to run. If you select another environment you will see another set of tests. Actually this is what we were trying to achieve.

Good luck with your tests!

Useful links: