Writing the Game of Tests in PHP and Laravel

Ranking developers by tests

Test all the code!
Check out the the demo

Writing tests is fun, well, that is how I feel. I started programming in PHP a long time ago, a little while before silly things as objects or testing really were a thing. But times change, as PHP matured, it slowly changed into a real programming language and now, writing tests is commonplace. At least if you take your job seriously. But not everyone is as inclined to do so, sometimes programmers needs a little nudge to really keep testing in mind.

The idea

I was looking around for ways to make testing more fun and ran across a little script on the Spotify GitHub account, the testing game. I’m a bit of a freak for statistics, and like the idea of using gamification to make testing more visible. In the end I didn’t really find anything that would do this for PHP so I decided to write it myself.

The idea is simple. You give the library a git repository, be it a bare local one, or an online url. It looks for test files and uses git blame to attribute tests to a user. Seems simple enough.

Getting started

When I start a package like this I tend to start creating a small proof of concept. I really believe in the mantra ‘Make it work, make it pretty, make it fast’. For this, I started looking into the different kinds of tests we write at SWIS and see if they had patterns I could detect. I booted a new Laravel installation and started typing away. Gitonomy Gitlib (http://gitonomy.com/doc/gitlib/master/) makes it pretty easy to checkout and search a repository.


if(is_dir(storage_path('repository/swisnl/webbeheer-testing'))){
    $repository = new \Gitonomy\Git\Repository(
        storage_path('repository/laravel/laravel')
    );
} else {
    $repository = Gitonomy\Git\Admin::cloneTo(
        storage_path('repository/laravel/laravel'),
        'git@github.com/laravel/laravel.git'
    );
}

From there it was pretty easy to start finding relevant files and start looking at the blame to find out who was resposible for different lines of code.


$log = $repository->getLog('develop', [
    'tests/*Cest.php',
    'tests/*Cept.php',
    'tests/*Test.php',
]);

$blame = $repository->getBlame(
$repository->getHead(),
    'tests/acceptance/Webbeheer/Elements/BatchuploadCest.php'
);

$log = $log->getCommits();

dd([
    $blame, $blame->getLines(),
    $log,
    $commit->getTree()->getEntries(),
    $commit->getTree()->resolvePath('tests')->getEntries()
]);

The first data is available, yay!

Who made it?

I knew I needed a few things, I probably needed a wat to talk to Git repositories easily and analyse the code. In my search for a library which does this I came across Gitonomy, this package offers an easy way to talk to git repositories in PHP. I wanted to make finding code as simple as possible.

I determined you need to know 2 things to analyse a test. You need to know a filename (or path) pattern to find the test files. From there you need to determine how the test is counted, which line defined the test or does the file itself define the full test.

Using Git for the file lists

I first started with the implementation for a PhpUnit test. Search the repository for “*Test.php” files in the latest revision of the repository. I played around with different ways to do this, but the fastest way to find files in a bare repository is using git ls-tree. This lists the files in the repository, and combined with  grep this would give me a nice list of PhpUnit test files in the project.


/**
* @param Repository $repository
* @param string $grepArgument
* @return array
*/
public function grep($repository, $grepArgument)
{
    $command = 'git ls-tree -r --name-only HEAD | grep ' .    ProcessUtils::escapeArgument($grepArgument);
    return $this->getCommandResult($repository, $command);
}

Later down the line I found that this does slow things down, since it lists every single file for that commit. Some of the repositories I ran the package against have 50k files, which was a slight problem. To make it faster I found that using git log combined with –since was quite a lot faster since it only lists the files that changed in that time period.


/**
* @param Repository $repository
* @param string $grepArgument
* @param string $since
* @return array
*/
public function grepTimed($repository, $grepArgument, $since = '1 day ago'){
    $command = 'git log --since \'' . ProcessUtils::escapeArgument($since) .
        '\' --oneline --pretty=format: --name-only | grep ' .
        ProcessUtils::escapeArgument($grepArgument);

    return $this->getCommandResult($repository, $command);
}

After finding the files, the class needed to find the tests in the file. In order to parse the file and find the owner or tests I used blame.

...
if ($validation->isValidFile($file) === false) {
    continue;
}

$blame = $repository->getBlame($repository->getHead(), $file);

/**
* @var $lines Line[]
*/
$lines = $blame->getLines();
foreach ($lines as $line) {
    if ($this->isTestLine($line->getContent())) {

...

Parsing to value objects

For PhpUnit it was pretty easy. The test methods need to start with it or test so a simple preg_match was enough to know if a line was a test. I read the files, check for proper lines, and create a value object to store the information about that result for later usage.


class Result
{

...

public function __construct($filename, $line, $commitHash, $author, $email, $date, $parser)
{
    $this->filename = $filename;
    $this->line = $line;
    $this->author = $author;
    $this->email = $email;
    $this->date = $date;
    $this->commitHash = $commitHash;
    $this->parser = $parser;
}

After getting PHPUnit working I added Behat and Codeception tests parsers. They are pretty much the same structure only Codeception Cept files are a little different. A whole test is contained in a single file, so no need to look around for function names.

Making it pretty

After making the parsing work decently enough it needed to get a little more pretty. The code was decent-ish but not as pretty as I liked. The code was still in a src directory in the Laravel project, so I started cleaning up.

First of I extracted a simple interface  for the parses for later use. I also had a lot of duplicate code for searching through the repositories, duplicate code is no fun, so I moved it to a helper class  and made the Parsers even dumber.


interface ParserInterface
{
    /**
    * @param Repository $repository
    * @param \Swis\GoT\Result\ValidationInterface $validation
    * @return \Swis\GoT\Result[]
    */
    public function run(Repository $repository, Result\ValidationInterface $validation);

}

To make the package portable I started to extract the source to separate packages. The first one was the reading of packages and parsing. Since I developed most of the logic for that part in a separate namespace and folder in the Laravel project that was quite easy.

After the extraction I started to look for settings. What parts of the code could be useful to configure through settings. I ended up making a settings class with the relevant Settings. Again, first making it work. Using static functions for the settings, after that making is pretty and remembering to use a instantiated settings class and a factory to create is. This also helps in making the Laravel package easier to customize by using dependency injection for the Settings class.

Laravel package

After extracting the library from the code, I started working on the package for Laravel. The code didn’t do much, it made a few commands available and provided some routes. So making a service provider from that was pretty easy.

I extracted the routes into a service provider, made a views folder with the specific views for the package, and added the migrations to the package.

After that I added a few pieces pieces of code to a config file and that was pretty much it.

Most of the work was making a decent readme file for the package which documents how to use it.

Demo site

The final piece was the site. This was pretty much the left over pieces. What parts of code didn’t fit in the other packages. Which wasn’t much really. It is pretty much a default Laravel installation with some CSS and some gulp commands. This wasn’t much effort and it really helps to get up and running easily. Since it now resides on packagist it can be installed in minutes.

Usage

  1. Install using composer: $ composer create-project swisnl/game-of-tests-demo
  2. Change .env to suit your needs.
  3. Migrate the databases: $ php artisan migrate
  4. Inspect an organisation: $ php got:inpect-github laravel

All done!

Check out the running demo that keeps track of the tests written in the Laravel GitHub organisation on http://gameoftests.swis.nl/.

Packages

This is the resulting list of packages on Github.