PHP TestFest 2017

Prepare for PHP Test Fest 2009

It’s Wednesday 7/6/2017 and Ben Ramsey emails the UG-ADMIN mailinglist with the subject “PHP TestFest 2017”. I instantaneously think about the 2009 edition which I attended (at Combell, organized by PHPBelgium).

I had a great time during that edition (I even managed to get some test code into PHP source) and was wondering if we could join this years edition with a couple of local user groups.

I co-organize PHP-WVL and currently work for Combell (based in Ghent), so after some chatting with the folks of Ghent PHP, we agreed to join forces and organize this years edition back at Combell.

As the date we’d picked came closer, I had to figure out how to test the code and get them into PHP Source these days, so I could “mentor” the visitors. I still remembered how to write phpt files, but had to figure out the rest of the flow.

I started by listening to PHP Roundtable episode 65 in the car, to get up to speed. Next up: the PHP TestFest website and the excellent video series of Sammy.

For specific questions, the mailing list is perfect, which helped me with some questions about code coverage.

I figured out there is a fork of the PHP source where all pull requests with tests should be created:

Figuring out what to test.

To find code that needs testing, you have a couple of options:

When you look for uncovered code on the gcov site, there are some things you need to take into account:

  • It is possible that the gcov server does not have all PHP modules loaded (making it appear lots of code fragments are uncovered), see — SKIPIF —;
  • Some tests use “exec” or “popen”, making it “untrackable” for the coverage tool (the cli_server test appear to be uncovered for this reason);
  • Not all testfest tests have been merged.

To solve the last problem, I created a fork of the phptestfest fork, where I merged all the approved pull requests in a branch. This way, I can render code coverage for a specific path, including all the phptestfest tests from other users.

Our contribution.

After 2 weeks, we managed to submit 13 pull requests, where most were approved!



Using multiple databases in phpunit/dbunit with composer

I’m using multiple databases in most of my projects, so having access to multiple databases in my test suite is a must.

phpunit/dbunit is excellent, but you are stuck with one database. The guys at Etsy created very good extensions to fix this problem (MultipleDatabase), but it took me a while to figure out how to use it.

Because PHPUnit is now available via Composer, you can fetch all dependencies with a single command.

The composer.json file to include all dependencies looks like this:

Install the dependencies by (installing and) running composer:

Now you can create a “parent” class to register the databases (mine is called DatabaseTest). Make sure you create a getDatabaseConfigs method (which is required and should return an array of PHPUnit_Extensions_MultipleDatabase_Database). For the fixtures, I use Xml Datasets, which look like this.

I’ve added a getConnection method, so I can use the same assertions as the normal dbunit testcase (see Database Assertions API):

The magic about to happen is quite cool. PHPUnit will read these database configs and use them to make sure all databases and tables are in a known state before every test and does this in following order:

  1. Connect to all databases
  2. TRUNCATE all tables supplied in the database fixture file
  3. Insert all rows supplied in the fixture file
  4. Execute the test
  5. TRUNCATE all tables supplied in the database fixture file

Now you’ll be able to run tests for code making changes in your database without affecting other tests.

Some improvements:

  • Add composer support to Hamcrest (once Hamcrest PHP has moved to GitHub).
  • Add composer support to the Etsy extensions (I’ve submitted a PR, but it won’t work until Hamcrest PHP is on GitHub).
  • See if the getConnection() method can be done in a better way.

make test every PHP version and send feedback

Since David Coallier’s talk during PHPBenelux, I realized the importance of running make test on all PHP releases and send feedback to PHP.

It is so easy, that I will run it from now on every time a new release is announced.

If you’re running on Mac, you might want to install Xcode, so you can run “make” on command line. If you’re on Linux, you’re all set to go.

How to do it:

  • Open a shell
  • Create a directory (e.g. mkdir ~/src/php)
  • Download the latest version into the directory (e.g. wget
  • Untar the file (e.g. tar -xzf php-5.4.0RC6.tar.gz)
  • Go into the directory and run ./configure, you should get output like:
    checking for grep that handles long lines and -e... /usr/bin/grep
    checking for egrep... /usr/bin/grep -E
    checking for a sed that does not truncate output... /usr/bin/sed
    checking build system type... i386-apple-darwin11.2.0
    checking host system type... i386-apple-darwin11.2.0
    checking target system type... i386-apple-darwin11.2.0
    Thank you for using PHP. 
  • After that, you can run make and should get a lot of output ending in:
    Build complete.
    Don't forget to run 'make test'.
  • After that, you should do as the previous command suggests and run make test. You should see all tests passing by.
  • After all tests are run, you will get a summary. In my case, I get the message: “You may have found a problem in PHP.”
    Bug #55509 (segfault on x86_64 using more than 2G memory) [Zend/tests/bug55509.phpt]
    Sort with SORT_LOCALE_STRING [ext/standard/tests/array/locale_sort.phpt]
  • Whether you get an error or not, you should always send the report back to PHP. You can do that by just answering Y to the question: “Do you want to send this report now?”
  • Enter your email address and PHP will thank you.

How hard was that?