Functional testing emails with mailcatcher, Zend_Mail and more

This is how I've got functional testing of email sending set up.

The idea is that in testing environment emails get caught on the server and phpunit can then query to check that the outgoing email looks correct.

1. Mailcatcher.

Mailcatcher is an SMTP server that receives email and provides a web, and more importantly REST API to see the emails sent.  So step 1 is to install it on the php application server VM.  It could, I suppose, have gone on a separate VM but that seemed overkill.

Install followed these instructions except that I needed some extra bits and bobs (on a Debian 7 box).  The error messages and some trial and error were enough to get this sorted.

sudo apt-get install ruby rubygems

sudo apt-get install ruby1.9.1-dev

sudo apt-get sqlite3 libsqlite3-dev

gem install mailcatcher

and the php.ini setting was actually

sendmail_path = "/usr/bin/env /var/lib/gems/1.9.1/gems/mailcatcher-0.5.12/bin/catchmail "

I don't think that in actual fact this is necessary in my case, because I'm using a Zend Framework SMTP transport, but setting this was helpful for testing while getting it all set up, and probably better safer to do just to avoid accidental spam mail.

Just running 'mailcatcher' works.  But that has it running on localhost (127.0.0.1) which isn't OK because I need to be able to access it from outside the VM.  mailcatcher is on the application server and my phpunit tests are running on the host box.  So starting like this:

$> mailcatcher --http-ip=0.0.0.0

did the job: it's now accessible from the host.  That one-liner is in a little bash script that can be called by the phing task on the host box that does all the VM initialisation prior to running tests (database reset, date/time reset, and now this).

So now I can see the mailcatcher interface at http://lamplight:1080 from the host box (where lamplight is mapping to that VM, of course).

2. Configuring the application.

We're using Zend_Mail (ZF 1, still) and the default transport is configured in the bootstrap.  The transport config options are in the core config file, so all I needed to do was set the host and port to 127.0.0.1 and 1025 respectively (I have a [functest] section of the config file).  It actually took me ages to get this working, but in the end it was irritatingly simple.

3. Querying mailcatcher from tests

I'm using a mailcatcher client without the behat stuff, which was as easy as adding "alexandresalome/mailcatcher": "*" to my composer.json file in the testing folder.  I can now use that client class in my phpunit tests to check that the emails sent are correct.

4. And next

I wonder if it'd be useful to have some assertions along the lines of this:

    $email = $this->retrieveLastEmailSent();
    $this->assertEmailRecipients(["bob@example.com", $email);
    $this->assertEmailSubject("Testing my lovely emails", $email);
    $this->assertEmailBody("Hi Bob, this is a test email", $email);

    $email2 = $this->retrieveEmailSent(1);  // gets the last but 1
    $this->assertEmailRecipients(["tracey@example.com", $email2);
    $this->assertEmailBody("Hi Tracey, this is a test email", $email2, "See how email-merges work!");
    $this->assertEmailAttachments("md5ofexpectedfileorsomething?", $email2);
    $this->assertEmailFrom("matt@example.com", $email2);



Along the lines of http://codeception.com/12-15-2013/testing-emails-in-php.html, most likely, although I think I'd like to have a try at creating my own assertion classes 'properly' (i.e. the way it's suggested in the phpunit manual in the section on extending it).

And a big thanks to the lovely people that have made these tools (Samuel Cochran and Alexandre Salomé).

Adding CLI options to PHPUnit

Issue:  I've got PHPUnit set up to run Selenium tests on varying combinations of browsers/platforms.  When I'm writing tests and doing day-to-day stuff, I want to run just on one (a VM with Firefox on Win 7, at the moment).  When I want to look into a new IE11 bug, I want to run a particular test just on Win 8/ IE 11.  When I'm running a full test run, I want to run across all target browsers/platforms.

The browser/platform combination to use has been specified up to now as a static property of my base TestCase class.  It's a shorthand: I can set it to 'FF' for Firefox on Win7, 'IE' for Internet Explorer 10 on Win7, '8' for Firefox, IE11 and Chrome on Windows 8, or 'all' for 3 browsers across Win7 and Win8.

It's not much, but I was wanting to set this as a command line option, rather than changing the value of the property on the class manually:

$ phpunit --browser-set="FF" tests/GoTestSomeStuffTest.php

The phpunit code makes it sound like this'll be straightforward.  Just subclass PHPUnit_TextUI_Command, add in a hook to the constructor and handler... easy.

Almost.

The suggested code is:

     * <?php

     * class MyCommand extends PHPUnit_TextUI_Command

     * {

     *     public function __construct()

     *     {

     *         $this->longOptions['--my-switch'] = 'myHandler';

     *     }

     *

     *     // --my-switch foo -> myHandler('foo')

     *     protected function myHandler($value)

     *     {

     *     }

     * }

(https://github.com/sebastianbergmann/phpunit/blob/master/src/TextUI/Command.php#L210)

Firstly, that's not quite right.  Looking at the other longOptions, it's clear that the initial double hyphen shouldn't be there:

     *     public function __construct()

     *     {

     *         $this->longOptions['my-switch'] = 'myHandler';

     *     }

Secondly, if you want to pass an option value you need to append an equals sign to the switch name:

     *     public function __construct()

     *     {

     *         $this->longOptions['my-switch='] = 'myHandler';

     *     }

But how do we get to use MyCommand instead of PHPUnit_TextUI_Command?  I'm not sure if this is the right answer, but it's an answer.  I've copied the bash script phpunit that starts it all up (from https://github.com/sebastianbergmann/phpunit/blob/master/phpunit), and instead of 

PHPUnit_TextUI_Command::main();

I have

require_once 'MyCommand.php';

MyCommand::main();

(OK it's not really called MyCommand, I'm just staying with the original docs!).  Plus a chmod to 755 to get it to run.

Almost there.  However, our new MyCommand constructor isn't being called still.  main() on the original looks like this:

    /**

     * @param boolean $exit

     */

    public static function main($exit = true)

    {

        $command = new static;



        return $command->run($_SERVER['argv'], $exit);

    }

so the line new static  gives us an instance of PHPUnit_TextUI_Command, because that's where we are when it gets called.  So MyCommand needs this method too, just copied straight across.  In the end it looks like this:

     <?php

     class MyCommand extends PHPUnit_TextUI_Command

     {

         public function __construct()

         {

             $this->longOptions['browser-set='] = 'setBrowserSet';

         }

     

         // --browser-set="FF" -> setBrowserSet("FF")

         protected function setBrowserSet($value)

         {

             MyTestSetup::setBrowserSetToUse($value);

         }




         public static function main($exit = true) 

         {

             $command = new static;

             return $command->run($_SERVER['argv'], $exit);

         }

     }

Validation and a default value, and handling it all, sit in MyTestSetup (which indirectly extends PHPUnit_Extensions_Selenium2TestCase).

Note there is a way to define $browsers in a configuration somehow - I think through phpunit.xml (I seem to remember seeing it there) but it sets it on PHPUnit_Extensions_SeleniumTestCase, but I'm using PHPUnit_Extensions_Selenium2TestCase and so don't think it'll work.    

(Edit: excuse the horrible code line spacing.  Another to-do for the list.)

Settings windows hosts

Note to self:

It's C:\Windows\system32\drivers\etc\hosts

And add

ip.address    hostname