Checking pages quickly in Selenium

My first lot of Selenium tests would painstakingly enter the data into a form, save it, and then painstakingly check that the data entered was showing up in the right place, in the right way.

A useful tip from Matthias Verraes at PHPUK conference was to use levenstein string comparison function to just look at the page, and compare it to a known good version.

So here's some helper code in the parent Test class.

First, we run the test and save the results.  We're assuming that everything's working at this point.

    /**
     * @param $content string
     */
    protected function saveResult ($content) {
        $filename = $this->generateFilenameForSavedContent();
        file_put_contents($filename, $content);
        $this->markTestIncomplete("This test is still saving results");
    }

    /**
     * @return string
     */
    private function generateFilenameForSavedContent () {
        $trace = debug_backtrace();
        // We want the name of the test class/method, so we jump up two
        // steps up the backtrace (as this is called by saveResult()
        $caller = array_slice($trace, 2, 1);
        return './expectedresults/' . $caller[0]['class'] . '-' . $caller[0]['function'];
    }

The tests then include a $this->saveResult($this->one('#idOfReturnElement')->text()); in them. This saves the text content (stripped of html, which is fine for me for current purposes) in the file. Note that the naming scheme means there can only be one file per test - that suits me and is probably a good limitation to place on the tests.

They also mark the test as incomplete: once these files are generated, the saveResult() call can and should be removed from the tests. There could, I suppose, be a file_exists() check, and only save and markIncomplete() if it doesn't. But then all the tests will all say ->saveResult() in them, which should be misleading most of the time. Anyway, this is how I've done it and then a quick find/replace removes the saveResult() calls in the tests.

In fact they're replaced by a call to the following method:

    /**
     * @param $actualContent string
     * @param $howCloseInPercent int
     * @return bool
     */
    protected function matchesSavedResult ($actualContent, $howCloseInPercent = 100) {
        $filename = $this->generateFilenameForSavedContent();
        $expectedContent = file_get_contents('./expectedresults/' . $filename);
        
        if ($howCloseInPercent === 100) {
            return ($actualContent === $expectedContent);
        }

        similar_text($actualContent, $expectedContent, $percent);

        return ($percent > $howCloseInPercent);
    }

This doesn't use the levenstein string comparison function: although it performs faster, it has a character limit of 255 which is too low in general. similar_text() is slower but doesn't have a string limit. If we want identity that's easy, but the percent closeness allows us some latitude, depending on the nature of the test. If we're passing in complete html, rather than just the textContent of the DOM, we might want a bit more latitude to allow for css changes, for example.

It all seems to work nicely, and has sped up writing tests no end.  I still have to write the input stuff, but then write the tests once to save the results (and commit them to git), remove the saveResults() calls and replace with matchesSavedResult().