Unit testing & web2py

Officially, web2py recommends using doctests to test your controllers. Doctest, however, is not always the ideal way to test your code. Doctests are especially inept at handling database-driven controllers, where it's important to return the database to a known state before running each test.

After a long discussion on the mailing list this article was created to provide a clear explanation of how to do unit testing in web2py using Python's unittest module. A thorough introduction to the unittest module can be found here.

How to write unit tests for web2py projects

Let's look at a sample unit test script, then break it down to understand what it's doing. The purpose of this article is to demonstrate how to use Python's unittest module with web2py projects. Unlike other such examples on the Internet, this one shows how to test controllers that interact with a database.

Example test suite: test.py

import unittest

from gluon.globals import Request

execfile("applications/api/controllers/10.py", globals())

db(db.game.id>0).delete()  # Clear the database
db.commit()

class TestListActiveGames(unittest.TestCase):
    def setUp(self):
        request = Request()  # Use a clean Request object

    def testListActiveGames(self):
        # Set variables for the test function
        request.post_vars["game_id"] = 1
        request.post_vars["username"] = "spiffytech"

        resp = list_active_games()
        db.commit()
        self.assertEquals(0, len(resp["games"]))

suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(TestListActiveGames))
unittest.TextTestRunner(verbosity=2).run(suite)

How to run the test script

Before we continue, you should know how to execute this script:

python web2py.py -S api -M -R applications/api/tests/test.py

Fill in the name of your own application after -S, and the location of your test script.

We use web2py.py to call our script because it sets up the operating environment for us; it brings in our database and gives us all of the variables that are normally passed into the controller, like request.

How it works

Let's break down the above example:

import unittest

from gluon.globals import Request  # So we can reset the request for each test

The first line, predictably, imports the unittest module.

The second line imports web2py's Request object. We want this to be available so we can use a fresh, clean, unmodified Request object in every test.

execfile("applications/api/controllers/10.py", globals())

Just like in the web2py shell, unit test scripts don't automatically have access to your controllers. This line executes your controller file, bringing all of the function declarations into the local namespace. Passing globals() to the execfile() command lets your controllers see your database.

db(db.game.id>0).delete()  # Clear the database
db.commit()

Unit testing with a database is only useful if the database looks the same when your tests run. These lines empty the database.

In your unit tests, you must run db.commit() in order for any db.update(), db.insert(), or db.delete() commands to take effect. web2py automatically runs db.commit() when a controller's function finishes, which is why you don't usually have to do it yourself. Not so in external scripts. You must also run db.commit() after calling any controller function that changes the database. There is no harm in calling db.commit() after all controller functions, just to be safe.

class TestListActiveGames(unittest.TestCase):
    def setUp(self):

Unit test suites are composed of classes whose names start with "Test". Each class has it's own setUp() function which is run before each test. You can use the setUp() function to set up any variables or conditions you need for every test in the class.

        request = Request()  # Use a clean Request object

It's important to clean up your mess between tests. In our simple example, the only thing in the operating environment we're changing is the global request object each controller function sees and works with.

    def testListActiveGames(self):
        # Set variables for the test function

The unittest module will run any function whose name starts with 'test'. Here, we've given our test function a name that describes what it's testing.

        request.post_vars["game_id"] = 1
        request.post_vars["username"] = "spiffytech"

These lines set up the variables needed by the function we're testing. post_vars is a dictionary that, in your controller, contains the values a user's browsers sent via POST. The controller function we're testing expects to see POST values, so we set them up.

        resp = list_active_games()
        db.commit()
        self.assertEquals(0, len(resp["games"]))

Now we actually test something! list_active_games() is a function in my controller. The function returns a dict of values, just like most web2py controller functions. I've captured the dict in a variable named resp, short for "response". It doesn't matter what you name the variable, as long as the name is meaningful.

The second line commits any changes to the database made by list_active_games().

The third line represents the heart of unit testing: making sure the output from a function is what we expect it to be. Since our test class, TestListActiveGames, is derived from unittest.TestCase the self object has a number of functions for testing values. Here, we use the basic assertEquals() function which, just like it sounds, checks that two values match. Unlike Python's regular assert() function, assertEquals() (and other unittest assert functions) prints useful information to the command line when assertions fail.

suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(TestListActiveGames))
unittest.TextTestRunner(verbosity=2).run(suite)

These last few lines get the unit tests started. We define a group (or "suite") of unit tests, then add to the suite all of our test classes. Here, we only have one test class, but if we had more, you'd simply repeat the second line with each classes name.

If you've used the unittest module before and wonder why we're not simply calling unittest.main(), see the Background section below.

One step further: Using a special testing database

The big problem with the above example is that it works on the database you're using to develop your application. Anything your tests do to the database (including the big fat "delete all" near the top of the script) will affect your development site.

To fix this, we need to tell web2py to create a copy of our database that we can safely use for testing. It's pretty simple:

Append this code to the bottom of db.py

# Create a test database that's laid out just like the "real" database
import copy
test_db = DAL('sqlite://testing.sqlite')  # Name and location of the test DB file
for tablename in db.tables:  # Copy tables!
    table_copy = [copy.copy(f) for f in db[tablename]]
    test_db.define_table(tablename, *table_copy)

Modify test.py to use the test DB

...
from gluon.globals import Request
db = test_db  # Rename the test database so that functions will use it instead of the real database
...

Background: for people who already use the unittest module

How unittest normally works

Unit tests are normally stored in a standalone Python script. The script imports the unittest module, some tests are defined, and a couple lines at the bottom of the file run the tests when the script is executed.

Here's an example:

import unittest

import myprogram  # Import the code you want these unit tests to test

class TestStuff(unittest.TestCase):
    def setUp(self):
        self.something = 5

    def testSomeFunction(self):
        result = myprogram.somefunction(self.something)
        self.assertEquals(result, 10)
        result = myprogram.somefunction(result)
        self.assertEquals(result, 20)

# Run the unit tests if the script is being executed from the command line
if __name__ == "__main__":
    unittest.main()

This script would be called from the command line like so:

python my_tests.py

Why this doesn't work with web2py

Explicit is better than implicit.
Flat is better than nested.
Namespaces are one honking great idea -- let's do more of those!
- Excerpted from the Zen of Python

web2py forgoes some staples of Pythonic programing philosophy in favor of being easy to teach and easy to use use. Some people think this winds results in "magic". Rather than treating each .py file as a module which is imported (a la Django), web2py sets up a ready-to-use environment behind the scenes before giving control over to the web developer. The developer sees their database and built-in web2py functions magically available in the controller, and rejoices. This is convenient when developing web applications, but causes problems for external scripts.

The normal command to run unit tests, unittest.main(), gets confused because it's run in the scope of web2py.py, rather than in the scope of a standalone script. unittest.main() also gets confused because web2py.py passes all of the command line arguments to test.py, and the unittest module doesn't know what to do with web2py.py's command line flags.

We have to do a few things different to get unit testing working with web2py:

1. Set up the web2py operating environment
2. Bring in our controllers
3. Change the way test suites are executed

Steps 1 and 2 are mandatory, and step 3 is a byproduct of the way I chose to solve 1 and 2. AlterEgo 213 shows a different way to set up the environment for unit tests than the way I describe. In AlterEgo 213, the test script handles the setup of the whole web2py environment. However, this clutters up the code with lots of stuff that you can delegate to web2py.py instead. The tradeoff is that you lose the ability to specify what test to run from the command line.

*Note that AlterEgo 213 is incomplete. Several changes must be made to it in order for your controllers to see your database. This complicates the code more than I cared for.