===========================

Testing Django applications

===========================

Automated testing is an extremely useful weapon in the bug-killing arsenal

of the modern developer. When initially writing code, a test suite can be

used to validate that code behaves as expected. When refactoring or

modifying code, tests serve as a guide to ensure that behavior hasn't

changed unexpectedly as a result of the refactor.

Testing a web application is a complex task, as there are many

components of a web application that must be validated and tested. To

help you test your application, Django provides a test execution

framework, and range of utilities that can be used to simulate and

inspect various facets of a web application.

    This testing framework is currently under development, and may change

    slightly before the next official Django release.

    (That's *no* excuse not to write tests, though!)

Writing tests

=============

Tests in Django come in two forms: doctests and unit tests.

Writing doctests

----------------

Doctests use Python's standard doctest_ module, which searches for tests in

your docstrings. Django's test runner looks for doctests in your ``models.py``

file, and executes any that it finds. Django will also search for a file

called ``tests.py`` in the application directory (i.e., the directory that

holds ``models.py``). If a ``tests.py`` is found, it will also be searched

for doctests.

.. admonition:: What's a **docstring**?

    A good explanation of docstrings (and some guidlines for using them

    effectively) can be found in :PEP:`257`:

        A docstring is a string literal that occurs as the first statement in

        a module, function, class, or method definition.  Such a docstring

        becomes the ``__doc__`` special attribute of that object.

    Since tests often make great documentation, doctest lets you put your

    tests directly in your docstrings.

You can put doctest strings on any object in your ``models.py``, but it's

common practice to put application-level doctests in the module docstring, and

model-level doctests in the docstring for each model.

For example::

    from django.db import model

    class Animal(models.Model):

        """

        An animal that knows how to make noise

        # Create some animals

        >>> lion = Animal.objects.create(name="lion", sound="roar")

        >>> cat = Animal.objects.create(name="cat", sound="meow")

        # Make 'em speak

        >>> lion.speak()

        'The lion says "roar"'

        >>> cat.speak()

        'The cat says "meow"'

        """

        name = models.CharField(maxlength=20)

        sound = models.CharField(maxlength=20)

        def speak(self):

            return 'The %s says "%s"' % (self.name, self.sound)

When you `run your tests`_, the test utility will find this docstring, notice

that portions of it look like an interactive Python session, and execute those

lines while checking that the results match.

For more details about how doctest works, see the `standard library

documentation for doctest`_

.. _doctest: http://docs.python.org/lib/module-doctest.html

.. _standard library documentation for doctest: doctest_

Writing unittests

-----------------

Like doctests, Django's unit tests use a standard library module: unittest_.

As with doctests, Django's test runner looks for any unit test cases defined

in ``models.py``, or in a ``tests.py`` file stored in the application

directory.

An equivalent unittest test case for the above example would look like::

    import unittest

    from myapp.models import Animal

    class AnimalTestCase(unittest.TestCase):

        def setUp(self):

            self.lion = Animal.objects.create(name="lion", sound="roar")

            self.cat = Animal.objects.create(name="cat", sound="meow")

        def testSpeaking(self):

            self.assertEquals(self.lion.speak(), 'The lion says "roar"')

            self.assertEquals(self.cat.speak(), 'The cat says "meow"')

When you `run your tests`_, the test utility will find all the test cases

(that is, subclasses of ``unittest.TestCase``) in ``models.py`` and

``tests.py``, automatically build a test suite out of those test cases,

and run that suite.

For more details about ``unittest``, see the `standard library unittest

documentation`_.

.. _unittest: http://docs.python.org/lib/module-unittest.html

.. _standard library unittest documentation: unittest_

.. _run your tests: `Running tests`_

Which should I use?

-------------------

Choosing a test framework is often contentious, so Django simply supports

both of the standard Python test frameworks. Choosing one is up to each

developer's personal tastes; each is supported equally. Since each test

system has different benefits, the best approach is probably to use both

together, picking the test system to match the type of tests you need to

write.

For developers new to testing, however, this choice can seem

confusing, so here are a few key differences to help you decide whether

doctests or unit tests are right for you.

If you've been using Python for a while, ``doctest`` will probably feel more

"pythonic". It's designed to make writing tests as easy as possible, so

there's no overhead of writing classes or methods; you simply put tests in

docstrings. This gives the added advantage of given your modules automatic

documentation -- well-written doctests can kill both the documentation and the

testing bird with a single stone.

For developers just getting started with testing, using doctests will probably

get you started faster.

The ``unittest`` framework will probably feel very familiar to developers

coming from Java.  Since ``unittest`` is inspired by Java's JUnit, if

you've used testing frameworks in other languages that similarly were

inspired by JUnit, ``unittest`` should also feel pretty familiar.

Since ``unittest`` is organized around classes and methods, if you need

to write a bunch of tests that all share similar code, you can easily use

subclass to abstract common tasks; this makes test code shorter and cleaner.

There's also support for explicit setup and/or cleanup routines, which give

you a high level of control over the environment your test cases run in.

Again, remember that you can use both systems side-by-side (even in the same

app). In the end, most projects will eventually end up using both; each shines

in different circumstances.

Testing Tools

=============

To assist in testing various features of your application, Django provides

tools that can be used to establish tests and test conditions.

* `Test Client`_

* Fixtures_

Test Client

-----------

The Test Client is a simple dummy browser. It allows you to simulate

GET and POST requests on a URL, and observe the response that is received.

This allows you to test that the correct view is executed for a given URL,

and that the view constructs the correct response.

As the response is generated, the Test Client gathers details on the

Template and Context objects that were used to generate the response. These

Templates and Contexts are then provided as part of the response, and can be

used as test conditions.

.. admonition:: Test Client vs Browser Automation?

    The Test Client is not intended as a replacement for Twill_, Selenium_,

    or other browser automation frameworks - it is intended to allow

    testing of the contexts and templates produced by a view,

    rather than the HTML rendered to the end-user.

    A comprehensive test suite should use a combination of both: Test Client

    tests to establish that the correct view is being called and that

    the view is collecting the correct context data, and Browser Automation

    tests to check that user interface behaves as expected.

.. _Twill: http://twill.idyll.org/

.. _Selenium: http://www.openqa.org/selenium/

Making requests

~~~~~~~~~~~~~~~

Creating an instance of ``Client`` (``django.test.client.Client``) requires

no arguments at time of construction. Once constructed, the following methods

can be invoked on the ``Client`` instance.

``get(path, data={})``

    Make a GET request on the provided ``path``. The key-value pairs in the

    data dictionary will be used to create a GET data payload. For example::

        c = Client()

        c.get('/customers/details/', {'name':'fred', 'age':7})

    will result in the evaluation of a GET request equivalent to::

        http://yoursite.com/customers/details/?name=fred&age=7

``post(path, data={}, content_type=MULTIPART_CONTENT)``

    Make a POST request on the provided ``path``. If you provide a content type

    (e.g., ``text/xml`` for an XML payload), the contents of ``data`` will be 

    sent as-is in the POST request, using the content type in the HTTP 

    ``Content-Type`` header.

    If you do not provide a value for ``content_type``, the values in 

    ``data`` will be transmitted with a content type of ``multipart/form-data``.

    The key-value pairs in the data dictionary will be encoded as a multipart

    message and used to create the POST data payload.

    To submit multiple values for a given key (for example, to specify 

    the selections for a multiple selection list), provide the values as a 

    list or tuple for the required key. For example, a data dictionary of

    ``{'choices': ('a','b','d')}`` would submit three selected rows for the

    field named ``choices``.

    Submitting files is a special case. To POST a file, you need only

    provide the file field name as a key, and a file handle to the file you wish to

    upload as a value. The Test Client will populate the two POST fields (i.e.,

    ``field`` and ``field_file``) required by Django's FileField. For example::

        c = Client()

        f = open('wishlist.doc')

        c.post('/customers/wishes/', {'name':'fred', 'attachment':f})

        f.close()

    will result in the evaluation of a POST request on ``/customers/wishes/``,

    with a POST dictionary that contains `name`, `attachment` (containing the

    file name), and `attachment_file` (containing the file data). Note that you

    need to manually close the file after it has been provided to the POST.

``login(path, username, password)``

    In a production site, it is likely that some views will be protected with

    the @login_required decorator provided by ``django.contrib.auth``. Interacting

    with a URL that has been login protected is a slightly complex operation,

    so the Test Client provides a simple method to automate the login process. A

    call to ``login()`` stimulates the series of GET and POST calls required

    to log a user into a @login_required protected view.

    If login is possible, the final return value of ``login()`` is the response

    that is generated by issuing a GET request on the protected URL. If login

    is not possible, ``login()`` returns False.

    Note that since the test suite will be executed using the test database,

    which contains no users by default. As a result, logins for your production

    site will not work. You will need to create users as part of the test suite

    to be able to test logins to your application.

Testing Responses

~~~~~~~~~~~~~~~~~

The ``get()``, ``post()`` and ``login()`` methods all return a Response

object. This Response object has the following properties that can be used

for testing purposes:

    ===============  ==========================================================

    Property         Description

    ===============  ==========================================================

    ``status_code``  The HTTP status of the response. See RFC2616_ for a

                     full list of HTTP status codes.

    ``content``      The body of the response. The is the final page

                     content as rendered by the view, or any error message

                     (such as the URL for a 302 redirect).

    ``template``     The Template instance that was used to render the final

                     content. Testing ``template.name`` can be particularly

                     useful; if the template was loaded from a file,

                     ``template.name`` will be the file name that was loaded.

                     If multiple templates were rendered, (e.g., if one

                     template includes another template),``template`` will

                     be a list of Template objects, in the order in which

                     they were rendered.

    ``context``      The Context that was used to render the template that

                     produced the response content.

                     As with ``template``, if multiple templates were rendered

                     ``context`` will be a list of Context objects, stored in

                     the order in which they were rendered.

    ===============  ==========================================================

.. _RFC2616: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

Exceptions

~~~~~~~~~~

If you point the Test Client at a view that raises an exception, that exception

will be visible in the test case. You can then use a standard ``try...catch``

block, or ``unittest.TestCase.assertRaises()`` to test for exceptions.

The only exceptions that are not visible in a Test Case are ``Http404``,

``PermissionDenied`` and ``SystemExit``. Django catches these exceptions

internally and converts them into the appropriate HTTP responses codes.

Persistent state

~~~~~~~~~~~~~~~~

The Test Client is stateful; if a cookie is returned as part of a response,

that cookie is provided as part of the next request issued by that Client

instance. Expiry policies for these cookies are not followed; if you want

a cookie to expire, either delete it manually or create a new Client

instance (which will effectively delete all cookies).

There are two properties of the Test Client which are used to store persistent

state information. If necessary, these properties can be interrogated as

part of a test condition.

    ===============  ==========================================================

    Property         Description

    ===============  ==========================================================

    ``cookies``      A Python ``SimpleCookie`` object, containing the current

                     values of all the client cookies.

    ``session``      A dictionary-like object containing session information.

                     See the `session documentation`_ for full details.

    ===============  ==========================================================

.. _`session documentation`: ../sessions/

Example

~~~~~~~

The following is a simple unit test using the Test Client::

    import unittest

    from django.test.client import Client

    class SimpleTest(unittest.TestCase):

        def setUp(self):

            # Every test needs a client

            self.client = Client()

        def test_details(self):

            # Issue a GET request

            response = self.client.get('/customer/details/')

            # Check that the respose is 200 OK

            self.failUnlessEqual(response.status_code, 200)

            # Check that the rendered context contains 5 customers

            self.failUnlessEqual(len(response.context['customers']), 5)

Fixtures

--------

A test case for a database-backed website isn't much use if there isn't any

data in the database. To make it easy to put test data into the database,

Django provides a fixtures framework.

A *Fixture* is a collection of files that contain the serialized contents of

the database. Each fixture has a unique name; however, the files that

comprise the fixture can be distributed over multiple directories, in

multiple applications.

.. note::

    If you have synchronized a Django project, you have already experienced 

    the use of one fixture -- the ``initial_data`` fixture. Every time you

    synchronize the database, Django installs the ``initial_data`` fixture.

    This provides a mechanism to populate a new database with any initial

    data (such as a default set of categories). Fixtures with other names

    can be installed manually using ``django-admin.py loaddata``. 

However, for the purposes of unit testing, each test must be able to 

guarantee the contents of the database at the start of each and every

test. To do this, Django provides a TestCase baseclass that can integrate

with fixtures.

Moving from a normal unittest TestCase to a Django TestCase is easy - just

change the base class of your test, and define a list of fixtures

to be used. For example, the test case from `Writing unittests`_ would 

look like::

    from django.test import TestCase

    from myapp.models import Animal

    class AnimalTestCase(TestCase):

        fixtures = ['mammals.json', 'birds']

        def setUp(self):

            # test definitions as before

At the start of each test case, before ``setUp()`` is run, Django will

flush the database, returning the database the state it was in directly 

after ``syncdb`` was called. Then, all the named fixtures are installed. 

In this example, any JSON fixture called ``mammals``, and any fixture

named ``birds`` will be installed. See the documentation on 

`loading fixtures`_ for more details on defining and installing fixtures.

.. _`loading fixtures`: ../django_admin/#loaddata-fixture-fixture

This flush/load procedure is repeated for each test in the test case, so you 

can be certain that the outcome of a test will not be affected by 

another test, or the order of test execution.

Running tests

=============

Run your tests using your project's ``manage.py`` utility::

    $ ./manage.py test

If you only want to run tests for a particular application, add the

application name to the command line. For example, if your

``INSTALLED_APPS`` contains ``myproject.polls`` and ``myproject.animals``,

but you only want to run the animals unit tests, run::

    $ ./manage.py test animals

When you run your tests, you'll see a bunch of text flow by as the test

database is created and models are initialized. This test database is

created from scratch every time you run your tests.

By default, the test database gets its name by prepending ``test_`` to

the database name specified by the ``DATABASE_NAME`` setting; all other

database settings will the same as they would be for the project normally.

If you wish to use a name other than the default for the test database,

you can use the ``TEST_DATABASE_NAME`` setting to provide a name.

Once the test database has been established, Django will run your tests.

If everything goes well, at the end you'll see::

    ----------------------------------------------------------------------

    Ran 22 tests in 0.221s

    OK

If there are test failures, however, you'll see full details about what tests

failed::

    ======================================================================

    FAIL: Doctest: ellington.core.throttle.models

    ----------------------------------------------------------------------

    Traceback (most recent call last):

      File "/dev/django/test/doctest.py", line 2153, in runTest

        raise self.failureException(self.format_failure(new.getvalue()))

    AssertionError: Failed doctest test for myapp.models

      File "/dev/myapp/models.py", line 0, in models

    ----------------------------------------------------------------------

    File "/dev/myapp/models.py", line 14, in myapp.models

    Failed example:

        throttle.check("actor A", "action one", limit=2, hours=1)

    Expected:

        True

    Got:

        False

    ----------------------------------------------------------------------

    Ran 2 tests in 0.048s

    FAILED (failures=1)

The return code for the script will indicate the number of tests that failed.

Regardless of whether the tests pass or fail, the test database is destroyed when

all the tests have been executed. 

Using a different testing framework

===================================

Doctest and Unittest are not the only Python testing frameworks. While

Django doesn't provide explicit support these alternative frameworks,

it does provide a mechanism to allow you to invoke tests constructed for

an alternative framework as if they were normal Django tests.

When you run ``./manage.py test``, Django looks at the ``TEST_RUNNER``

setting to determine what to do. By default, ``TEST_RUNNER`` points to 

``django.test.simple.run_tests``. This method defines the default Django

testing behavior. This behavior involves:

#. Performing global pre-test setup

#. Creating the test database

#. Running ``syncdb`` to install models and initial data into the test database

#. Looking for Unit Tests and Doctests in ``models.py`` and ``tests.py`` file for each installed application

#. Running the Unit Tests and Doctests that are found

#. Destroying the test database

#. Performing global post-test teardown

If you define your own test runner method and point ``TEST_RUNNER``

at that method, Django will execute your test runner whenever you run

``./manage.py test``. In this way, it is possible to use any test

framework that can be executed from Python code.

Defining a test runner

----------------------

By convention, a test runner should be called ``run_tests``; however, you

can call it anything you want. The only requirement is that it accept two

arguments:

``run_tests(module_list, verbosity=1)``

    The module list is the list of Python modules that contain the models to be

    tested. This is the same format returned by ``django.db.models.get_apps()``

    Verbosity determines the amount of notification and debug information that

    will be printed to the console; `0` is no output, `1` is normal output,

    and `2` is verbose output.

    This method should return the number of tests that failed.

Testing utilities

-----------------

To assist in the creation of your own test runner, Django provides

a number of utility methods in the ``django.test.utils`` module.

``setup_test_environment()``

    Performs any global pre-test setup, such as the installing the

    instrumentation of the template rendering system.

``teardown_test_environment()``

    Performs any global post-test teardown, such as removing the instrumentation