===========================

Testing Django applications

===========================

.. module:: django.test

   :synopsis: Testing tools for Django applications.

Automated testing is an extremely useful bug-killing tool for the modern

Web developer. You can use a collection of tests -- a **test suite** -- to

solve, or avoid, a number of problems:

    * When you're writing new code, you can use tests to validate your code

      works as expected.

    * When you're refactoring or modifying old code, you can use tests to

      ensure your changes haven't affected your application's behavior

      unexpectedly.

Testing a Web application is a complex task, because a Web application is made

of several layers of logic -- from HTTP-level request handling, to form

validation and processing, to template rendering. With Django's test-execution

framework and assorted utilities, you can simulate requests, insert test data,

inspect your application's output and generally verify your code is doing what

it should be doing.

The best part is, it's really easy.

This document is split into two primary sections. First, we explain how to

write tests with Django. Then, we explain how to run them.

Writing tests

=============

There are two primary ways to write tests with Django, corresponding to the

two test frameworks that ship in the Python standard library. The two

frameworks are:

    * **Doctests** -- tests that are embedded in your functions' docstrings and

      are written in a way that emulates a session of the Python interactive

      interpreter. For example::

          def my_func(a_list, idx):

              """

              >>> a = ['larry', 'curly', 'moe']

              >>> my_func(a, 0)

              'larry'

              >>> my_func(a, 1)

              'curly'

              """

              return a_list[idx]

    * **Unit tests** -- tests that are expressed as methods on a Python class

      that subclasses ``unittest.TestCase``. For example::

          import unittest

          class MyFuncTestCase(unittest.TestCase):

              def testBasic(self):

                  a = ['larry', 'curly', 'moe']

                  self.assertEquals(my_func(a, 0), 'larry')

                  self.assertEquals(my_func(a, 1), 'curly')

You can choose the test framework you like, depending on which syntax you

prefer, or you can mix and match, using one framework for some of your code and

the other framework for other code. You can also use any *other* Python test

frameworks, as we'll explain in a bit.

Writing doctests

----------------

Doctests use Python's standard doctest_ module, which searches your docstrings

for statements that resemble a session of the Python interactive interpreter.

A full explanation of how doctest works is out of the scope of this document;

read Python's official documentation for the details.

.. admonition:: What's a **docstring**?

    A good explanation of docstrings (and some guidelines for using them

    effectively) can be found in :pep:`257`:

        A docstring is a string literal that occurs as the first statement in

        a module, function, class, or method definition.  Such a docstring

        becomes the ``__doc__`` special attribute of that object.

    For example, this function has a docstring that describes what it does::

        def add_two(num):

          "Return the result of adding two to the provided number."

           return num + 2

    Because tests often make great documentation, putting tests directly in

    your docstrings is an effective way to document *and* test your code.

For a given Django application, the test runner looks for doctests in two

places:

    * The ``models.py`` file. You can define module-level doctests and/or a

      doctest for individual models. It's common practice to put

      application-level doctests in the module docstring and model-level

      doctests in the model docstrings.

    * A file called ``tests.py`` in the application directory -- i.e., the

      directory that holds ``models.py``. This file is a hook for any and all

      doctests you want to write that aren't necessarily related to models.

Here is an example model doctest::

    # models.py

    from django.db import models

    class Animal(models.Model):

        """

        An animal that knows how to make noise

        # Create some animals

        >>> lion = Animal.objects.create(name="lion", sound="roar")

        >>> cat = Animal.objects.create(name="cat", sound="meow")

        # Make 'em speak

        >>> lion.speak()

        'The lion says "roar"'

        >>> cat.speak()

        'The cat says "meow"'

        """

        name = models.CharField(max_length=20)

        sound = models.CharField(max_length=20)

        def speak(self):

            return 'The %s says "%s"' % (self.name, self.sound)

When you :ref:`run your tests <running-tests>`, the test runner will find this

docstring, notice that portions of it look like an interactive Python session,

and execute those lines while checking that the results match.

In the case of model tests, note that the test runner takes care of creating

its own test database. That is, any test that accesses a database -- by

creating and saving model instances, for example -- will not affect your

production database. However, the database is not refreshed between doctests,

so if your doctest requires a certain state you should consider flushing the

database or loading a fixture. (See the section on fixtures, below, for more

on this.) Note that to use this feature, the database user Django is connecting

as must have ``CREATE DATABASE`` rights.

For more details about how doctest works, see the `standard library

documentation for doctest`_.

.. _doctest: http://docs.python.org/library/doctest.html

.. _standard library documentation for doctest: doctest_

Writing unit tests

------------------

Like doctests, Django's unit tests use a standard library module: unittest_.

This module uses a different way of defining tests, taking a class-based

approach.

As with doctests, for a given Django application, the test runner looks for

unit tests in two places:

    * The ``models.py`` file. The test runner looks for any subclass of

      ``unittest.TestCase`` in this module.

    * A file called ``tests.py`` in the application directory -- i.e., the

      directory that holds ``models.py``. Again, the test runner looks for any

      subclass of ``unittest.TestCase`` in this module.

This example ``unittest.TestCase`` subclass is equivalent to the example given

in the doctest section above::

    import unittest

    from myapp.models import Animal

    class AnimalTestCase(unittest.TestCase):

        def setUp(self):

            self.lion = Animal.objects.create(name="lion", sound="roar")

            self.cat = Animal.objects.create(name="cat", sound="meow")

        def testSpeaking(self):

            self.assertEquals(self.lion.speak(), 'The lion says "roar"')

            self.assertEquals(self.cat.speak(), 'The cat says "meow"')

When you :ref:`run your tests <running-tests>`, the default behavior of the

test utility is to find all the test cases (that is, subclasses of

``unittest.TestCase``) in ``models.py`` and ``tests.py``, automatically build a

test suite out of those test cases, and run that suite.

There is a second way to define the test suite for a module: if you define a

function called ``suite()`` in either ``models.py`` or ``tests.py``, the

Django test runner will use that function to construct the test suite for that

module. This follows the `suggested organization`_ for unit tests. See the

Python documentation for more details on how to construct a complex test

suite.

For more details about ``unittest``, see the `standard library unittest

documentation`_.

.. _unittest: http://docs.python.org/library/unittest.html

.. _standard library unittest documentation: unittest_

.. _suggested organization: http://docs.python.org/library/unittest.html#organizing-tests

Which should I use?

-------------------

Because Django supports both of the standard Python test frameworks, it's up to

you and your tastes to decide which one to use. You can even decide to use

*both*.

For developers new to testing, however, this choice can seem confusing. Here,

then, are a few key differences to help you decide which approach is right for

you:

    * If you've been using Python for a while, ``doctest`` will probably feel

      more "pythonic". It's designed to make writing tests as easy as possible,

      so it requires no overhead of writing classes or methods. You simply put

      tests in docstrings. This has the added advantage of serving as

      documentation (and correct documentation, at that!).

      If you're just getting started with testing, using doctests will probably

      get you started faster.

    * The ``unittest`` framework will probably feel very familiar to developers

      coming from Java. ``unittest`` is inspired by Java's JUnit, so you'll

      feel at home with this method if you've used JUnit or any test framework

      inspired by JUnit.

    * If you need to write a bunch of tests that share similar code, then

      you'll appreciate the ``unittest`` framework's organization around

      classes and methods. This makes it easy to abstract common tasks into

      common methods. The framework also supports explicit setup and/or cleanup

      routines, which give you a high level of control over the environment

      in which your test cases are run.

Again, remember that you can use both systems side-by-side (even in the same

app). In the end, most projects will eventually end up using both. Each shines

in different circumstances.

.. _running-tests:

Running tests

=============

Once you've written tests, run them using the :djadmin:`test` command of

your project's ``manage.py`` utility::

    $ ./manage.py test

By default, this will run every test in every application in

:setting:`INSTALLED_APPS`. If you only want to run tests for a particular

application, add the application name to the command line. For example, if your

:setting:`INSTALLED_APPS` contains ``'myproject.polls'`` and

``'myproject.animals'``, you can run the ``myproject.animals`` unit tests alone

with this command::

    $ ./manage.py test animals

Note that we used ``animals``, not ``myproject.animals``.

.. versionadded:: 1.0

   You can now choose which test to run.

You can be even *more* specific by naming an individual test case. To

run a single test case in an application (for example, the

``AnimalTestCase`` described in the "Writing unit tests" section), add

the name of the test case to the label on the command line::

    $ ./manage.py test animals.AnimalTestCase

And it gets even more granular than that! To run a *single* test

method inside a test case, add the name of the test method to the

label::

    $ ./manage.py test animals.AnimalTestCase.testFluffyAnimals

.. versionadded:: 1.2

   The ability to select individual doctests was added.

You can use the same rules if you're using doctests. Django will use the

test label as a path to the test method or class that you want to run.

If your ``models.py`` or ``tests.py`` has a function with a doctest, or

class with a class-level doctest, you can invoke that test by appending the

name of the test method or class to the label::

    $ ./manage.py test animals.classify

If you want to run the doctest for a specific method in a class, add the

name of the method to the label::

    $ ./manage.py test animals.Classifier.run

If you're using a ``__test__`` dictionary to specify doctests for a

module, Django will use the label as a key in the ``__test__`` dictionary

for defined in ``models.py`` and ``tests.py``.

.. versionadded:: 1.2

   You can now trigger a graceful exit from a test run by pressing ``Ctrl-C``.

If you press ``Ctrl-C`` while the tests are running, the test runner will

wait for the currently running test to complete and then exit gracefully.

During a graceful exit the test runner will output details of any test

failures, report on how many tests were run and how many errors and failures

were encountered, and destroy any test databases as usual. Thus pressing

``Ctrl-C`` can be very useful if you forget to pass the :djadminopt:`--failfast`

option, notice that some tests are unexpectedly failing, and want to get details

on the failures without waiting for the full test run to complete.

If you do not want to wait for the currently running test to finish, you

can press ``Ctrl-C`` a second time and the test run will halt immediately,

but not gracefully. No details of the tests run before the interruption will

be reported, and any test databases created by the run will not be destroyed.

.. admonition:: Test with warnings enabled

    It is a good idea to run your tests with ``python -Wall manage.py

    test``. This will allow you to catch any deprecation warnings that

    might be in your code. Django (as well as many other libraries) use

    warnings to flag when features are deprecated. It can also flag

    areas in your code that are not strictly wrong, but may benefit

    from a better implementation.

Running tests outside the test runner

-------------------------------------

If you want to run tests outside of ``./manage.py test`` -- for example,

from a shell prompt -- you will need to set up the test

environment first. Django provides a convenience method to do this::

    >>> from django.test.utils import setup_test_environment

    >>> setup_test_environment()

This convenience method sets up the test database, and puts other

Django features into modes that allow for repeatable testing.

The call to :meth:`~django.test.utils.setup_test_environment` is made

automatically as part of the setup of `./manage.py test`. You only

need to manually invoke this method if you're not using running your

tests via Django's test runner.

The test database

-----------------

Tests that require a database (namely, model tests) will not use your "real"

(production) database. Separate, blank databases are created for the tests.

Regardless of whether the tests pass or fail, the test databases are destroyed

when all the tests have been executed.

By default the test databases get their names by prepending ``test_``

to the value of the :setting:`NAME` settings for the databases

defined in :setting:`DATABASES`. When using the SQLite database engine

the tests will by default use an in-memory database (i.e., the

database will be created in memory, bypassing the filesystem

entirely!). If you want to use a different database name, specify

:setting:`TEST_NAME` in the dictionary for any given database in

:setting:`DATABASES`.

Aside from using a separate database, the test runner will otherwise

use all of the same database settings you have in your settings file:

:setting:`ENGINE`, :setting:`USER`, :setting:`HOST`, etc. The test

database is created by the user specified by ``USER``, so you'll need

to make sure that the given user account has sufficient privileges to

create a new database on the system.

.. versionadded:: 1.0

For fine-grained control over the character encoding of your test

database, use the :setting:`TEST_CHARSET` option. If you're using

MySQL, you can also use the :setting:`TEST_COLLATION` option to

control the particular collation used by the test database. See the

:doc:`settings documentation </ref/settings>` for details of these

advanced settings.

.. _topics-testing-masterslave:

Testing master/slave configurations

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. versionadded:: 1.2

If you're testing a multiple database configuration with master/slave

replication, this strategy of creating test databases poses a problem.

When the test databases are created, there won't be any replication,

and as a result, data created on the master won't be seen on the

slave.

To compensate for this, Django allows you to define that a database is

a *test mirror*. Consider the following (simplified) example database

configuration::

    DATABASES = {

        'default': {

            'ENGINE': 'django.db.backends.mysql',

            'NAME': 'myproject',

            'HOST': 'dbmaster',

             # ... plus some other settings

        },

        'slave': {

            'ENGINE': 'django.db.backends.mysql',

            'NAME': 'myproject',

            'HOST': 'dbslave',

            'TEST_MIRROR': 'default'

            # ... plus some other settings

        }

    }

In this setup, we have two database servers: ``dbmaster``, described

by the database alias ``default``, and ``dbslave`` described by the

alias ``slave``. As you might expect, ``dbslave`` has been configured

by the database administrator as a read slave of ``dbmaster``, so in

normal activity, any write to ``default`` will appear on ``slave``.

If Django created two independent test databases, this would break any

tests that expected replication to occur. However, the ``slave``

database has been configured as a test mirror (using the

:setting:`TEST_MIRROR` setting), indicating that under testing,

``slave`` should be treated as a mirror of ``default``.

When the test environment is configured, a test version of ``slave``

will *not* be created. Instead the connection to ``slave``

will be redirected to point at ``default``. As a result, writes to

``default`` will appear on ``slave`` -- but because they are actually

the same database, not because there is data replication between the

two databases.

.. _topics-testing-creation-dependencies:

Controlling creation order for test databases

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. versionadded:: 1.2.4

By default, Django will always create the ``default`` database first.

However, no guarantees are made on the creation order of any other

databases in your test setup.

If your database configuration requires a specific creation order, you

can specify the dependencies that exist using the

:setting:`TEST_DEPENDENCIES` setting. Consider the following

(simplified) example database configuration::

    DATABASES = {

        'default': {

             # ... db settings

             'TEST_DEPENDENCIES': ['diamonds']

        },

        'diamonds': {

            # ... db settings

        },

        'clubs': {

            # ... db settings

            'TEST_DEPENDENCIES': ['diamonds']

        },

        'spades': {

            # ... db settings

            'TEST_DEPENDENCIES': ['diamonds','hearts']

        },

        'hearts': {

            # ... db settings

            'TEST_DEPENDENCIES': ['diamonds','clubs']

        }

    }

Under this configuration, the ``diamonds`` database will be created first,

as it is the only database alias without dependencies. The ``default``` and

``clubs`` alias will be created next (although the order of creation of this

pair is not guaranteed); then ``hearts``; and finally ``spades``.

If there are any circular dependencies in the

:setting:`TEST_DEPENDENCIES` definition, an ``ImproperlyConfigured``

exception will be raised.

Other test conditions

---------------------

Regardless of the value of the :setting:`DEBUG` setting in your configuration

file, all Django tests run with :setting:`DEBUG`\=False. This is to ensure that

the observed output of your code matches what will be seen in a production

setting.

Understanding the test output

-----------------------------

When you run your tests, you'll see a number of messages as the test runner

prepares itself. You can control the level of detail of these messages with the

``verbosity`` option on the command line::

    Creating test database...

    Creating table myapp_animal

    Creating table myapp_mineral

    Loading 'initial_data' fixtures...

    No fixtures found.

This tells you that the test runner is creating a test database, as described

in the previous section.

Once the test database has been created, Django will run your tests.

If everything goes well, you'll see something like this::

    ----------------------------------------------------------------------

    Ran 22 tests in 0.221s

    OK

If there are test failures, however, you'll see full details about which tests

failed::

    ======================================================================

    FAIL: Doctest: ellington.core.throttle.models

    ----------------------------------------------------------------------

    Traceback (most recent call last):

      File "/dev/django/test/doctest.py", line 2153, in runTest

        raise self.failureException(self.format_failure(new.getvalue()))

    AssertionError: Failed doctest test for myapp.models

      File "/dev/myapp/models.py", line 0, in models

    ----------------------------------------------------------------------

    File "/dev/myapp/models.py", line 14, in myapp.models

    Failed example:

        throttle.check("actor A", "action one", limit=2, hours=1)

    Expected:

        True

    Got:

        False

    ----------------------------------------------------------------------

    Ran 2 tests in 0.048s

    FAILED (failures=1)