--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/thirdparty/google_appengine/lib/django/docs/testing.txt Tue Aug 26 21:49:54 2008 +0000
@@ -0,0 +1,550 @@
+===========================
+Testing Django applications
+===========================
+
+Automated testing is an extremely useful weapon in the bug-killing arsenal
+of the modern developer. When initially writing code, a test suite can be
+used to validate that code behaves as expected. When refactoring or
+modifying code, tests serve as a guide to ensure that behavior hasn't
+changed unexpectedly as a result of the refactor.
+
+Testing a web application is a complex task, as there are many
+components of a web application that must be validated and tested. To
+help you test your application, Django provides a test execution
+framework, and range of utilities that can be used to simulate and
+inspect various facets of a web application.
+
+ This testing framework is currently under development, and may change
+ slightly before the next official Django release.
+
+ (That's *no* excuse not to write tests, though!)
+
+Writing tests
+=============
+
+Tests in Django come in two forms: doctests and unit tests.
+
+Writing doctests
+----------------
+
+Doctests use Python's standard doctest_ module, which searches for tests in
+your docstrings. Django's test runner looks for doctests in your ``models.py``
+file, and executes any that it finds. Django will also search for a file
+called ``tests.py`` in the application directory (i.e., the directory that
+holds ``models.py``). If a ``tests.py`` is found, it will also be searched
+for doctests.
+
+.. admonition:: What's a **docstring**?
+
+ A good explanation of docstrings (and some guidlines for using them
+ effectively) can be found in :PEP:`257`:
+
+ A docstring is a string literal that occurs as the first statement in
+ a module, function, class, or method definition. Such a docstring
+ becomes the ``__doc__`` special attribute of that object.
+
+ Since tests often make great documentation, doctest lets you put your
+ tests directly in your docstrings.
+
+You can put doctest strings on any object in your ``models.py``, but it's
+common practice to put application-level doctests in the module docstring, and
+model-level doctests in the docstring for each model.
+
+For example::
+
+ from django.db import model
+
+ class Animal(models.Model):
+ """
+ An animal that knows how to make noise
+
+ # Create some animals
+ >>> lion = Animal.objects.create(name="lion", sound="roar")
+ >>> cat = Animal.objects.create(name="cat", sound="meow")
+
+ # Make 'em speak
+ >>> lion.speak()
+ 'The lion says "roar"'
+ >>> cat.speak()
+ 'The cat says "meow"'
+ """
+
+ name = models.CharField(maxlength=20)
+ sound = models.CharField(maxlength=20)
+
+ def speak(self):
+ return 'The %s says "%s"' % (self.name, self.sound)
+
+When you `run your tests`_, the test utility will find this docstring, notice
+that portions of it look like an interactive Python session, and execute those
+lines while checking that the results match.
+
+For more details about how doctest works, see the `standard library
+documentation for doctest`_
+
+.. _doctest: http://docs.python.org/lib/module-doctest.html
+.. _standard library documentation for doctest: doctest_
+
+Writing unittests
+-----------------
+
+Like doctests, Django's unit tests use a standard library module: unittest_.
+As with doctests, Django's test runner looks for any unit test cases defined
+in ``models.py``, or in a ``tests.py`` file stored in the application
+directory.
+
+An equivalent unittest test case for the above example would look like::
+
+ import unittest
+ from myapp.models import Animal
+
+ class AnimalTestCase(unittest.TestCase):
+
+ def setUp(self):
+ self.lion = Animal.objects.create(name="lion", sound="roar")
+ self.cat = Animal.objects.create(name="cat", sound="meow")
+
+ def testSpeaking(self):
+ self.assertEquals(self.lion.speak(), 'The lion says "roar"')
+ self.assertEquals(self.cat.speak(), 'The cat says "meow"')
+
+When you `run your tests`_, the test utility will find all the test cases
+(that is, subclasses of ``unittest.TestCase``) in ``models.py`` and
+``tests.py``, automatically build a test suite out of those test cases,
+and run that suite.
+
+For more details about ``unittest``, see the `standard library unittest
+documentation`_.
+
+.. _unittest: http://docs.python.org/lib/module-unittest.html
+.. _standard library unittest documentation: unittest_
+.. _run your tests: `Running tests`_
+
+Which should I use?
+-------------------
+
+Choosing a test framework is often contentious, so Django simply supports
+both of the standard Python test frameworks. Choosing one is up to each
+developer's personal tastes; each is supported equally. Since each test
+system has different benefits, the best approach is probably to use both
+together, picking the test system to match the type of tests you need to
+write.
+
+For developers new to testing, however, this choice can seem
+confusing, so here are a few key differences to help you decide whether
+doctests or unit tests are right for you.
+
+If you've been using Python for a while, ``doctest`` will probably feel more
+"pythonic". It's designed to make writing tests as easy as possible, so
+there's no overhead of writing classes or methods; you simply put tests in
+docstrings. This gives the added advantage of given your modules automatic
+documentation -- well-written doctests can kill both the documentation and the
+testing bird with a single stone.
+
+For developers just getting started with testing, using doctests will probably
+get you started faster.
+
+The ``unittest`` framework will probably feel very familiar to developers
+coming from Java. Since ``unittest`` is inspired by Java's JUnit, if
+you've used testing frameworks in other languages that similarly were
+inspired by JUnit, ``unittest`` should also feel pretty familiar.
+
+Since ``unittest`` is organized around classes and methods, if you need
+to write a bunch of tests that all share similar code, you can easily use
+subclass to abstract common tasks; this makes test code shorter and cleaner.
+There's also support for explicit setup and/or cleanup routines, which give
+you a high level of control over the environment your test cases run in.
+
+Again, remember that you can use both systems side-by-side (even in the same
+app). In the end, most projects will eventually end up using both; each shines
+in different circumstances.
+
+Testing Tools
+=============
+
+To assist in testing various features of your application, Django provides
+tools that can be used to establish tests and test conditions.
+
+* `Test Client`_
+* Fixtures_
+
+Test Client
+-----------
+
+The Test Client is a simple dummy browser. It allows you to simulate
+GET and POST requests on a URL, and observe the response that is received.
+This allows you to test that the correct view is executed for a given URL,
+and that the view constructs the correct response.
+
+As the response is generated, the Test Client gathers details on the
+Template and Context objects that were used to generate the response. These
+Templates and Contexts are then provided as part of the response, and can be
+used as test conditions.
+
+.. admonition:: Test Client vs Browser Automation?
+
+ The Test Client is not intended as a replacement for Twill_, Selenium_,
+ or other browser automation frameworks - it is intended to allow
+ testing of the contexts and templates produced by a view,
+ rather than the HTML rendered to the end-user.
+
+ A comprehensive test suite should use a combination of both: Test Client
+ tests to establish that the correct view is being called and that
+ the view is collecting the correct context data, and Browser Automation
+ tests to check that user interface behaves as expected.
+
+.. _Twill: http://twill.idyll.org/
+.. _Selenium: http://www.openqa.org/selenium/
+
+Making requests
+~~~~~~~~~~~~~~~
+
+Creating an instance of ``Client`` (``django.test.client.Client``) requires
+no arguments at time of construction. Once constructed, the following methods
+can be invoked on the ``Client`` instance.
+
+``get(path, data={})``
+ Make a GET request on the provided ``path``. The key-value pairs in the
+ data dictionary will be used to create a GET data payload. For example::
+
+ c = Client()
+ c.get('/customers/details/', {'name':'fred', 'age':7})
+
+ will result in the evaluation of a GET request equivalent to::
+
+ http://yoursite.com/customers/details/?name=fred&age=7
+
+``post(path, data={}, content_type=MULTIPART_CONTENT)``
+ Make a POST request on the provided ``path``. If you provide a content type
+ (e.g., ``text/xml`` for an XML payload), the contents of ``data`` will be
+ sent as-is in the POST request, using the content type in the HTTP
+ ``Content-Type`` header.
+
+ If you do not provide a value for ``content_type``, the values in
+ ``data`` will be transmitted with a content type of ``multipart/form-data``.
+ The key-value pairs in the data dictionary will be encoded as a multipart
+ message and used to create the POST data payload.
+
+ To submit multiple values for a given key (for example, to specify
+ the selections for a multiple selection list), provide the values as a
+ list or tuple for the required key. For example, a data dictionary of
+ ``{'choices': ('a','b','d')}`` would submit three selected rows for the
+ field named ``choices``.
+
+ Submitting files is a special case. To POST a file, you need only
+ provide the file field name as a key, and a file handle to the file you wish to
+ upload as a value. The Test Client will populate the two POST fields (i.e.,
+ ``field`` and ``field_file``) required by Django's FileField. For example::
+
+ c = Client()
+ f = open('wishlist.doc')
+ c.post('/customers/wishes/', {'name':'fred', 'attachment':f})
+ f.close()
+
+ will result in the evaluation of a POST request on ``/customers/wishes/``,
+ with a POST dictionary that contains `name`, `attachment` (containing the
+ file name), and `attachment_file` (containing the file data). Note that you
+ need to manually close the file after it has been provided to the POST.
+
+``login(path, username, password)``
+ In a production site, it is likely that some views will be protected with
+ the @login_required decorator provided by ``django.contrib.auth``. Interacting
+ with a URL that has been login protected is a slightly complex operation,
+ so the Test Client provides a simple method to automate the login process. A
+ call to ``login()`` stimulates the series of GET and POST calls required
+ to log a user into a @login_required protected view.
+
+ If login is possible, the final return value of ``login()`` is the response
+ that is generated by issuing a GET request on the protected URL. If login
+ is not possible, ``login()`` returns False.
+
+ Note that since the test suite will be executed using the test database,
+ which contains no users by default. As a result, logins for your production
+ site will not work. You will need to create users as part of the test suite
+ to be able to test logins to your application.
+
+Testing Responses
+~~~~~~~~~~~~~~~~~
+
+The ``get()``, ``post()`` and ``login()`` methods all return a Response
+object. This Response object has the following properties that can be used
+for testing purposes:
+
+ =============== ==========================================================
+ Property Description
+ =============== ==========================================================
+ ``status_code`` The HTTP status of the response. See RFC2616_ for a
+ full list of HTTP status codes.
+
+ ``content`` The body of the response. The is the final page
+ content as rendered by the view, or any error message
+ (such as the URL for a 302 redirect).
+
+ ``template`` The Template instance that was used to render the final
+ content. Testing ``template.name`` can be particularly
+ useful; if the template was loaded from a file,
+ ``template.name`` will be the file name that was loaded.
+
+ If multiple templates were rendered, (e.g., if one
+ template includes another template),``template`` will
+ be a list of Template objects, in the order in which
+ they were rendered.
+
+ ``context`` The Context that was used to render the template that
+ produced the response content.
+
+ As with ``template``, if multiple templates were rendered
+ ``context`` will be a list of Context objects, stored in
+ the order in which they were rendered.
+ =============== ==========================================================
+
+.. _RFC2616: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
+
+Exceptions
+~~~~~~~~~~
+
+If you point the Test Client at a view that raises an exception, that exception
+will be visible in the test case. You can then use a standard ``try...catch``
+block, or ``unittest.TestCase.assertRaises()`` to test for exceptions.
+
+The only exceptions that are not visible in a Test Case are ``Http404``,
+``PermissionDenied`` and ``SystemExit``. Django catches these exceptions
+internally and converts them into the appropriate HTTP responses codes.
+
+Persistent state
+~~~~~~~~~~~~~~~~
+
+The Test Client is stateful; if a cookie is returned as part of a response,
+that cookie is provided as part of the next request issued by that Client
+instance. Expiry policies for these cookies are not followed; if you want
+a cookie to expire, either delete it manually or create a new Client
+instance (which will effectively delete all cookies).
+
+There are two properties of the Test Client which are used to store persistent
+state information. If necessary, these properties can be interrogated as
+part of a test condition.
+
+ =============== ==========================================================
+ Property Description
+ =============== ==========================================================
+ ``cookies`` A Python ``SimpleCookie`` object, containing the current
+ values of all the client cookies.
+
+ ``session`` A dictionary-like object containing session information.
+ See the `session documentation`_ for full details.
+ =============== ==========================================================
+
+.. _`session documentation`: ../sessions/
+
+Example
+~~~~~~~
+
+The following is a simple unit test using the Test Client::
+
+ import unittest
+ from django.test.client import Client
+
+ class SimpleTest(unittest.TestCase):
+ def setUp(self):
+ # Every test needs a client
+ self.client = Client()
+ def test_details(self):
+ # Issue a GET request
+ response = self.client.get('/customer/details/')
+
+ # Check that the respose is 200 OK
+ self.failUnlessEqual(response.status_code, 200)
+ # Check that the rendered context contains 5 customers
+ self.failUnlessEqual(len(response.context['customers']), 5)
+
+Fixtures
+--------
+
+A test case for a database-backed website isn't much use if there isn't any
+data in the database. To make it easy to put test data into the database,
+Django provides a fixtures framework.
+
+A *Fixture* is a collection of files that contain the serialized contents of
+the database. Each fixture has a unique name; however, the files that
+comprise the fixture can be distributed over multiple directories, in
+multiple applications.
+
+.. note::
+ If you have synchronized a Django project, you have already experienced
+ the use of one fixture -- the ``initial_data`` fixture. Every time you
+ synchronize the database, Django installs the ``initial_data`` fixture.
+ This provides a mechanism to populate a new database with any initial
+ data (such as a default set of categories). Fixtures with other names
+ can be installed manually using ``django-admin.py loaddata``.
+
+
+However, for the purposes of unit testing, each test must be able to
+guarantee the contents of the database at the start of each and every
+test. To do this, Django provides a TestCase baseclass that can integrate
+with fixtures.
+
+Moving from a normal unittest TestCase to a Django TestCase is easy - just
+change the base class of your test, and define a list of fixtures
+to be used. For example, the test case from `Writing unittests`_ would
+look like::
+
+ from django.test import TestCase
+ from myapp.models import Animal
+
+ class AnimalTestCase(TestCase):
+ fixtures = ['mammals.json', 'birds']
+
+ def setUp(self):
+ # test definitions as before
+
+At the start of each test case, before ``setUp()`` is run, Django will
+flush the database, returning the database the state it was in directly
+after ``syncdb`` was called. Then, all the named fixtures are installed.
+In this example, any JSON fixture called ``mammals``, and any fixture
+named ``birds`` will be installed. See the documentation on
+`loading fixtures`_ for more details on defining and installing fixtures.
+
+.. _`loading fixtures`: ../django_admin/#loaddata-fixture-fixture
+
+This flush/load procedure is repeated for each test in the test case, so you
+can be certain that the outcome of a test will not be affected by
+another test, or the order of test execution.
+
+Running tests
+=============
+
+Run your tests using your project's ``manage.py`` utility::
+
+ $ ./manage.py test
+
+If you only want to run tests for a particular application, add the
+application name to the command line. For example, if your
+``INSTALLED_APPS`` contains ``myproject.polls`` and ``myproject.animals``,
+but you only want to run the animals unit tests, run::
+
+ $ ./manage.py test animals
+
+When you run your tests, you'll see a bunch of text flow by as the test
+database is created and models are initialized. This test database is
+created from scratch every time you run your tests.
+
+By default, the test database gets its name by prepending ``test_`` to
+the database name specified by the ``DATABASE_NAME`` setting; all other
+database settings will the same as they would be for the project normally.
+If you wish to use a name other than the default for the test database,
+you can use the ``TEST_DATABASE_NAME`` setting to provide a name.
+
+Once the test database has been established, Django will run your tests.
+If everything goes well, at the end you'll see::
+
+ ----------------------------------------------------------------------
+ Ran 22 tests in 0.221s
+
+ OK
+
+If there are test failures, however, you'll see full details about what tests
+failed::
+
+ ======================================================================
+ FAIL: Doctest: ellington.core.throttle.models
+ ----------------------------------------------------------------------
+ Traceback (most recent call last):
+ File "/dev/django/test/doctest.py", line 2153, in runTest
+ raise self.failureException(self.format_failure(new.getvalue()))
+ AssertionError: Failed doctest test for myapp.models
+ File "/dev/myapp/models.py", line 0, in models
+
+ ----------------------------------------------------------------------
+ File "/dev/myapp/models.py", line 14, in myapp.models
+ Failed example:
+ throttle.check("actor A", "action one", limit=2, hours=1)
+ Expected:
+ True
+ Got:
+ False
+
+ ----------------------------------------------------------------------
+ Ran 2 tests in 0.048s
+
+ FAILED (failures=1)
+
+The return code for the script will indicate the number of tests that failed.
+
+Regardless of whether the tests pass or fail, the test database is destroyed when
+all the tests have been executed.
+
+Using a different testing framework
+===================================
+
+Doctest and Unittest are not the only Python testing frameworks. While
+Django doesn't provide explicit support these alternative frameworks,
+it does provide a mechanism to allow you to invoke tests constructed for
+an alternative framework as if they were normal Django tests.
+
+When you run ``./manage.py test``, Django looks at the ``TEST_RUNNER``
+setting to determine what to do. By default, ``TEST_RUNNER`` points to
+``django.test.simple.run_tests``. This method defines the default Django
+testing behavior. This behavior involves:
+
+#. Performing global pre-test setup
+#. Creating the test database
+#. Running ``syncdb`` to install models and initial data into the test database
+#. Looking for Unit Tests and Doctests in ``models.py`` and ``tests.py`` file for each installed application
+#. Running the Unit Tests and Doctests that are found
+#. Destroying the test database
+#. Performing global post-test teardown
+
+If you define your own test runner method and point ``TEST_RUNNER``
+at that method, Django will execute your test runner whenever you run
+``./manage.py test``. In this way, it is possible to use any test
+framework that can be executed from Python code.
+
+Defining a test runner
+----------------------
+By convention, a test runner should be called ``run_tests``; however, you
+can call it anything you want. The only requirement is that it accept two
+arguments:
+
+``run_tests(module_list, verbosity=1)``
+ The module list is the list of Python modules that contain the models to be
+ tested. This is the same format returned by ``django.db.models.get_apps()``
+
+ Verbosity determines the amount of notification and debug information that
+ will be printed to the console; `0` is no output, `1` is normal output,
+ and `2` is verbose output.
+
+ This method should return the number of tests that failed.
+
+Testing utilities
+-----------------
+
+To assist in the creation of your own test runner, Django provides
+a number of utility methods in the ``django.test.utils`` module.
+
+``setup_test_environment()``
+ Performs any global pre-test setup, such as the installing the
+ instrumentation of the template rendering system.
+
+``teardown_test_environment()``
+ Performs any global post-test teardown, such as removing the instrumentation
+ of the template rendering system.
+
+``create_test_db(verbosity=1, autoclobber=False)``
+ Creates a new test database, and run ``syncdb`` against it.
+
+ ``verbosity`` has the same behavior as in the test runner.
+
+ ``Autoclobber`` describes the behavior that will occur if a database with
+ the same name as the test database is discovered. If ``autoclobber`` is False,
+ the user will be asked to approve destroying the existing database. ``sys.exit``
+ is called if the user does not approve. If autoclobber is ``True``, the database
+ will be destroyed without consulting the user.
+
+ ``create_test_db()`` has the side effect of modifying
+ ``settings.DATABASE_NAME`` to match the name of the test database.
+
+``destroy_test_db(old_database_name, verbosity=1)``
+ Destroys the database with the name ``settings.DATABASE_NAME`` matching,
+ and restores the value of ``settings.DATABASE_NAME`` to the provided name.
+
+ ``verbosity`` has the same behavior as in the test runner.