|
1 =========================== |
|
2 Testing Django applications |
|
3 =========================== |
|
4 |
|
5 Automated testing is an extremely useful weapon in the bug-killing arsenal |
|
6 of the modern developer. When initially writing code, a test suite can be |
|
7 used to validate that code behaves as expected. When refactoring or |
|
8 modifying code, tests serve as a guide to ensure that behavior hasn't |
|
9 changed unexpectedly as a result of the refactor. |
|
10 |
|
11 Testing a web application is a complex task, as there are many |
|
12 components of a web application that must be validated and tested. To |
|
13 help you test your application, Django provides a test execution |
|
14 framework, and range of utilities that can be used to simulate and |
|
15 inspect various facets of a web application. |
|
16 |
|
17 This testing framework is currently under development, and may change |
|
18 slightly before the next official Django release. |
|
19 |
|
20 (That's *no* excuse not to write tests, though!) |
|
21 |
|
22 Writing tests |
|
23 ============= |
|
24 |
|
25 Tests in Django come in two forms: doctests and unit tests. |
|
26 |
|
27 Writing doctests |
|
28 ---------------- |
|
29 |
|
30 Doctests use Python's standard doctest_ module, which searches for tests in |
|
31 your docstrings. Django's test runner looks for doctests in your ``models.py`` |
|
32 file, and executes any that it finds. Django will also search for a file |
|
33 called ``tests.py`` in the application directory (i.e., the directory that |
|
34 holds ``models.py``). If a ``tests.py`` is found, it will also be searched |
|
35 for doctests. |
|
36 |
|
37 .. admonition:: What's a **docstring**? |
|
38 |
|
39 A good explanation of docstrings (and some guidlines for using them |
|
40 effectively) can be found in :PEP:`257`: |
|
41 |
|
42 A docstring is a string literal that occurs as the first statement in |
|
43 a module, function, class, or method definition. Such a docstring |
|
44 becomes the ``__doc__`` special attribute of that object. |
|
45 |
|
46 Since tests often make great documentation, doctest lets you put your |
|
47 tests directly in your docstrings. |
|
48 |
|
49 You can put doctest strings on any object in your ``models.py``, but it's |
|
50 common practice to put application-level doctests in the module docstring, and |
|
51 model-level doctests in the docstring for each model. |
|
52 |
|
53 For example:: |
|
54 |
|
55 from django.db import model |
|
56 |
|
57 class Animal(models.Model): |
|
58 """ |
|
59 An animal that knows how to make noise |
|
60 |
|
61 # Create some animals |
|
62 >>> lion = Animal.objects.create(name="lion", sound="roar") |
|
63 >>> cat = Animal.objects.create(name="cat", sound="meow") |
|
64 |
|
65 # Make 'em speak |
|
66 >>> lion.speak() |
|
67 'The lion says "roar"' |
|
68 >>> cat.speak() |
|
69 'The cat says "meow"' |
|
70 """ |
|
71 |
|
72 name = models.CharField(maxlength=20) |
|
73 sound = models.CharField(maxlength=20) |
|
74 |
|
75 def speak(self): |
|
76 return 'The %s says "%s"' % (self.name, self.sound) |
|
77 |
|
78 When you `run your tests`_, the test utility will find this docstring, notice |
|
79 that portions of it look like an interactive Python session, and execute those |
|
80 lines while checking that the results match. |
|
81 |
|
82 For more details about how doctest works, see the `standard library |
|
83 documentation for doctest`_ |
|
84 |
|
85 .. _doctest: http://docs.python.org/lib/module-doctest.html |
|
86 .. _standard library documentation for doctest: doctest_ |
|
87 |
|
88 Writing unittests |
|
89 ----------------- |
|
90 |
|
91 Like doctests, Django's unit tests use a standard library module: unittest_. |
|
92 As with doctests, Django's test runner looks for any unit test cases defined |
|
93 in ``models.py``, or in a ``tests.py`` file stored in the application |
|
94 directory. |
|
95 |
|
96 An equivalent unittest test case for the above example would look like:: |
|
97 |
|
98 import unittest |
|
99 from myapp.models import Animal |
|
100 |
|
101 class AnimalTestCase(unittest.TestCase): |
|
102 |
|
103 def setUp(self): |
|
104 self.lion = Animal.objects.create(name="lion", sound="roar") |
|
105 self.cat = Animal.objects.create(name="cat", sound="meow") |
|
106 |
|
107 def testSpeaking(self): |
|
108 self.assertEquals(self.lion.speak(), 'The lion says "roar"') |
|
109 self.assertEquals(self.cat.speak(), 'The cat says "meow"') |
|
110 |
|
111 When you `run your tests`_, the test utility will find all the test cases |
|
112 (that is, subclasses of ``unittest.TestCase``) in ``models.py`` and |
|
113 ``tests.py``, automatically build a test suite out of those test cases, |
|
114 and run that suite. |
|
115 |
|
116 For more details about ``unittest``, see the `standard library unittest |
|
117 documentation`_. |
|
118 |
|
119 .. _unittest: http://docs.python.org/lib/module-unittest.html |
|
120 .. _standard library unittest documentation: unittest_ |
|
121 .. _run your tests: `Running tests`_ |
|
122 |
|
123 Which should I use? |
|
124 ------------------- |
|
125 |
|
126 Choosing a test framework is often contentious, so Django simply supports |
|
127 both of the standard Python test frameworks. Choosing one is up to each |
|
128 developer's personal tastes; each is supported equally. Since each test |
|
129 system has different benefits, the best approach is probably to use both |
|
130 together, picking the test system to match the type of tests you need to |
|
131 write. |
|
132 |
|
133 For developers new to testing, however, this choice can seem |
|
134 confusing, so here are a few key differences to help you decide whether |
|
135 doctests or unit tests are right for you. |
|
136 |
|
137 If you've been using Python for a while, ``doctest`` will probably feel more |
|
138 "pythonic". It's designed to make writing tests as easy as possible, so |
|
139 there's no overhead of writing classes or methods; you simply put tests in |
|
140 docstrings. This gives the added advantage of given your modules automatic |
|
141 documentation -- well-written doctests can kill both the documentation and the |
|
142 testing bird with a single stone. |
|
143 |
|
144 For developers just getting started with testing, using doctests will probably |
|
145 get you started faster. |
|
146 |
|
147 The ``unittest`` framework will probably feel very familiar to developers |
|
148 coming from Java. Since ``unittest`` is inspired by Java's JUnit, if |
|
149 you've used testing frameworks in other languages that similarly were |
|
150 inspired by JUnit, ``unittest`` should also feel pretty familiar. |
|
151 |
|
152 Since ``unittest`` is organized around classes and methods, if you need |
|
153 to write a bunch of tests that all share similar code, you can easily use |
|
154 subclass to abstract common tasks; this makes test code shorter and cleaner. |
|
155 There's also support for explicit setup and/or cleanup routines, which give |
|
156 you a high level of control over the environment your test cases run in. |
|
157 |
|
158 Again, remember that you can use both systems side-by-side (even in the same |
|
159 app). In the end, most projects will eventually end up using both; each shines |
|
160 in different circumstances. |
|
161 |
|
162 Testing Tools |
|
163 ============= |
|
164 |
|
165 To assist in testing various features of your application, Django provides |
|
166 tools that can be used to establish tests and test conditions. |
|
167 |
|
168 * `Test Client`_ |
|
169 * Fixtures_ |
|
170 |
|
171 Test Client |
|
172 ----------- |
|
173 |
|
174 The Test Client is a simple dummy browser. It allows you to simulate |
|
175 GET and POST requests on a URL, and observe the response that is received. |
|
176 This allows you to test that the correct view is executed for a given URL, |
|
177 and that the view constructs the correct response. |
|
178 |
|
179 As the response is generated, the Test Client gathers details on the |
|
180 Template and Context objects that were used to generate the response. These |
|
181 Templates and Contexts are then provided as part of the response, and can be |
|
182 used as test conditions. |
|
183 |
|
184 .. admonition:: Test Client vs Browser Automation? |
|
185 |
|
186 The Test Client is not intended as a replacement for Twill_, Selenium_, |
|
187 or other browser automation frameworks - it is intended to allow |
|
188 testing of the contexts and templates produced by a view, |
|
189 rather than the HTML rendered to the end-user. |
|
190 |
|
191 A comprehensive test suite should use a combination of both: Test Client |
|
192 tests to establish that the correct view is being called and that |
|
193 the view is collecting the correct context data, and Browser Automation |
|
194 tests to check that user interface behaves as expected. |
|
195 |
|
196 .. _Twill: http://twill.idyll.org/ |
|
197 .. _Selenium: http://www.openqa.org/selenium/ |
|
198 |
|
199 Making requests |
|
200 ~~~~~~~~~~~~~~~ |
|
201 |
|
202 Creating an instance of ``Client`` (``django.test.client.Client``) requires |
|
203 no arguments at time of construction. Once constructed, the following methods |
|
204 can be invoked on the ``Client`` instance. |
|
205 |
|
206 ``get(path, data={})`` |
|
207 Make a GET request on the provided ``path``. The key-value pairs in the |
|
208 data dictionary will be used to create a GET data payload. For example:: |
|
209 |
|
210 c = Client() |
|
211 c.get('/customers/details/', {'name':'fred', 'age':7}) |
|
212 |
|
213 will result in the evaluation of a GET request equivalent to:: |
|
214 |
|
215 http://yoursite.com/customers/details/?name=fred&age=7 |
|
216 |
|
217 ``post(path, data={}, content_type=MULTIPART_CONTENT)`` |
|
218 Make a POST request on the provided ``path``. If you provide a content type |
|
219 (e.g., ``text/xml`` for an XML payload), the contents of ``data`` will be |
|
220 sent as-is in the POST request, using the content type in the HTTP |
|
221 ``Content-Type`` header. |
|
222 |
|
223 If you do not provide a value for ``content_type``, the values in |
|
224 ``data`` will be transmitted with a content type of ``multipart/form-data``. |
|
225 The key-value pairs in the data dictionary will be encoded as a multipart |
|
226 message and used to create the POST data payload. |
|
227 |
|
228 To submit multiple values for a given key (for example, to specify |
|
229 the selections for a multiple selection list), provide the values as a |
|
230 list or tuple for the required key. For example, a data dictionary of |
|
231 ``{'choices': ('a','b','d')}`` would submit three selected rows for the |
|
232 field named ``choices``. |
|
233 |
|
234 Submitting files is a special case. To POST a file, you need only |
|
235 provide the file field name as a key, and a file handle to the file you wish to |
|
236 upload as a value. The Test Client will populate the two POST fields (i.e., |
|
237 ``field`` and ``field_file``) required by Django's FileField. For example:: |
|
238 |
|
239 c = Client() |
|
240 f = open('wishlist.doc') |
|
241 c.post('/customers/wishes/', {'name':'fred', 'attachment':f}) |
|
242 f.close() |
|
243 |
|
244 will result in the evaluation of a POST request on ``/customers/wishes/``, |
|
245 with a POST dictionary that contains `name`, `attachment` (containing the |
|
246 file name), and `attachment_file` (containing the file data). Note that you |
|
247 need to manually close the file after it has been provided to the POST. |
|
248 |
|
249 ``login(path, username, password)`` |
|
250 In a production site, it is likely that some views will be protected with |
|
251 the @login_required decorator provided by ``django.contrib.auth``. Interacting |
|
252 with a URL that has been login protected is a slightly complex operation, |
|
253 so the Test Client provides a simple method to automate the login process. A |
|
254 call to ``login()`` stimulates the series of GET and POST calls required |
|
255 to log a user into a @login_required protected view. |
|
256 |
|
257 If login is possible, the final return value of ``login()`` is the response |
|
258 that is generated by issuing a GET request on the protected URL. If login |
|
259 is not possible, ``login()`` returns False. |
|
260 |
|
261 Note that since the test suite will be executed using the test database, |
|
262 which contains no users by default. As a result, logins for your production |
|
263 site will not work. You will need to create users as part of the test suite |
|
264 to be able to test logins to your application. |
|
265 |
|
266 Testing Responses |
|
267 ~~~~~~~~~~~~~~~~~ |
|
268 |
|
269 The ``get()``, ``post()`` and ``login()`` methods all return a Response |
|
270 object. This Response object has the following properties that can be used |
|
271 for testing purposes: |
|
272 |
|
273 =============== ========================================================== |
|
274 Property Description |
|
275 =============== ========================================================== |
|
276 ``status_code`` The HTTP status of the response. See RFC2616_ for a |
|
277 full list of HTTP status codes. |
|
278 |
|
279 ``content`` The body of the response. The is the final page |
|
280 content as rendered by the view, or any error message |
|
281 (such as the URL for a 302 redirect). |
|
282 |
|
283 ``template`` The Template instance that was used to render the final |
|
284 content. Testing ``template.name`` can be particularly |
|
285 useful; if the template was loaded from a file, |
|
286 ``template.name`` will be the file name that was loaded. |
|
287 |
|
288 If multiple templates were rendered, (e.g., if one |
|
289 template includes another template),``template`` will |
|
290 be a list of Template objects, in the order in which |
|
291 they were rendered. |
|
292 |
|
293 ``context`` The Context that was used to render the template that |
|
294 produced the response content. |
|
295 |
|
296 As with ``template``, if multiple templates were rendered |
|
297 ``context`` will be a list of Context objects, stored in |
|
298 the order in which they were rendered. |
|
299 =============== ========================================================== |
|
300 |
|
301 .. _RFC2616: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html |
|
302 |
|
303 Exceptions |
|
304 ~~~~~~~~~~ |
|
305 |
|
306 If you point the Test Client at a view that raises an exception, that exception |
|
307 will be visible in the test case. You can then use a standard ``try...catch`` |
|
308 block, or ``unittest.TestCase.assertRaises()`` to test for exceptions. |
|
309 |
|
310 The only exceptions that are not visible in a Test Case are ``Http404``, |
|
311 ``PermissionDenied`` and ``SystemExit``. Django catches these exceptions |
|
312 internally and converts them into the appropriate HTTP responses codes. |
|
313 |
|
314 Persistent state |
|
315 ~~~~~~~~~~~~~~~~ |
|
316 |
|
317 The Test Client is stateful; if a cookie is returned as part of a response, |
|
318 that cookie is provided as part of the next request issued by that Client |
|
319 instance. Expiry policies for these cookies are not followed; if you want |
|
320 a cookie to expire, either delete it manually or create a new Client |
|
321 instance (which will effectively delete all cookies). |
|
322 |
|
323 There are two properties of the Test Client which are used to store persistent |
|
324 state information. If necessary, these properties can be interrogated as |
|
325 part of a test condition. |
|
326 |
|
327 =============== ========================================================== |
|
328 Property Description |
|
329 =============== ========================================================== |
|
330 ``cookies`` A Python ``SimpleCookie`` object, containing the current |
|
331 values of all the client cookies. |
|
332 |
|
333 ``session`` A dictionary-like object containing session information. |
|
334 See the `session documentation`_ for full details. |
|
335 =============== ========================================================== |
|
336 |
|
337 .. _`session documentation`: ../sessions/ |
|
338 |
|
339 Example |
|
340 ~~~~~~~ |
|
341 |
|
342 The following is a simple unit test using the Test Client:: |
|
343 |
|
344 import unittest |
|
345 from django.test.client import Client |
|
346 |
|
347 class SimpleTest(unittest.TestCase): |
|
348 def setUp(self): |
|
349 # Every test needs a client |
|
350 self.client = Client() |
|
351 def test_details(self): |
|
352 # Issue a GET request |
|
353 response = self.client.get('/customer/details/') |
|
354 |
|
355 # Check that the respose is 200 OK |
|
356 self.failUnlessEqual(response.status_code, 200) |
|
357 # Check that the rendered context contains 5 customers |
|
358 self.failUnlessEqual(len(response.context['customers']), 5) |
|
359 |
|
360 Fixtures |
|
361 -------- |
|
362 |
|
363 A test case for a database-backed website isn't much use if there isn't any |
|
364 data in the database. To make it easy to put test data into the database, |
|
365 Django provides a fixtures framework. |
|
366 |
|
367 A *Fixture* is a collection of files that contain the serialized contents of |
|
368 the database. Each fixture has a unique name; however, the files that |
|
369 comprise the fixture can be distributed over multiple directories, in |
|
370 multiple applications. |
|
371 |
|
372 .. note:: |
|
373 If you have synchronized a Django project, you have already experienced |
|
374 the use of one fixture -- the ``initial_data`` fixture. Every time you |
|
375 synchronize the database, Django installs the ``initial_data`` fixture. |
|
376 This provides a mechanism to populate a new database with any initial |
|
377 data (such as a default set of categories). Fixtures with other names |
|
378 can be installed manually using ``django-admin.py loaddata``. |
|
379 |
|
380 |
|
381 However, for the purposes of unit testing, each test must be able to |
|
382 guarantee the contents of the database at the start of each and every |
|
383 test. To do this, Django provides a TestCase baseclass that can integrate |
|
384 with fixtures. |
|
385 |
|
386 Moving from a normal unittest TestCase to a Django TestCase is easy - just |
|
387 change the base class of your test, and define a list of fixtures |
|
388 to be used. For example, the test case from `Writing unittests`_ would |
|
389 look like:: |
|
390 |
|
391 from django.test import TestCase |
|
392 from myapp.models import Animal |
|
393 |
|
394 class AnimalTestCase(TestCase): |
|
395 fixtures = ['mammals.json', 'birds'] |
|
396 |
|
397 def setUp(self): |
|
398 # test definitions as before |
|
399 |
|
400 At the start of each test case, before ``setUp()`` is run, Django will |
|
401 flush the database, returning the database the state it was in directly |
|
402 after ``syncdb`` was called. Then, all the named fixtures are installed. |
|
403 In this example, any JSON fixture called ``mammals``, and any fixture |
|
404 named ``birds`` will be installed. See the documentation on |
|
405 `loading fixtures`_ for more details on defining and installing fixtures. |
|
406 |
|
407 .. _`loading fixtures`: ../django_admin/#loaddata-fixture-fixture |
|
408 |
|
409 This flush/load procedure is repeated for each test in the test case, so you |
|
410 can be certain that the outcome of a test will not be affected by |
|
411 another test, or the order of test execution. |
|
412 |
|
413 Running tests |
|
414 ============= |
|
415 |
|
416 Run your tests using your project's ``manage.py`` utility:: |
|
417 |
|
418 $ ./manage.py test |
|
419 |
|
420 If you only want to run tests for a particular application, add the |
|
421 application name to the command line. For example, if your |
|
422 ``INSTALLED_APPS`` contains ``myproject.polls`` and ``myproject.animals``, |
|
423 but you only want to run the animals unit tests, run:: |
|
424 |
|
425 $ ./manage.py test animals |
|
426 |
|
427 When you run your tests, you'll see a bunch of text flow by as the test |
|
428 database is created and models are initialized. This test database is |
|
429 created from scratch every time you run your tests. |
|
430 |
|
431 By default, the test database gets its name by prepending ``test_`` to |
|
432 the database name specified by the ``DATABASE_NAME`` setting; all other |
|
433 database settings will the same as they would be for the project normally. |
|
434 If you wish to use a name other than the default for the test database, |
|
435 you can use the ``TEST_DATABASE_NAME`` setting to provide a name. |
|
436 |
|
437 Once the test database has been established, Django will run your tests. |
|
438 If everything goes well, at the end you'll see:: |
|
439 |
|
440 ---------------------------------------------------------------------- |
|
441 Ran 22 tests in 0.221s |
|
442 |
|
443 OK |
|
444 |
|
445 If there are test failures, however, you'll see full details about what tests |
|
446 failed:: |
|
447 |
|
448 ====================================================================== |
|
449 FAIL: Doctest: ellington.core.throttle.models |
|
450 ---------------------------------------------------------------------- |
|
451 Traceback (most recent call last): |
|
452 File "/dev/django/test/doctest.py", line 2153, in runTest |
|
453 raise self.failureException(self.format_failure(new.getvalue())) |
|
454 AssertionError: Failed doctest test for myapp.models |
|
455 File "/dev/myapp/models.py", line 0, in models |
|
456 |
|
457 ---------------------------------------------------------------------- |
|
458 File "/dev/myapp/models.py", line 14, in myapp.models |
|
459 Failed example: |
|
460 throttle.check("actor A", "action one", limit=2, hours=1) |
|
461 Expected: |
|
462 True |
|
463 Got: |
|
464 False |
|
465 |
|
466 ---------------------------------------------------------------------- |
|
467 Ran 2 tests in 0.048s |
|
468 |
|
469 FAILED (failures=1) |
|
470 |
|
471 The return code for the script will indicate the number of tests that failed. |
|
472 |
|
473 Regardless of whether the tests pass or fail, the test database is destroyed when |
|
474 all the tests have been executed. |
|
475 |
|
476 Using a different testing framework |
|
477 =================================== |
|
478 |
|
479 Doctest and Unittest are not the only Python testing frameworks. While |
|
480 Django doesn't provide explicit support these alternative frameworks, |
|
481 it does provide a mechanism to allow you to invoke tests constructed for |
|
482 an alternative framework as if they were normal Django tests. |
|
483 |
|
484 When you run ``./manage.py test``, Django looks at the ``TEST_RUNNER`` |
|
485 setting to determine what to do. By default, ``TEST_RUNNER`` points to |
|
486 ``django.test.simple.run_tests``. This method defines the default Django |
|
487 testing behavior. This behavior involves: |
|
488 |
|
489 #. Performing global pre-test setup |
|
490 #. Creating the test database |
|
491 #. Running ``syncdb`` to install models and initial data into the test database |
|
492 #. Looking for Unit Tests and Doctests in ``models.py`` and ``tests.py`` file for each installed application |
|
493 #. Running the Unit Tests and Doctests that are found |
|
494 #. Destroying the test database |
|
495 #. Performing global post-test teardown |
|
496 |
|
497 If you define your own test runner method and point ``TEST_RUNNER`` |
|
498 at that method, Django will execute your test runner whenever you run |
|
499 ``./manage.py test``. In this way, it is possible to use any test |
|
500 framework that can be executed from Python code. |
|
501 |
|
502 Defining a test runner |
|
503 ---------------------- |
|
504 By convention, a test runner should be called ``run_tests``; however, you |
|
505 can call it anything you want. The only requirement is that it accept two |
|
506 arguments: |
|
507 |
|
508 ``run_tests(module_list, verbosity=1)`` |
|
509 The module list is the list of Python modules that contain the models to be |
|
510 tested. This is the same format returned by ``django.db.models.get_apps()`` |
|
511 |
|
512 Verbosity determines the amount of notification and debug information that |
|
513 will be printed to the console; `0` is no output, `1` is normal output, |
|
514 and `2` is verbose output. |
|
515 |
|
516 This method should return the number of tests that failed. |
|
517 |
|
518 Testing utilities |
|
519 ----------------- |
|
520 |
|
521 To assist in the creation of your own test runner, Django provides |
|
522 a number of utility methods in the ``django.test.utils`` module. |
|
523 |
|
524 ``setup_test_environment()`` |
|
525 Performs any global pre-test setup, such as the installing the |
|
526 instrumentation of the template rendering system. |
|
527 |
|
528 ``teardown_test_environment()`` |
|
529 Performs any global post-test teardown, such as removing the instrumentation |
|
530 of the template rendering system. |
|
531 |
|
532 ``create_test_db(verbosity=1, autoclobber=False)`` |
|
533 Creates a new test database, and run ``syncdb`` against it. |
|
534 |
|
535 ``verbosity`` has the same behavior as in the test runner. |
|
536 |
|
537 ``Autoclobber`` describes the behavior that will occur if a database with |
|
538 the same name as the test database is discovered. If ``autoclobber`` is False, |
|
539 the user will be asked to approve destroying the existing database. ``sys.exit`` |
|
540 is called if the user does not approve. If autoclobber is ``True``, the database |
|
541 will be destroyed without consulting the user. |
|
542 |
|
543 ``create_test_db()`` has the side effect of modifying |
|
544 ``settings.DATABASE_NAME`` to match the name of the test database. |
|
545 |
|
546 ``destroy_test_db(old_database_name, verbosity=1)`` |
|
547 Destroys the database with the name ``settings.DATABASE_NAME`` matching, |
|
548 and restores the value of ``settings.DATABASE_NAME`` to the provided name. |
|
549 |
|
550 ``verbosity`` has the same behavior as in the test runner. |