1 =======================

     2 Test Driven Development

     3 =======================

     4 

    82 

    83 Let us put all these in a file and call this file **gcd.py**::

    84 

    85   def gcd(a, b):

    86       pass

    87 

    88   if __name__ == '__main__':

    89       tc1 = gcd(48, 64)

    90       if tc1 != 16:

    91           print "Test failed for the case a=48 and b=64. Expected 16. Obtained %d instead." % tc1

    92           exit(1)

    93 

    94       tc2 = gcd(44, 19)

    95       if tc2 != 1:

    96           print "Test failed for the case a=44 and b=19. Expected 1. Obtained %d instead." % tc2

    97           exit(1)

    98 

    99       print "All tests passed!"

   100 

   101 Note that we have introduced a new semantic which uses two new magic names

   102 in Python *__name__* and *__main__*. This is a very common idiom used in

   103 Python. Every Python code in a file can be run in two ways: Either as an

   104 independent stand-alone script or as a Python module which can be imported

   105 by other Python scripts or modules. When the idiom::

   106 

   107   if __name__ == '__main__':

   108 

   109 is used, the code within this if block is executed first when we run the

   110 Python file as a stand-alone script. In other words, when we run this

   111 python file as a stand-alone script the control of the program first starts

   112 from the code that is within this if block from which the control is

   113 transferred to other parts of the program or to other modules from

   114 here. This comes as an extremely handy feature especially when we want to

   115 test our modules individually. Now let us run our code as a stand-alone

   116 script.::

   117 

   118   $ python gcd.py

   119   Traceback (most recent call last):

   120     File "gcd.py", line 7, in <module> print "Test failed for the case a=48 and b=64. Expected 16. Obtained %d instead." % tc1

   121   TypeError: %d format: a number is required, not NoneType

   122 

   123 Now we have our tests, the test cases and the code unit stub at

   124 hand. We also have the failing test. So we know for sure that we have

   125 cleared the first check point of TDD where the tests have failed. The

   126 failing tests also give a green signal for us to go ahead to our next

   127 check point i.e. to write the actual code in our code unit and make

   128 the test pass. So let us write the code for the gcd function by

   129 removing the **pass** control statement which had just created a gcd

   130 function stub for us.

   131 

   132 Most of us have learnt in high school math classes that the best and

   133 the easiest known algorithm to compute the gcd of two numbers was

   134 given to us 2300 years ago by a greek mathematician named Euclid. So

   135 let us use the Euclid's algorithm to compute the gcd of two numbers a

   136 and b::

   137 

   138   def gcd(a, b):

   139       if a == 0:

   140           return b

   141       while b != 0:

   142           if a > b:

   143               a = a - b

   144           else:

   145               b = b - a

   146       return a

   147 

   148 **Note**: If you are unaware of Euclidean algorithm to compute the gcd

   149 of two numbers please refer to it on wikipedia. It has a very detailed

   150 explanation of the algorithm and its proof of validity among other

   151 things.

   152 

   153 Now let us run our script which already has the tests written in it

   154 and see what happens::

   155 

   156   $ python gcd.py

   157   All tests passed!

   158 

   159 Success! We managed to pass all the tests. But wasn't that code simple

   160 enough? Indeed it was. If you take a closer look at the code you will

   161 soon realize that the chain of subtraction operations can be replaced

   162 by a modulo operation i.e. taking remainders of the division between

   163 the two numbers since they are equivalent operations. Also modulo

   164 operation is far better than chain of subtractions because you will

   165 reduce much faster using modulo operation than the subtraction. For

   166 example if let us take 25, 5 as a and b in our example. If we write

   167 down the steps of the algorithm written above we have the following:

   168 

   169 Step 1: a = 25 b = 5: Since both a and b are not 0 and b is greater

   170 than a: b = 25 - 5 = 20

   171 Step 2: Since b is still not 0 and b is greater than a: b = 20 - 5 =

   172 15

   173 Step 3: Since b is still not 0 and b is greater than a: b = 15 - 5 =

   174 10

   175 Step 4: Since b is still not 0 and b is greater than a: b = 10 - 5 = 5

   176 Step 5: Since b is still not 0 and b is equal to a: b = 5 - 5 = 0

   177 Step 6: Since b is 0 the gcd is a = 5 which is returned

   178 

   179 If we adopt the modulo operation instead of subtraction and follow the

   180 steps:

   181 

   182 Step 1: a = 25 b = 5: Since both a and b are not 0 and b is greater

   183 than a: b = 25 % 5 = 0

   184 Step 2: Since b is 0 the gcd is a = 5 which is returned

   185 

   186 Wow! That was overwhelmingly lesser number of steps! So now we are

   187 convinced that if we replace the subtraction operation with the modulo

   188 operation our code performs much better. But if we think carefully we

   189 know that the modulo of a and b is less than b irrespective of how

   190 large the value of a is, including the case where a is already less

   191 than b. So we can eliminate that extra conditional **if** statement by

   192 just swapping the result of the modulo operation to the position of b

   193 and b to the position of a. This ensures that a is always greater than

   194 b and if not the swapping combined with modulo operation takes care of

   195 it. To exemplify it, if a = 5 and b = 25 then by swapping and

   196 performing modulo we have a = b = 25 and b = a % b = 5 % 25 = 5 and

   197 hence we proceed. So let us replace our original code with this new

   198 improved code we have come up with simple observations::

   199 

   200   def gcd(a, b):

   201       while b != 0:

   202           a, b = b, a % b

   203       return a

   204 

   205 Executing our script again we will see that all the tests pass. One

   206 final improvement we can think of which is not necessary in terms of

   207 efficiency but is certainly good to do keeping in mind the readability

   208 is that we can use the concept of recursion for the same

   209 algorithm. Without going into much detail this is how the code looks

   210 if we use a recursive approach::

   211 

   212   def gcd(a, b):

   213       if b == 0:

   214           return a

   215       return gcd(b, a%b)

   216 

   217 Much shorter and sweeter! And it passes all the tests! But there is

   218 one small problem yet. For the users of this function there is no way

   219 to determine how to use it, how many parameters it takes what it

   220 returns among other things. And same as well for those who read the

   221 code. So this function is not a very well written piece of code since

   222 it lacks documentation. So to make this function mode readable let us

   223 add the docstring for this function. Rewriting the function with the

   224 docstring looks like this::

   225 

   226   def gcd(a, b):

   227       """Returns the Greatest Common Divisor of the two integers

   228       passed as arguments.

   229 

   230       Args:

   231         a: an integer

   232         b: another integer

   233 

   234       Returns: Greatest Common Divisor of a and b

   235       """

   236       if b == 0:

   237           return a

   238       return gcd(b, a%b)

   239 

   240 Now we have refactored our code enough to make it well written piece

   241 of code. Let us move on.

   242 

   243 More realistic "Tests"

   244 ======================

   245 

   246 Now we have successfully completed writing our first test, writing the

   247 relevant code and ensured the tests passed. We also refactored our

   248 code to perform better. With the knowledge of all these and some

   249 concepts and semantics like __main__ magic names we learnt we have

   250 come a long way with respect to writing tests. But our thirst is still

   251 unquenched! We want to do more and more tests! Not just write better

   252 code but also better tests! So let us keep building upon what we have

   253 learnt so far.

   254 

   255 Let us start writing tests for more realistic test cases. Generally

   256 tests are predetermined as said above, if not the software design in

   257 itself is flawed. The predetermined tests are stored along with the

   258 test code in some persistent format like in a database, a text file, a

   259 file of specific format like XML or in some other way. Let us continue

   260 with our example of GCD function. We will keep all our test cases in a

   261 text file, which is indeed persistent. Let us specify the format of

   262 the test data in our file as follows.

   263 

   264   1. The file has multiple lines of test data.

   265   2. Each line in this file corresponds to a single test case.

   266   3. Each line consists of three comma separated coloumns:

   267 

   268      i. First two coloumns are the integers for which the GCD has to

   269         be computed

   270      ii. Third coloumn is the expected GCD to the preceding two

   271          numbers.

   272 

   273 So how do we write our tests to use these test cases? Pretty simple, let

   274 us review the machinery required first.

   275 

   276   1. File reading: We already have learnt this in the modules on

   277      Basic Python.

   278   2. Parsing the read data from the file: This just involves a using a

   279      **for** loop which iterates over the data line by line since we

   280      know that the file contains each test case as a sepate line which

   281      are equivalent to the file records and hence parse the data line

   282      by line as strings as we iterate over it and convert it to the

   283      required data type.

   284 

   285 Since we already have all the machinery required, let us proceed writing

   286 our test cases. We do not need not make any changes to the gcd

   287 function so we will just write down the test here. Let us call our

   288 data file gcd_testcases.dat::

   289 

   290   if __name__ == '__main__':

   291       for line in open('gcd_testcases.dat'):

   292           values = line.split(', ')

   293           a = int(values[0])

   294           b = int(values[1])

   295           g = int(values[2])

   296 

   297           tc = gcd(a, b)

   298           if tc != g:

   299               print "Test failed for the case a=%d and b=%d. Expected %d. Obtained %d instead." % (a, b, g, tc)

   300               exit(1)

   301 

   302       print "All tests passed!"

   303 

   304 When we execute the gcd.py script again we will notice that all the

   305 tests passed.

   306 

   307 Python Testing Framework

   308 ========================

   309 

   310 Python provides two ways to test the code we have written. One of them

   311 is the unittest framework and the the other is the doctest module.

   312 

   313 doctest

   314 ~~~~~~~

   315 

   316 To start with let us discuss the doctest module. As we have already

   317 discussed a well written piece of code must always be accompanied by

   318 its documentation. For a function or a module we document them in their

   319 respective docstrings. In addition to this, we can also place the

   320 samples of using these functions or modules in the Python interactive

   321 interpreter in the docstrings. When we run the doctest module it picks

   322 up all such interactive session samples, executes them and determines

   323 if the documented piece of code runs as it is documented. Let us see

   324 how to write doctests for our gcd function::

   325 

   326   def gcd(a, b):

   327       """Returns the Greatest Common Divisor of the two integers

   328       passed as arguments.

   329 

   330       Args:

   331         a: an integer

   332         b: another integer

   333 

   334       Returns: Greatest Common Divisor of a and b

   335 

   336       >>> gcd(48, 64)

   337       16

   338       >>> gcd(44, 19)

   339       1

   340       """

   341       if b == 0:

   342           return a

   343       return gcd(b, a%b)

   344 

   345 This is all a doctest is. To explain it in more simple terms tests

   346 which are written as part of the docstrings are called as

   347 doctests. Now how do we use our doctest module to execute this

   348 tests. That is fairly straight forward as well. All we need to do is

   349 tell the doctest module to execute. Let us place this piece of code at

   350 the same place where we placed our tests earlier. So putting all these

   351 together we have our gcd.py module which looks as follows::

   352 

   353   def gcd(a, b):

   354       """Returns the Greatest Common Divisor of the two integers

   355       passed as arguments.

   356 

   357       Args:

   358         a: an integer

   359         b: another integer

   360 

   361       Returns: Greatest Common Divisor of a and b

   362 

   363       >>> gcd(48, 64)

   364       16

   365       >>> gcd(44, 19)

   366       1

   367       """

   368       if b == 0:

   369           return a

   370       return gcd(b, a%b)

   371 

   372   if __name__ == "__main__":

   373       import doctest

   374       doctest.testmod()

   375 

   376 All we need to do is import the doctest module that is part of the

   377 Python's standard library. Call the testmod() function in this

   378 module. This function automatically checks for all the docstrings that

   379 have sample sessions from the interactive interpreter, if they exist

   380 it executes them and compares the output with the results as specified

   381 in the sample sessions. It complains if the results don't match as

   382 documented. When we execute this script as a stand-alone script we

   383 will get back the prompt with no messages which means all the tests

   384 passed::

   385 

   386   $ python gcd.py

   387   $ 

   388 

   389 If we further want to get a more detailed report of the tests that

   390 were executed we can run python with -v as the command line option

   391 to the script::

   392 

   393   $ python gcd.py -v

   394   Trying:

   395       gcd(48, 64)

   396   Expecting:

   397     16

   398   ok

   399   Trying:

   400       gcd(44, 19)

   401   Expecting:

   402       1

   403   ok

   404   1 items had no tests:

   405       __main__

   406   1 items passed all tests:

   407      2 tests in __main__.gcd

   408   2 tests in 2 items.

   409   2 passed and 0 failed.

   410   Test passed.

   411 

   412 

   413 **Note:** We can have the sample sessions as test cases as long as the

   414 outputs of the test cases do not contain any blank lines. In such

   415 cases we may have to use the exact string *<BLANKLINE>*

   416 

   417 For the sake of illustrating a failing test case, let us assume that

   418 we made a small mistake in our code. Instead of returning **a** when b

   419 = 0 we typed it as return b when b = 0. So all the gcds returned will

   420 have the value of 0 in such a piece of code. The code looks as

   421 follows::

   422 

   423   def gcd(a, b):

   424       """Returns the Greatest Common Divisor of the two integers

   425       passed as arguments.

   426 

   427       Args:

   428         a: an integer

   429         b: another integer

   430 

   431       Returns: Greatest Common Divisor of a and b

   432 

   433       >>> gcd(48, 64)

   434       16

   435       >>> gcd(44, 19)

   436       1

   437       """

   438       if b == 0:

   439           return a

   440       return gcd(b, a%b)

   441 

   442 Executing this code snippet without -v option to the script::

   443 

   444   $ python gcd.py

   445   **********************************************************************

   446   File "gcd.py", line 11, in __main__.gcd

   447   Failed example:

   448       gcd(48, 64)

   449   Expected:

   450       16

   451   Got:

   452       0

   453   **********************************************************************

   454   File "gcd.py", line 13, in __main__.gcd

   455   Failed example:

   456       gcd(44, 19)

   457   Expected:

   458       1

   459   Got:

   460       0

   461   **********************************************************************

   462   1 items had failures:

   463      2 of   2 in __main__.gcd

   464   ***Test Failed*** 2 failures.

   465 

   466 The output clearly complains that there were exactly two test cases

   467 that failed. If we want a more verbose report we can pass -v option to

   468 the script. This is pretty much about the doctest module in

   469 Python. doctest is extremely useful when we want to test each Python

   470 function or module individually. For more information about the

   471 doctest module refer to the Python library reference on doctest[0].

   472 

   473 unittest framework

   474 ~~~~~~~~~~~~~~~~~~

   475 

   476 Not too far ahead we go we, we will start complaining that the doctest

   477 is not sufficient to write complicated tests especially when we want

   478 to automate our tests, write tests that need to test for more

   479 convoluted code pieces. For such scenarios Python provides a unittest

   480 framework.  unittest framework provides methods to efficiently

   481 automate tests, setup and teardown functionalities which helps to

   482 setup the initializing code and data for executing the specific tests

   483 and cleanly shutting them down once the tests are executed and ways to

   484 aggregate tests into collections and better way of reporting the

   485 tests.

   486 

   487 Let us continue testing our gcd function in the Python module named

   488 gcd.py. To get ourselves started, the unittest framework expects us to

   489 subclass TestCase class in unittest module and place all our test code

   490 as methods of this class. We will begin the name of the test method

   491 with **test_** so that the test runner knows which methods are to be

   492 executed as tests. We will use the test cases supplied by

   493 gcd_testcases.dat. Lastly, to illustrate the way to test Python code

   494 as a module let create a new file called test_gcd.py following the

   495 same convention used to name the test methods. We will place our test

   496 code within test_gcd.py module. Our test code looks like this::

   497   

   498   import gcd

   499   import unittest

   500 

   501   class TestGcdFunction(unittest.TestCase):

   502 

   503       def setUp(self):

   504           self.test_file = open('gcd_testcases.dat')

   505           self.test_cases = []

   506           for line in self.test_file:

   507               values = line.split(', ')

   508               a = int(values[0])

   509               b = int(values[1])

   510               g = int(values[2])

   511 

   512               self.test_cases.append([a, b, g])

   513 

   514       def test_gcd(self):

   515           for case in self.test_cases:

   516               a = case[0]

   517               b = case[1]

   518               g = case[2]

   519               self.assertEqual(gcd.gcd(a, b), g)

   520 

   521       def tearDown(self):

   522           self.test_file.close()

   523           del self.test_cases

   524 

   525   if __name__ == '__main__':

   526       unittest.main()

   527 

   528 Since we don't want to read this file into memory each time we run a

   529 separate test method, we will read all the data in the file into

   530 Python lists in the setUp method. The entire data file is kept in a

   531 list called test_cases which happens to be an attribute of the

   532 TestGCDFunction class. In the tearDown method of the class we

   533 will delete this attribute to free up the memory and close the

   534 opened file.

   535 

   536 Our actual test code sits in the method which begins with the name

   537 **test_** as said earlier, the test_gcd method. Note that we import

   538 the gcd Python module we have written at the top of this test file and

   539 from this test method we call the gcd function within the gcd module

   540 to be tested with the each set of **a** and **b** values in the

   541 attribute test_cases. Once we execute the function we obtain the

   542 result and compare it with the expected result as stored in the

   543 corresponding test_cases attribute using the assertEqual methods

   544 provided by our parent class TestCase in the unittest framework. There

   545 are several other assertion methods supplied by the unittest

   546 framework. For a more detailed information about this, refer to the

   547 unittest library reference at [1].

   548 

   549 nose

   550 ====

   551 

   552 Now we know almost all the varities of tests we may have to use to

   553 write self-sustained, automated tests for our code. There is one last

   554 thing that is left. However one question remains, how do we easily

   555 organize choose and run the tests that is scattered around several

   556 files? 

   557 

   558 To further explain, the idea of placing tests with in the Python

   559 scripts and executing that test scripts themselves as stand-alone

   560 scripts works well as long as we have our code in a single Python file

   561 or as long as the tests for each script can be run separately. But in

   562 a more realistic software development scenario, often this is not the

   563 case. The code is spread around multiple Python modules and may be

   564 even across several Python packages.

   565 

   566 In such a such a scenario we wish we had a better tool to

   567 automatically aggregate these tests and execute them. Fortunately for

   568 us there exists a tool called nose. Although nose is not part of the

   569 standard Python distribution itself, it can be very easily installed

   570 by using easy_install command as follows::

   571 

   572   $ easy_install nose

   573 

   574 Or download the nose package from [2], extracting the archive and

   575 running the command from the extracted directory::

   576 

   577   $ python setup.py install

   578 

   579 Now we have nose up and running, but how do we use it? It is very

   580 straight forward as well. We will use the command provided by nose

   581 called as nosetests. Run the following command in the top level

   582 directory of your code::

   583 

   584   $ nosetests

   585 

   586 Thats all, nose automatically picks all the tests in all the

   587 directories and subdirectories in our code base and executes them

   588 all. However if we want to execute specific tests we can pass the test

   589 file names or the directories as arguments to nosetests command. For a

   590 detailed explanation about this, refer to [3]

   591 

   592 Conclusion

   593 ==========

   594 

   595 Now we have all the trappings we want to write state-of-the art

   596 tests. To emphasize the same point again, any code which was written

   597 before writing the test and the testcases in hand is flawed by

   598 design. So it is recommended to follow the three step approach while

   599 writing code for any project as below:

   600 

   601   1. Write failing tests with testcases in hand.

   602   2. Write the code to pass the tests.

   603   3. Refactor the code for better performance.

   604 

   605 This approach is very famously known to the software development world

   606 as "Red-Green-Refactor" approach[4].

   607 

   608 

   609 [0] - http://docs.python.org/library/doctest.html

   610 [1] - http://docs.python.org/library/unittest.html

   611 [2] - http://pypi.python.org/pypi/nose/

   612 [3] - http://somethingaboutorange.com/mrl/projects/nose/0.11.2/usage.html

   613 [4] - http://en.wikipedia.org/wiki/Test-driven_development