OddThinking

This is a message from the Julian of 2010, to any Julian of the future who is starting out a new software project.

Message begins: Doctest is a interesting and clever testing harness built right in to Python. Try not to use it. Message ends.

On second thoughts, maybe it is a mistake to assume that messages to the future have limited bandwidth. I probably have room to explain further.

As you know, Python allows you to annotate each module, class and method with a “docstring” – a special multi-line comment which describes it, and is available to various meta-tools.

One way to describe a function is to provide examples on how to call it and what you might expect when it returns.

Here’s a snippet from the example in the doctest documentation:

def factorial(n):
    """Return the factorial of n, an exact integer >= 0.

    If the result is small enough to fit in an int, return an int.
    Else return a long.

    >>> [factorial(n) for n in range(6)]
    [1, 1, 2, 6, 24, 120]
    >>> [factorial(long(n)) for n in range(6)]
    [1, 1, 2, 6, 24, 120]
	
    [snip]"""

Doctest is built around an interesting idea.

Doctest offers a way of testing these examples. Call doctest.testmod() and it will examine the docstrings in the current module, looking for specially formatted examples (note the >>> prefix), execute the code and check it against the provided example results.

if __name__ == '__main__':
    import doctest
    doctest.testmod()

So doctest is a great way to check your code’s comments don’t get out of date. Well, one very limited aspect of your code’s comments. If that’s all you are using it for, that’s fine.

But, Julian, it’s not how you use it, is it? For simple functions with just two test cases, and two examples, why would you bother creating a whole unit-test, when you have everything you need already there?

Another test case? Why not just throw it in as another example, rather creating a separate unit-test. Two more? How confusing could it be?

Oh what’s that? You now have ten examples of how to call your code and it turns out it can be very confusing? That’s okay, because doctest has a great facility to extend testing into a separate file. Put all your examples in a file called “examples.doctest” and call doctest.testfile("examples.doctest").

Oh, don’t forget you’ll need to add an “import” statement, to load your original module… and probably more for all the types that appear in function parameters. But still, how about that! A really quick unit-test framework, for practically no effort!

Except that every time you try to execute the current document in your IDE, it will get shirty. Windows doesn’t know how to run a doctest, and nor does Python. You need to find the code file being tested, and run that instead.

After you have made that mistake 50 times on any a particular file, you’ll be tempted to hack around it. Put """ marks around the whole doctest file to make it a docstring, append the magic idiom for unit-tests, above. Then some magic cookies to the top for your Unix code, or teach your editor to run it as a Python file in Windows, or even rename it with a .py extension. Now you have a fully executable test code.

Oh, except every reference to the newline character, \n (or other special characters) needs to be double-slashed as \\n, but only when it is executed as testmod (when it is treated as a text file), and not as testfile (when it is treated as a docstring) so good luck working around that.

More importantly, look at what you’ve done here.

You’ve started using a new language. It is similar to Python, but not quite. Every line needs to be prefixed with >>> or ... to show it should be executed, the escaping rules are different, and, most importantly, all your fancy language-based tools can’t figure it out.

No more syntax-colouring.

No more more autocompletion.

No more pylint suggestions.

All that, just to avoid using a real unit-testing framework?

Not worth it. Just not worth it.

So, dear Julian of the future (sorry, may I call you Julian, or should I be addressing you as Mr President?), please don’t do that again, okay?

Oh, and if you are going to reply to this message, please remember to include this week’s lotto numbers. Thanks.