Software Carpentry:

Hello, and welcome to the sixth episode of the Software Carpentry lecture on testing. In this episode, we'll have a closer look at how to set up fixtures to run tests on.

Let's go back to the fields of Saskatchewan.

We're trying to find places where photographs of those fields overlap.

Each photograph contains one or more rectangles.

In programming terms, this means that a photo is a collection of rectangles, such as a set or a list.

What we really want to do is find all the overlaps between the rectangles in two collections.

For example, if the first photo contains the three fields shown in yellow, and the second contains the two fields shown in green, our output should be the five rectangles shown in blue.

We've already tested a function called overlap_rect that finds the overlap between two rectangles.

We're confident it works, so we now want to test overlap_photo.

We don't actually need to know how it works, but we imagine that its implementation looks something like this:

The two loops get rectangles from the first and second photo respectively, so that we're comparing all against all.

Inside the loop, we compare those rectangles and save their overlap if it's non-empty.

Here's our first test: a photo containing only the unit square compared with itself. The output should be just the unit square.

And here's how we turn that test into code.

Six lines of code: that's not too bad.

OK, let's write a second test: the unit square against a checkerboard pattern containing four squares. Again, the output should contain only the unit square.

Again, it's only six lines of code…

…but those six lines are harder to read.

We could introduce a variable called unit so that we don't write down the unit square's coordinates twice, but it doesn't really help much.

Here's our third test: a short and wide rectangle against the checkerboard, with two squares as output.

Again, it's only six lines of code…

…but it isn't particularly easy to read…

…and we can now see a new problem: there's too much duplicated code. We've now defined the unit square four or five times, and the checkerboard twice. Duplication is almost always a sign that a program could be simplified; the only question is how.

In the case of testing, the answer is simple: let's create our fixtures outside the tests, so that we only have to define them once, and many different tests can share them.

(Remember, a fixture is something that we run a test on—in this case, our photos.)

This is a common need, so like other testing libraries, Nose can help us out. If a file contains a function called setup, Nose will automatically run that function before it runs any of the tests in the file.

Here's an example showing how it works.

If we were actually testing, setup would define the fixtures our tests used, but we'll have it print to standard error so that we can see when it runs.

Similarly, test_1 and test_2 would use those fixtures to do some testing, but we'll have them print as well.

When we run this file with the nosetests command, it produces this output.

The stuff shown in blue is Nose's regular output; our print statements are interleaved with it, which makes things a bit hard to read.

This is what Nose's regular output would look like without our print statements.

As you can see, Nose ran setup once at the start…

…and then ran our two tests (in some arbitrary order—remember, Nose gets to choose).

All right, let's write a setup function to create some fixtures for testing photo overlap.

Here's the code.

We start by creating a global variable called Photos to hold our fixtures. We use a global variable, outside any particular function, so that both setup and our individual tests can see it.

Inside setup, we create sets of rectangles and store them in Photos.

Now let's use those fixtures in some tests.

Here's the overlap of the unit square with itself.

And here's the overlap of our short and wide rectangle with the checkerboard.

We don't have to put all our fixtures in one variable, of course—we could create a bunch of global variables, and store one fixture in each.

Here's the code to do that. Notice that we have to assign each global variables some value just to create it.

Which you use is a matter of personal taste and project style.

Of course, in this particular case you don't actually need a setup function at all—you can assign fixtures to Unit and Short_And_Wide when they're first defined.

However, when you're working with more complicated programs, creating fixtures can require many lines of code, and calls to helper functions, and bundling all of that into a setup function is tidier.

And even when you can do everything in one go, there's still another good reason to use a setup function. What happens if some or all of your tests modify the fixtures they run on?

For example, suppose we're testing a function called photo_crop that removes rectangles lying completely outside some cropping window.

Since our tests modify our fixtures, we can't use the same fixtures over and over in separate tests—if we did, a bug in one place would contaminate the results of tests that were run later.

The solution is to re-create the fixtures for each test, i.e., to run setup over again just before each test function.

There are several ways to do this in Nose (and in other libraries), but the simplest to type is to use a decorator.

Here's what the code looks like.

We import the decorator, called with_setup, from the Nose library, just as if it were a function…

…because it actually is a function, just one that behaves in a special way.

We then put @decorator—in this case, @with_setup—right before the definition of each function that we want to apply it to. This decorator takes an argument—the name of the setup function to run before the test—so we pass that to the decorator just like an argument to a function call.

Thanks to a bit of magic that we won't go into here, doing this tells Nose to run the setup_each function right before test_1, and again before test_2. If we wanted to run different setup functions before the two tests, we would simply pass different function names to with_setup.

Here's the output when we run this program with nosetests.

Once again, the standard Nose output in blue shows that two tests were run successfully.

If we look at the output from our own print statements, we can see that Nose ran setup_each right before it ran test_1…

…and ran it again before test_2, just as we wanted.

Here's an example showing setup per test that actually does some testing. (We've left out the definition of create_fixtures because you've probably seen enough rectangles by now.

When we run this program, Nose calls create_fixtures to create the first copy of the checkerboard fixture…

…then runs test_crop_unit, which modifies that checkerboard.

Nose then runs create_fixtures again, assigning a fresh photo to checkerboard…

…and then runs test_crop_keep_everything, which modifies that copy in turn, and so on.

Re-running our setup function over and over again does waste a few microseconds of the computer's time.

But that's much less valuable than any of yours. And if you ever decide that creating a thousand fixtures, but only using one or two, really is too expensive, you can always break fixture creation up into several functions, and use the decorator to call only the ones you need for a particular test.