Hello, and welcome to the fourth episode of the Software Carpentry lecture on Make. In this episode, we'll see how to write rules that capture general patterns in dependencies and rules.
As in previous episodes, we're looking at how to manage tasks and dependencies automatically using Make.
For the paper the robot is working on, paper.pdf depends on paper.wdp, figure-1.svg, and figure-2.svg, while figure-1.svg depends on summary-1.dat, which in turn depends on all the files with names like data-1-1.dat, data-1-2.dat, and so on.
In the previous episode, we saw how to use automatic variables and wildcards to write rules to handle any number of files with names matching certain patterns.
We still have a lot of redundancy in our Makefile, though. The rules for figure-1.svg and figure-2.svg are identical except for the '1' and '2' in their names, as are the rules for summary-1.dat and summary-2.dat.
Here's the Makefile so far.
We'd like to "fold" the rules for the figures together for two reasons. First, if we add a third figure, we don't want to have to duplicate rules a third time. Second, if we ever want to change the way we generate figures, we'd like to make that change once, in one place: if we have to make it in several places, the odds are good we'll forget one, and then waste time trying to figure out why some of our commands aren't running.
The way to do this in Make is to use a pattern rule to capture the common idea.
Here's our Makefile rewritten to use such a rule.
In this rule, % is a wildcard.
When it is expanded, it has the same value on both sides of the rule, i.e., if it matches '1' on the left, it must match '1' on the right as well.
% only means something to Make, though: it doesn't have a value in the rule's action, which is handed off to the shell for execution.
So in the action, we have to use the automatic variables $@ and $^ as before.
Let's try running our modified Makefile.
Hm: summary-1.dat is updated, but not summary-2.dat or either of the figure files.
Why didn't our other commands run?
The reason is that pattern rules don't create dependencies: they just tell Make what to do if there's a dependency.
In other words, if Make decides it wants to create figure-1.svg, it can use our pattern rule…
…but we still have to tell Make to care about figure-1.svg.
Let's do this by putting the rule for paper.pdf back in our Makefile.
Here's the full Makefile. In it, paper.pdf depends on figure-1.svg and figure-2.svg.
Make now knows that it needs these figures. Since there aren't specific rules for them, it uses the pattern rule instead.
It's tempting to go one step further, and make paper.pdf depend on figure-*.svg, but this doesn't work.
The reason is that the figure files may not exist when Make starts to run—after all, Make creates them. In that case, figure-*.svg will expand to nothing, so Make would mistakenly believe that paper.pdf depended only on paper.wdp. This kind of bug can be very hard to figure out, and unfortunately, Make doesn't come with a debugger to help you track these problems down.
Our raw data files do always exist, though, so we can get rid of some more redundancy by taking these two rules…
…and folding them into one using the * wildcard. It's safe to do this because Make isn't responsible for creating data-1-whatever.dat and data-2-whatever.dat: there's no possibility of the * missing things because it's evaluated when Make starts running.
Just as a reminder, the % is a Make wildcard: it matches the same thing on the left and right side of a pattern rule.
* is a shell wildcard: it matches zero or more characters in a filename when it's evaluated.
Can we get rid of the last bit of redundancy by making summary-%.dat depend on stats.py? No—this doesn't work.
Even with this pattern rule, the summary files only depend on the corresponding raw data files, not on stats.py.
Why? Because when Make sees two or more pattern rules that could match a filename, it uses the first and ignores the other. It's another wart, and another source of hard-to-find headaches in Makefiles.
If we really want to avoid making summary-1.dat and summary-2.dat depend on stats.py separately, the only way is to go back to the false dependencies we introduced in the previous episode. This Makefile tells Make to update the timestamps on the raw data files using touch whenever stats.py changes. This indirectly triggers the re-creation of the summary files—it does what we want, just in a roundabout way.
As useful as it is, Make is less than perfect…
In our next episode, we'll see how to define collections of files in Make, and how to get information in from the outside world.