Hello, and welcome to the ninth episode of the Software Carpentry lecture on Python. In this episode, we'll take a closer look at some things you can do with functions that will allow you to do more with less code.

As we've seen in previous episodes, an integer is just 32 bits of data…

…that a variable can refer to.

And a string is just a sequence of bytes…

…that variables can also refer to.

Well, it turns out that a function is just another sequence of bytes—ones that happen to represent instructions…

…and yes, variables can refer to them to.

This insight—the fact that code is just another kind of data, and can be manipulated like integers or strings—is one of the most useful and powerful in all computing.

To understand why, let's have a closer look at what actually happens when we define a function.

These two lines of code tell Python that `threshold`

is a function that returns one over the sum of the values in `signal`

.

When we define it, Python translates the statements in the function into a blob of bytes, then creates a variable called `threshold`

and makes it point at that blob.

This is not really any different from assigning the string `'alan turing'`

to the variable `name`

: the only difference is what's in the memory the variable points to.

Well, if `threshold`

is just a reference to a value in memory, we should be able to assign that reference to another variable.

Here's our starting point once again.

And here's the assignment: `t = threshold`

. As you can see, `t`

now points to the same data as `threshold`

: it is an alias for the function.

To prove this is so, let's try calling `t`

. The result is exactly what we would get if we called `threshold`

with the same parameters, because `t`

and `threshold`

*are* the same function.

Here's another thought: if the function is "just" data, can we put a reference to it in a list?

Let's define two functions, `area`

and `circumference`

, each of which takes a circle's radius as a parameter and returns the appropriate value.

Once those functions are defined, we can put them into a list like this. Of course, what we really mean is we're copying the references to the functions stored in the variables `area`

and `circumference`

into the list, so that its first and second elements refer to the same blobs of instructions.

We can now loop through the functions in the list, calling each in turn.

Sure enough, the output is what we would get if we called `area`

and then `circumference`

with the parameter 1.0.

Let's go a little further. Instead of storing a reference to a function in a list, let's pass that reference into another function, just as we would pass a reference to an integer, a string, or a list.

Here's a function called `call_it`

that takes two parameters: a reference to some other function, and some other value. All `call_it`

does is call that other function with the given value as a parameter.

Let's test it with `area`

and 1.0: that's right.

And with `circumference`

and 1.0: right again.

So far, so pointless, but now it's time for the payoff: functions of functions.

Here's a function called `do_all`

that, as its name suggests, applies some function—anything at all that takes one argument—to each value in a list, and returns a list of the results.

If we call `do_all`

with `area`

and a list of numbers…

…we get what we would get if we called `area`

directly on each number in turn.

And if we define a function to "slim down" strings of text by throwing away their first and last characters…

…we can apply it to every string in a list, without having to copy the code that loops through the list, calls the function, and concatenates the results.

Functions that operate on other functions are called *higher-order functions*. They're common in mathematics: integration, for example, is a function that takes some other function and two endpoints as parameters. In programming, higher-order functions allow us to re-use control flow rather than rewriting it.

Here's another example. `combine_values`

takes a function and a list of values as parameters, and "adds up" the values in the list using the function provided, returning a single value as its result.

To show how this works, let's define `add`

and `mul`

to add and multiply values.

If we combine 1, 3, and 5 with `add`

, we get their sum, 9.

If we combine the same values with `mul`

, we get their product, 15. This same higher-order function `combine_values`

could concatenate lists of strings, too, or multiply several matrices together, or whatever else we wanted, without us writing the loop logic ever again.

Higher-order functions are a good thing because they let us do more with less code. If we don't use higher-order functions…

…then we have to write one function for each combination of data structure and operation, i.e., one function to add numbers, another to concatenate strings, a third to sum matrices…

…and so on.

*With* higher-order functions, on the other hand…

…we only write one function for each basic operation, and one function for each kind of data structure…

…and since A plus B is usually a lot smaller than A times B, this saves us coding, testing, and debugging.

Of course, we have to know *something* about the function our higher-order function is operating on…

…like how many arguments it takes.

But in Python and many other languages, we can even get around that.

Here's a function called `add_all`

that sums up values using plus.

Notice the '*' in front of the parameter `args`

. This tells Python to take all of the parameters passed into the function when it's called and put them together in a special type of list called a tuple, which we'll explore in more detail in a couple of lectures.

If we call `add_all`

with no arguments, the tuple that's assigned to `args`

has no elements, so `add_all`

returns 0.

If we call `add_all`

with the integers 1, 2, and 3, though, `args`

has three elements, which `add_all`

sums up to get 6.

We can use this "catch-all" parameter with regular parameters, as long as the catch-all comes last.

Here, for example, is another version of `combine_values`

.

The only difference between it and the previous version is the '*' in front of the parameter `values`

.

This small change means that we don't have to put the values we want to combine into a list before calling the function: the first actual parameter is assigned to `func`

as before, and everything else goes into the tuple `values`

.

As an aside before we finish this episode, what do you think `combine_values`

will do if we only provide a function, and no values for that function to operate on?

More importantly, what do you think it *should* do?

Several higher-order functions are actually built in to Python. One is `filter`

, which constructs a new list containing all the values in an original list for which some function is true.

Another is `map`

, which applies a function to every element of a list, returning a list of results…

…and then there's `reduce`

, which combines values using a binary function, returning a single value as a result.

For example, if `positive`

is True when its argument is greater than or equal to 0, then `filter`

of `positive`

and a list of numbers returns a list of non-negative numbers.

If `negate`

changes the sign of its argument, `map`

of `negate`

returns a list of negated values…

…and of course, using `reduce`

with `add`

returns the sum of the values in the list.

It usually takes a while to get used to working with higher-order functions, but while the ideas are digesting, let's step back and ask, "What *is* programming anyway?"

Novices usually think that it means writing instructions for a computer.

But more experienced programmers think of it as creating and combining abstractions.

When we're programming, our goal is to spot a pattern, like "combine all elements of a list using a binary function".

And then write it down once, as clearly as possible…

…so that we can build more patterns on top of it.

Be cautious, though: the limits of human short-term memory still apply.

If you pile too many abstractions on top of one another, it can be difficult to figure out what the end result actually does.

Thank you.