Find us on GitHub

Teaching basic lab skills
for research computing

Systems Programming: Manipulating Files and Directories

Systems Programming/Manipulating Files and Directories at YouTube
media 001

Hello and welcome to the fifth episode of the Software Carpentry lectures on handling files and directories in Python. We'll now look at creating and deleting directories, and moving directories and files around.

media 002

Let us assume we are in an empty directory called user. And we want to create one called data. For this we use the mkdir function from the os module.

media 003

media 004

We can use listdir to see that we indeed have a new directory called data.

media 005

And, that this new directory is empty.

media 006

Now, if we try to use mkdir again...

media 007

...we get an error since it already exists.

media 008

Suppose we want to make a nested set of directories for a more complex organisation of files. Let's give that a try.

media 009

It fails.

media 010

But, no matter, as we can use the makedirs function.

media 011

This will create all the directories in the path if they don't already exist.

media 012

Rmdir allows us to remove directories. So let's remove towns.

media 014

Now let's try and remove country and its children.

media 015

This fails. Rmdir only removes empty directories and is not recursive.

media 016

But there is a removedirs function that operates recursively. This removes each directory in the path up to and including the first one mentioned.

media 017

So it removes regions.

media 018

Then country.

media 019

But removedirs only works if the directories are empty. Suppose we had some data files in regions. Then if we call removedirs...

media 020

...we'd get an error. Removedirs cannot remove directories that have files.

media 021

But rmtree in the shutil module can.

media 023

Now that we've mentioned it, what about deleting files? Well we could use rmtree on individual files too. But there is the simpler, remove function. So, if we had two files in data and wanted to remove one.

media 024

We just use remove.

media 025

We can rename and move directories and files too using the rename function. We can rename a directory...

media 027

Note that the destination directory must not exist. If it does exist we get an error.

media 028

media 029

To keep the same directory name after the rename, we must provide it explicitly.

media 030

media 031

Rename also renames files.

media 032

media 033

Unlike for directories, a destination file name does not need to be given.

media 034

As, in that case, the source file name will be used.

media 035

Alternatively, we can rename files and directories using the shutil module's move function. This is more powerful. Asides from moving the file or directory, it also...

media 036

Preserves permission bits, the group and owner.

media 037

The last access and modification times.

media 038

And other flags.

media 039

Renames behaves like both rename and makedirs.

media 040

In that it creates any intermediate directories, that do not already exist, like D.

media 041

We can copy directories via copytree from shutil.

media 042

This copies the entire contents of the directory recursively. Like move, it preserves permission bits, groups and owners, last access and modification times and other flags.

media 043

We can copy files via copyfile from shutil.

media 044

media 045

Like move and rename for directories...

media 046

...a destination file name must always be given

media 047

Shutil's copy function also copies files.

media 048

It is like copyfile but we don't have to provide a destination file name.

media 049

It also copies the existing file permissions.

media 050

Shutil also has a copy2 function.

media 051

This preserves permission bits, groups and owners, last access and modification times and other flags. Copytree, which we saw earlier, uses copy2.

media 052

In this episode we have seen how to create, move, copy and delete directories and files using a variety of functions from the os and shutil modules.

media 053

Thank you for listening.