Batch rename files by finding and replacing terms in the filename

I had a set of files that have a common naming scheme and I wantd to replace a word common to all the filenames with another word. Unix-style shell utilities make it easy to do this sort of batch rename operation by finding and replacing patterns in filenames. Windows users can gain access to these powerful programs by installing Cygwin. In this short tutorial I will explain how you can do a batch rename of all files in the current directory by replacing a specific common word in the filenames with another word. I have only tested this on Windows with Cygwin, but it should be very similar on Linux or Unix.

If you want to just replace the file name extension, then see my tutorial on batch renaming file extensions.

Problem statement

I had a number of pictures from my digital camera that are named like so:

IMG_2378.JPG
IMG_2379.JPG
IMG_2380.JPG

I wanted to do a batch rename to replace “IMG_” with “Fireworks”:

Fireworks2378.JPG
Fireworks2379.JPG
Fireworks2380.JPG

Method

To perform a batch rename, the basic command looks like this:

for filename in *foo*; do echo mv \"$filename\" \"${filename//foo/bar}\"; done > rename.txt

The command works as follows:

  1. The for loop goes through all files with name *foo* in the current directory.
  2. For each filename, it constructs and echoes a command of the form mv “filename” “newfilename”, where the filename and new file name are surrounded by double quotes (to account for spaces in the file name) and the new file name has all instances of foo replaced with bar. The substitution function ${filename//foo/bar} has two slashes (//) to replace every occurrence of foo with bar.
  3. Finally, the entire output is saved to rename.txt for user review to ensure that the rename commands are being generated correctly.

If you are confident that the command did not mess anything up, then you can directly pipe it directly to /bin/bash:

for filename in *foo*; do echo mv \"$filename\" \"${filename//foo/bar}\"; done | /bin/bash

Or, you can have Bash execute the mv command directly (note that the double quotes are no longer escaped with a \):

for filename in *foo*; do mv "$filename" "${filename//foo/bar}"; done

I do not recommend using either of these direct execution options without checking the output first.

If no files are found matching the *foo*, the loop still executes but the generated command is wrong. Therefore, always check the output before using it.

Example

The image renaming example at the beginning of this page can be now be solved. In the following code blocks, the command after the $ character is what I typed into the Bash prompt. First, I can list the files in the directory:

$ ls -1 IMG_*
IMG_2378.JPG
IMG_2379.JPG
IMG_2380.JPG

Next, I can modify the batch renaming command for this renaming job:

$ for filename in IMG_*; do echo mv \"$filename\" \"${filename//IMG_/Fireworks}\"; done
mv "IMG_2378.JPG" "Fireworks2378.JPG"
mv "IMG_2379.JPG" "Fireworks2379.JPG"
mv "IMG_2380.JPG" "Fireworks2380.JPG"

Finally, by piping the above output into /bin/bash, the commands can be executed and the files are renamed.

Alternative methods

Here are some altnative methods that I have used in the past, each with a few weaknesses that need to be taken into account. I want to document them here because they are instructive.

Batch rename with awk and sed

This was my first workable attempt at doing batch renaming:

ls -1 *foo* | awk '{print("mv "$1 " " $1)}' | sed 's/foo/bar/2' > rename.txt

The command works as follows:

  1. ls -1 *foo* lists all the files in the current directory with foo in the file name. It lists one filename per line.
  2. The ouptut is piped to awk '{print("mv "$1 " " $1)}' command. This produces new output where each line is mv FILENAME FILENAME, with FILENAME being the corresponding filename.
  3. The output from the awk command is piped to sed 's/foo/bar/2', which replaces the second instance of foo in a line with bar. The second instance of foo corresponds to the second FILENAME in the mv FILENAME FILENAME lines generated by the awk command. This creates output of the form mv FILENAME NEWFILENAME, where the new filename is desired filename with foo replaced with bar.
  4. Finally, the entire output is saved to rename.txt for user review to ensure that the rename commands are being generated correctly. As with the main batch rename technique, you could pipe the output to /bin/bash but this is not recommended.

This technique has some caveats. The first is that since it does not work with file names with spaces. The awk command prints the first field $1 in each line of input, and the spaces in the file name causes the first word of the filename to be treated as the entire first field.

The second is that since sed replaces the second instance of foo in a line, it does not handle the case where the file name had multiple instances of foo in the first place. For example, if you had a file named “foofoo.jpg”, then the corresponding command generated by the above code would be:

mv foobar.jpg foofoo.jpg

The second foo is replaced with bar, which is not what you want. Thus, this technique does not work when there is more than one instance of the word you want to replace in the filename.

Batch rename with paste and sed

An alternative I tried to get around the limitation in the previous technique of only one instance of the search term being allowed to occur per file name:

paste <(ls -1 *foo* | sed 's/^/mv "/;s/$/"/') <(ls -1 *foo* | sed 's/^/"/;s/$/"/;s/foo/bar/g') -d ' ' > rename.txt

How it works:

  1. The commands in <( … ) are processed first. This is a process substitution in Bash and the output of the command inside is given to paste.
  2. The first process substitution lists all files named *foo* and sed prepends mv ” and appends a double quote to the end. This surrounds the file name with double quotes to take care of spaces.
  3. The second process substitution again lists all the files named *foo* and then uses sed to surround the file name with double quotes and replace “foo” with “bar” in the file name.
  4. The paste command concatenates the output of the two process substitutions line by line, and uses a space as a delimiter. This produces a list of mv commands that can be saved to a file or piped to /bin/bash for execution.

I do not see too many weaknesses in this technique but it seems cryptic and perhaps inelegant.

Discussion

Rose, 2011/03/31 09:41
Hi Peter,
Thanks for your example of renaming files. I only wanted to rename some music files. I found the fancy sed commands confusing and I found your simple suggestion (the first one) worked well. I especially appreciated how one can check this first with rename.txt, before renaming the files.

Rose
james santinello, 2012/03/18 20:56
Hi: i have a folder with 700 song titles.i wish to have them all in upper case.is that possible?
Peter Yu, 2012/03/19 02:08
James - A quick search found this to convert file names to uppercase:

for filename in *foo*; do echo mv \"$filename\" \"`echo "$filename"|awk '{print toupper($filename)}'`\"; done
David, 2012/07/05 20:31
Thank you for your example. This is just what I needed (the first option). It worked perfectly on my MacBook Pro.
Cygwin_user, 2012/10/10 14:41
The Windows command prompt had a renaming utility called "rename" or "ren." So you can do make cmd execute a command from your cygwin terminal as follows:

cmd /C rename \*foo\* \*bar\*



Note "cmd /C [windows_cmd_command]" makes cmd execute the cmd command that follows "/C," and that you need to escape the asterisks ("\*") to prevent your cygwin shell (bash by default) from doing the globbing before sending the command to cmd.
Peter Yu, 2012/10/25 02:22
That's interesting, but I wasn't able to get it to work properly. Sometimes it complains about "duplicate files" and sometimes it just tacks on the "bar", so that files named "foo0001.cr2" become "foo0001.cr2bar".
Roey, 2012/12/21 11:13
Hmm, with the first method I get an error that the file isn't found
e.g. `mv: cannot stat `"454Reads.HAL3W_2_L.sff"': No such file or directory`.
The `awk` and `sed` methods work great.

Thanks
Peter Yu, 2012/12/24 04:36
I've never seen that before. I'd do the first method and check the output of the rename.txt file to see if everything is output properly.
Roey, 2012/12/25 11:30
sorry my mistake, all works now...
Kooalia, 2013/06/27 00:37
for change winter_2013_festvial.jpg to-> summer_2013_festvial.jpg

On windows by cmd command dos, go in directory and type:

rename winter*. summer*.*
Eric C, 2014/01/08 20:22
For what it's worth, Cygwin has the rename command which would make this very easy to accomplish:

rename IMG_ Fireworks *

Syntax: rename <Characters You Don't Want> <Characters You Do Want> filenames (wildcards are acceptable)
I would love to hear your feedback. Enter your comment below [ Terms of Use ]:
USURX
 

About Peter Yu I am a research and development professional with expertise in the areas of image processing, remote sensing and computer vision. I received BASc and MASc degrees in Systems Design Engineering at the University of Waterloo. My working experience covers industries ranging from district energy to medical imaging to cinematic visual effects. I like to dabble in 3D artwork, I enjoy cycling recreationally and I am interested in sustainable technology. More about me...

Feel free to contact me with any questions about this site at [user]@[host] where [user]=web and [host]=peteryu.ca

Copyright © 1997 - 2014 Peter Yu