Edit and process text files with sed

The sed utility is very useful for manipulating and processing text files from the command line or in a shell script. Linux and Unix systems should have it installed already, while it can be downloaded for Windows. I use the version that comes with Cygwin. sed stands for “stream editor” and can perform complex operations on text data. This page consists of my own notes on how to use sed but keep in mind that the types of things sed can do go far beyond the examples here.

Since Windows and Unix text files differ in the line ending characters used, be aware that there can be some issues when the line ending in a file is not what is expected. The Cygwin version of sed expect and produce Unix line endings. To convert between Unix and Windows (DOS) line endings, use dos2unix and unix2dos, both tools that come with Cygwin.

Search and replace text in a file

sed can be used to find and replace text in a file using the following command:

sed -i 's/apples/oranges/g' file.txt

The command will change all instances of “apples” with “oranges” in file.txt. The option -i tells sed to edit the file “in-place”, which will save the modified result back to the same file name. If it is omitted, the result is output to the console.

The command between the single quotes is the search and replace command. s tells sed to search and replace, the text to find is apples, the substitution text is oranges and g means global search (replace all instances in each line of the file).

Regardless of whether sed finds the search term in the file, the file modification date will be changed.

Case insensitive matching

It is also possible to make the matching case insensitive:

sed -i 's/apples/oranges/gI' file.txt

This would match APPLES as well as apples and any other combinations of upper and lower case letters.

Prepend and append to each line

You can use sed to add some characters to the beginning of each line (prepend) in the input and add other characters after (append) each line. The following prepends the string PREPEND to each line of input:

sed 's/^/PREPEND/' filename.txt

The following appends the string APPEND to each line of input:

sed 's/$/APPPEND/' filename.txt

Since the sed processes the command you pass into it as a program (the commands are actually in the sed programming language), you can include more than one action in the same call to sed. The following will prepend and append the corresponding strings to each line of input:

sed 's/^/PREPEND/;s/$/APPPEND/' filename.txt

You can also pipe some text to sed. Or, use it to save directly back to filename.txt by using the in-place option -i.

Discussion

I would love to hear your feedback. Enter your comment below [ Terms of Use ]:
SNSPW
 

About Peter Yu I am a research and development professional with expertise in the areas of image processing, remote sensing and computer vision. I received BASc and MASc degrees in Systems Design Engineering at the University of Waterloo. My working experience covers industries ranging from district energy to medical imaging to cinematic visual effects. I like to dabble in 3D artwork, I enjoy cycling recreationally and I am interested in sustainable technology. More about me...

Feel free to contact me with any questions about this site at [user]@[host] where [user]=web and [host]=peteryu.ca

Copyright © 1997 - 2019 Peter Yu