https://linuxize.com/post/how-to-use-sed-to-find-and-replace-string-in-files/
When working with text files, you’ll often need to find and replace strings of text in one or more files.
sed is a stream editor. It can perform basic text manipulation on files and input streams such as pipelines. With sed, you can search, find and replace, insert, and delete words and lines. It supports basic and extended regular expressions that allow you to match complex patterns.
In this article, we’ll talk about how to find and replace strings with sed. We’ll also show you how to perform a recursive search and replace.
Find and Replace String with sed #
There are several versions of sed, with some functional differences between them. macOS uses the BSD version, while most Linux distributions come with GNU sed pre-installed by default. We’ll use the GNU version.
The general form of searching and replacing text using sed takes the following form:
sed -i 's/SEARCH_REGEX/REPLACEMENT/g' INPUTFILE
-i- By default,sedwrites its output to the standard output. This option tellssedto edit files in place. If an extension is supplied (ex -i.bak), a backup of the original file is created.s- The substitute command, probably the most used command in sed./ / /- Delimiter character. It can be any character but usually the slash (/) character is used.SEARCH_REGEX- Normal string or a regular expression to search for.REPLACEMENT- The replacement string.g- Global replacement flag. By default,sedreads the file line by line and changes only the first occurrence of theSEARCH_REGEXon a line. When the replacement flag is provided, all occurrences are replaced.INPUTFILE- The name of the file on which you want to run the command.
It is a good practice to put quotes around the argument so the shell meta-characters won’t expand.
Let’s see how we can use the sed command to search and replace text in files with some of its most commonly used options and flags.
For demonstration purposes, we will be using the following file:
file.txt
123 Foo foo foo
foo /bin/bash Ubuntu foobar 456
If the g flag is omitted, only the first instance of the search string in each line is replaced:
sed -i 's/foo/linux/' file.txt
123 Foo linux foo
linux /bin/bash Ubuntu foobar 456
With the global replacement flag sed replaces all occurrences of the search pattern:
sed -i 's/foo/linux/g' file.txt
123 Foo linux linux
linux /bin/bash Ubuntu linuxbar 456
As you might have noticed, the substring foo inside the foobar string is also replaced in the previous example. If this is not the wanted behavior, use the word-boundary expression (\b) at both ends of the search string. This ensures the partial words are not matched.
sed -i 's/\bfoo\b/linux/g' file.txt
123 Foo linux linux
linux /bin/bash Ubuntu foobar 456
To make the pattern match case insensitive, use the I flag. In the example below we are using both the g and I flags:
sed -i 's/foo/linux/gI' file.txt
123 linux linux linux
linux /bin/bash Ubuntu linuxbar 456
If you want to find and replace a string that contains the delimiter character (/) you’ll need to use the backslash (\) to escape the slash. For example to replace /bin/bash with /usr/bin/zsh you would use
sed -i 's/\/bin\/bash/\/usr\/bin\/zsh/g' file.txt
The easier and much more readable option is to use another delimiter character. Most people use the vertical bar (|) or colon (:) but you can use any other character:
sed -i 's|/bin/bash|/usr/bin/zsh|g' file.txt
123 Foo foo foo
foo /usr/bin/zsh Ubuntu foobar 456
You can also use regular expressions. For example, to search all 3 digit numbers and replace them with the string number you would use:
sed -i 's/\b[0-9]\{3\}\b/number/g' file.txt
number Foo foo foo
foo /bin/bash demo foobar number
Another useful feature of sed is that you can use the ampersand character & which corresponds to the matched pattern. The character can be used multiple times.
For example, if you want to add curly braces {} around each 3 digit number, type:
sed -i 's/\b[0-9]\{3\}\b/{&}/g' file.txt
{123} Foo foo foo
foo /bin/bash demo foobar {456}
Last but not least, it is always a good idea to make a backup when editing a file with sed. To do that, just provide an extension for the backup file to the -i option. For example, to edit the file.txt and save the original file as file.txt.bak you would use:
sed -i.bak 's/foo/linux/g' file.txt
To make sure that the backup is created, list the files with the ls command:
ls
file.txt file.txt.bak
Recursive Find and Replace #
Sometimes you may want to recursively search directories for files containing a string and replace the string in all files. This can be done using commands such as find or grep to recursively find files in the directory and piping the file names to sed.
The following command will recursively search for files in the current working directory and pass the file names to sed.
find . -type f -exec sed -i 's/foo/bar/g' {} +
To avoid issues with files containing space in their names, use the -print0 option, which tells find to print the file name, followed by a null character and pipe the output to sed using xargs -0 :
find . -type f -print0 | xargs -0 sed -i 's/foo/bar/g'
To exclude a directory, use the -not -path option. For example, if you are replacing a string in your local git repo to exclude all files starting with dot (.), use:
find . -type f -not -path '*/\.*' -print0 | xargs -0 sed -i 's/foo/bar/g'
If you want to search and replace text only on files with a specific extension, you will use:
find . -type f -name "*.md" -print0 | xargs -0 sed -i 's/foo/bar/g'
Another option is to use the grep command to recursively find all files containing the search pattern and then pipe the filenames to sed:
grep -rlZ 'foo' . | xargs -0 sed -i.bak 's/foo/bar/g'
Conclusion #
Although it may seem complicated and complex, at first, searching and replacing text in files with sed is very simple.
To learn more about sed commands, options, and flags, visit the GNU sed manual and Grymoire sed tutorial