Skip to content

Instantly share code, notes, and snippets.

@sv99
Last active December 5, 2023 13:43
  • Star 14 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save sv99/6852cc2e2a09bd3a68ed to your computer and use it in GitHub Desktop.
How can I replace a newline (\n) using sed?

stackoverflow (hdorio)

Fast answer:

sed ':a;N;$!ba;s/\n/ /g' file
  1. :a create a label 'a'
  2. N append the next line to the pattern space
  3. $! if not the last line, ba branch (go to) label 'a'
  4. s substitute, /\n/ regex for new line, / / by a space, /g global match (as many times as it can)

sed will loop through step 1 to 3 until it reach the last line, getting all lines fit in the pattern space where sed will substitute all \n characters


Alternatives:

All alternatives, unlike sed will not need to reach the last line to begin the process

with bash, slow

while read line; do printf "%s" "$line "; done < file

with perl, sed-like speed

perl -p -e 's/\n/ /' file

with tr, faster than sed, can replace by one character only

tr '\n' ' ' < file

with paste, tr-like speed, can replace by one character only

paste -s -d ' ' file

with awk, tr-like speed

awk 1 ORS=' ' file

Other alternative like "echo $(< file)" is slow, works only on small files and needs to process the whole file to begin the process.


Long answer from the sed FAQ 5.10:

5.10. Why can't I match or delete a newline using the \n escape sequence? Why can't I match 2 or more lines using \n?

The \n will never match the newline at the end-of-line because the newline is always stripped off before the line is placed into the pattern space. To get 2 or more lines into the pattern space, use the 'N' command or something similar (such as 'H;...;g;').

Sed works like this: sed reads one line at a time, chops off the terminating newline, puts what is left into the pattern space where the sed script can address or change it, and when the pattern space is printed, appends a newline to stdout (or to a file). If the pattern space is entirely or partially deleted with 'd' or 'D', the newline is not added in such cases. Thus, scripts like

sed 's/\n//' file       # to delete newlines from each line             
sed 's/\n/foo\n/' file  # to add a word to the end of each line         

will NEVER work, because the trailing newline is removed before the line is put into the pattern space. To perform the above tasks, use one of these scripts instead:

tr -d '\n' < file              # use tr to delete newlines              
sed ':a;N;$!ba;s/\n//g' file   # GNU sed to delete newlines             
sed 's/$/ foo/' file           # add "foo" to end of each line          

Since versions of sed other than GNU sed have limits to the size of the pattern buffer, the Unix 'tr' utility is to be preferred here. If the last line of the file contains a newline, GNU sed will add that newline to the output but delete all others, whereas tr will delete all newlines.

To match a block of two or more lines, there are 3 basic choices: (1) use the 'N' command to add the Next line to the pattern space; (2) use the 'H' command at least twice to append the current line to the Hold space, and then retrieve the lines from the hold space with x, g, or G; or (3) use address ranges (see section 3.3, above) to match lines between two specified addresses.

Choices (1) and (2) will put an \n into the pattern space, where it can be addressed as desired ('s/ABC\nXYZ/alphabet/g'). One example of using 'N' to delete a block of lines appears in section 4.13 ("How do I delete a block of specific consecutive lines?"). This example can be modified by changing the delete command to something else, like 'p' (print), 'i' (insert), 'c' (change), 'a' (append), or 's' (substitute).

Choice (3) will not put an \n into the pattern space, but it does match a block of consecutive lines, so it may be that you don't even need the \n to find what you're looking for. Since GNU sed version 3.02.80 now supports this syntax:

sed '/start/,+4d'  # to delete "start" plus the next 4 lines,           

in addition to the traditional '/from here/,/to there/{...}' range addresses, it may be possible to avoid the use of \n entirely.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment