awk_sed_commands.md

awk and sed sample commands:

Delete trailing commas in line

sed 's/,*\r*$//'

like this:

input:
1,2,3,,,

output:
1,2,3

command:

cat test.csv | sed 's/,*\r*$//' | less

Remove a word:

sed -e 's/\<No\>//g' filename >>>> Remove No from lines

sed 's/\@xx\.domain\.net//g'

Remove blank lines:

sed '/^$/d' filename

Remove empty columns:

sed 's/||/|/g'  filename.csv >>>delimiter |
sed 's/,,/,/g' filename.csv >>>delimiter ,

Print with delimiter awk:

cat filename.csv | awk -F '|' '{print $1FS$2FS$6}' >>> Use FS

Add a new column awk:

cat filename.csv | awk -F '|' '{$13="No" OFS $13; print $2FS$1FS$8FS$9FS$10FS$13}'

Remove first 1000 lines from a file

******* it'll rewrite your file *******
sed -i 1,1000d dump.sql

Remove quotation

sed "s/'//g"

sed -i -e 's/^\[\(.*\)\] \(-?[0-9\.]*\)$/\1 \2/g' $file
            ^ ^^ ^^    ^  ^   ^        ^  ^  ^ 
            | || ||    |  |   |        |  |  + -the second match (the number)  
            | || ||    |  |   |        | +---- the first match (the n-1 first fields)      
            | || ||    |  |   |        +------ end of line
            | || ||    |  |   +--------------- a number
            | || ||    |  +------------------- save in memory (\2)
            | || ||    +---------------------- your closing bracket
            | || |+-------------------------- the n-1 first fields
            | || +--------------------------- save in memory (\1)
            | |+----------------------------- your opening bracket
            | +------------------------------ beginning of line    
            +-------------------------------- substitution mode
            

Sort IPs

cat vm_list.txt | awk -F ',' '{print $5}' | sed '/^$/d' | sed "s/\"//g" | sort -t . -k 2,2n -k 3,3n -k 4,4n

print col 5, remove empty lines, remove " and then sort from second oct

delimited output

awk -F":" -v OFS=',' '{print $1, $2, 9}' fname

List files based on their size

ls -lh -S | grep 'G\|T' | awk -F' '  '{print $5, "->", $9}'