AWK • SED • GREP: Together we can change anything!.md

AWK • SED • GREP: Together we can change anything!

Download a PDF of my slides and notes

Example code from my MacAdmins Conference at Penn State 2024 presentation.

Battery cycle count

Using awk, sed, and grep to get the battery cycle couunt:

ioreg -r -c "AppleSmartBattery" | grep -w "CycleCount" | awk '{ print $3 }' | sed '/{.*}/d'
  
116

Using awk to get the battery cycle count:

ioreg -r -c "AppleSmartBattery" | awk -F ' = ' '/"CycleCount" = / { print $2 }'

116

Using sed to get the battery cycle count:

ioreg -r -c "AppleSmartBattery" | sed -e '/"CycleCount" =/!d' -e 's/.* = //'

116

Using grep to get the battery cycle count:

ioreg -r -c "AppleSmartBattery" | grep -e "\"CycleCount\" = " | grep -o '\d*'

116

Even shorter:

ioreg -l | grep -e "\"CycleCount\" = " | grep -o '\d*'

116

Demonstrating ed

# from the ed Wikipedia page
# https://en.wikipedia.org/wiki/Ed_(software)

$ed # start running ed
a
ed is the standard Unix text editor.
This is line number two.
.
2i
  
.
,l

ed is the standard Unix text editor.$
$
This is line number two.$

w text

63

3s/two/three/
,l

ed is the standard Unix text editor.$
$
This is line number three.$

w text

65

q
$

Similarities and differences in function

Using grep to return HTTP response code

#!/bin/zsh

function checkResponseCode()	{
	httpErrorCodes="000 No HTTP code received
200 Request successful
201 Request to create or update object successful
400 Bad request
401 Authentication failed
403 Invalid permissions
404 Object/resource not found
409 Conflict
500 Internal server error"
	
	responseCode=${1: -3}
	code=$( grep "$responseCode" <<< "$httpErrorCodes" )
	
	echo "$code"
}

responseCode=$( /usr/bin/curl \
--silent \
--url "https://www.google.com" \
--write-out "%{http_code}" )

checkResponseCode "$responseCode"

Examples of addressing

#!/bin/zsh

poem="Mary had a litte lamb.
Its fleece was white as snow.
And everywhere that Mary went,
The lamb was sure to go."

grep 'Mary' <<< "$poem"

sed -n '/Mary/p' <<< "$poem"

awk '/Mary/ { print $0 }' <<< "$poem"
_____________________________________

Mary had a litte lamb.
And everywhere that Mary went,

When to use each

The useless use of cat vs. each command's ability to read files on its own

cat ~/Desktop/list.txt | grep "tacos"
grep "tacos" ~/Desktop/list.txt

cat ~/Desktop/list.txt | sed -n "tacos/p"
sed -n "tacos/p" ~/Desktop/list.txt

cat ~/Desktop/list.txt | awk '/tacos/ { print $0 }'
awk '/tacos/ { print $0 }' ~/Desktop/list.txt

The useless use of echo vs. here strings

echo "$variable" | grep "tacos"
grep "tacos" <<< "$variable"

echo "$variable" | sed -n "tacos/p"
sed -n "tacos/p" <<< "$variable"

echo "$variable" | awk '/tacos/ { print $0 }'
awk '/tacos/ { print $0 }' <<< "$variable"

Single quotes, double quotes, and no quotes

grep tacos <<< "$variable"
grep 'too many tacos' <<< "$variable"
grep "$variable" ~/Desktop/list.txt

sed -n 'tacos/p' <<< "$variable"
sed -n "$variable/p" ~/Desktop/list.txt

awk '/tacos/ { print $0 }' <<< "$variable"

Syntax of a command statement

program options address/pattern command input

Grep examples

xml="<mobile_device_model>
		<model_name>Watch7,3</model_name>
		<display_name>Apple Watch Series 8</display_name>
</mobile_device_model>
<mobile_device_model>
		<model_name>Watch7,4</model_name>
		<display_name>Apple Watch Series 9</display_name>
</mobile_device_model>"

grep "model_name" <<< "$xml"

	<model_name>Watch7,3</model_name>
	<model_name>Watch7,4</model_name>

xml="<mobile_device_model>
		<model_name>Watch7,3</model_name>
		<display_name>Apple Watch Series 8</display_name>
</mobile_device_model>
<mobile_device_model>
		<model_name>Watch7,4</model_name>
		<display_name>Apple Watch Series 9</display_name>
</mobile_device_model>"

grep --after-context 1 "model_name" <<< "$xml"

	<model_name>Watch7,3</model_name>
	<display_name>Apple Watch Series 8</display_name>
--
	<model_name>Watch7,4</model_name>
	<display_name>Apple Watch Series 9</display_name>

xml="<mobile_device_model>
		<model_name>Watch7,3</model_name>
		<display_name>Apple Watch Series 8</display_name>
</mobile_device_model>
<mobile_device_model>
		<model_name>Watch7,4</model_name>
		<display_name>Apple Watch Series 9</display_name>
</mobile_device_model>"

grep --after-context 1 --line-number "model_name" <<< "$xml"

2:	<model_name>Watch7,3</model_name>
3-	<display_name>Apple Watch Series 8</display_name>
--
6:	<model_name>Watch7,4</model_name>
7-	<display_name>Apple Watch Series 9</display_name>

Sed examples

Replace "Watch7,3" model identifier with "Apple Watch Series 9"

modelNames="Watch7,3
Watch7,4
Watch7,5"

sed 's/Watch7,3/Apple Watch Series 9/' <<< "$modelNames"


Apple Watch Series 9
Watch7,4
Watch7,5

Replace each every model identifier with "Apple Watch Series 9" using a single command for each.

modelNames="Watch7,3
Watch7,4
Watch7,5"

sed 's/Watch7,3/Apple Watch Series 9/ ; s/Watch7,4/Apple Watch Series 9/ ; s/Watch7,5/Apple Watch Series 9/' <<< "$modelNames"

Apple Watch Series 9
Apple Watch Series 9
Apple Watch Series 9

Replace every model identifier with "Apple Watch Series 9" using a regular express to match multiple model identifiers.

modelNames="Watch7,3
Watch7,4
Watch7,5"

sed 's/Watch7,\d/Apple Watch Series 9/' <<< "$modelNames"


Apple Watch Series 9
Apple Watch Series 9
Apple Watch Series 9

Syntax for substitution in sed. This is the only syntax where the command precedes the address.

's/pattern/replacement/'

Syntax for all other sed commands. This follows the normal syntax where the command follows the address.

'/pattern/one-letter-command'

The address '2,4' refers to the range of lines 2 through 4. The command 'd' means delete.

list="Line 1
Line 2
Line 3
Line 4
Line 5"

sed '2,4 d' <<< "$list"

Line 1
Line 5

Instead of deleting the range of lines, write them to a file named "numbersFile.txt". The on-screen output still displays every line evaluated, but the text file only contains lines 2 through 4.

list="Line 1
Line 2
Line 3
Line 4
Line 5"

sed '2,4 w /Users/Shared/numbersFile.txt' <<< "$list"

Line 1
Line 2
Line 3
Line 4
Line 5

Awk example

Create mailing labels for everyone in California from a mailing list

mailingList="Abigail Adams, 100 A Street, Albany, CA 94706
Bob Bright, 200 B Street, Bakersfield, CA 93301
Charlie Cartwright, 300 C Street, Cambridge, NY 12816
Denise Darling, 400 D Street, Dale, NY 14039
Edith Ebbing, 500 E Street, Eagleville, CA 96110"

fixedMailingList=$( sed 's/, /\t/g' <<< "$mailingList" )

awk -F "\t" '/CA/ {
print $1
print $2
print $3 ", " $4 $5
print ""
}' <<< "$fixedMailingList"

Resources

Flops to gigaflops to teraflops converter

ed (software) | Wikipedia

Teletype Model 33 | Wikipedia

Unix | Wikipedia

Regular expression | Wikipedia

Grep | Wikipedia

Gres: A Research UNIX Reader: Annotated Excerpts from the Programmer’s Manual, 1971-1986 | Darmouth

sed | Wikipedia

AWK | Wikipedia

Unix philosophy | Wikipedia

An Introduction to regex | YouTube

Semantic Versioning 2.0 format

Book sed & awk, 2nd Edition from O'Reilly Media

Book grep Pocket Reference from O'Reilly Media