Skip to content

Instantly share code, notes, and snippets.

@tcely
Last active January 12, 2023 19:42
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save tcely/cf1abacf82cc856096619d74ae836a31 to your computer and use it in GitHub Desktop.
Save tcely/cf1abacf82cc856096619d74ae836a31 to your computer and use it in GitHub Desktop.
A good portable awk shebang is not easy to find.
#!/usr/bin/awk -f
#!/usr/bin/awk -E
#
# If you have an awk version that doesn't support the -f flag,
# then you are just out of luck.
#
# If you just have no clue where awk will be, or you prefer to use -E,
# then you can try this bash snippet to launch awk for you.
#
# You might think the -E tests below are overly complex. You'd be wrong.
# I thank Apple for their wonderfully broken awk on Mac OS X.
# If you find a system, this doesn't work on please let me know!
#
# Copy the below line to the top of the file:
#!/usr/bin/env bash
#
# Inspired by: https://unix.stackexchange.com/a/97280
#
_src="${0}"
function exec_awk() {
_cmd="$(command -v awk)"
_awk_prog="/awk:/ && /option/ && /-E/ {exit 1;} {next}"
_flag="-$("$_cmd" -E /dev/null </dev/null 2>&1 | "$_cmd" "$_awk_prog" && echo 'E' || echo 'f')"
true bash#; exec "${_cmd:-false}" "${_flag}" "${_src}" "$@"; exit;
}
true {}#; set -e; exec_awk "$@"
# You place your awk program below this line and make sure this file has
# execute permissions.
@seamusdemora
Copy link

I've got macos mojave, and awk reports its version as awk version 20070501.

I've tried your gist w/ both shebangs, the first shebang alone, and the second shebang alone, but none work. Potentially relevant stuff follows:

$ ls -l
total 48
-rwxr-xr-x 1 seamus staff 1054 Oct 28 22:35 01_simple.awk
...

$ which awk
/usr/bin/awk

01_simple.awk is simply a c&p of your gist w/ the following 2 lines appended:

 {
        a=$0;
        print "This is a test: a is " a;
}   

I am a bit confused by your directions wrt the bash shebang... what exactly is to be done with that??

@tcely
Copy link
Author

tcely commented Oct 28, 2019

Copy #!/usr/bin/env bash to the top of the file. Just as the comment says.

What happened when you ran ./01_simple.awk ?

@seamusdemora
Copy link

seamusdemora commented Oct 29, 2019

Your gist is a fine piece of work, and I truly appreciate you sharing it. I've no expertise in scripting, but I do imagine that I understand the English language fairly well. And so, respectfully, I feel compelled to opine that your instructions are a wee bit ambiguous:

Just above the comment that says, "Copy the below line to the top of the file:"... is another comment that says, "If you just have no clue where awk will be, or you prefer to use -E, then you can try this bash snippet to launch awk for you."

Since I do know exactly where awk is (from which awk output in my Q), I process this statement to mean that I don't need to try the bash snippet to launch awk, and I took that to include the bash shebang line. If the bash shebang were necessary in all cases, I assumed that you would have put it at the top of the file with the other awk shebangs. That's how I understood it.

Hope that's clear & doesn't offend. Wrt the results, they were simply that the script appeared to "hang"; the command prompt did not return until after I entered ^c.

UPDATE:

I've made a bit of progress after realizing I did something stupid. This snippet, when appended to a cut&paste of the entire gist as listed does work (and terminate) properly:

BEGIN {
        print "This is a test";
        exit(0);
}

And so, if you can clarify which shebangs are needed vs which are comments, I'll consider my question answered :)

@tcely
Copy link
Author

tcely commented Oct 30, 2019

You have three options:

  1. Use bash to find awk and determine which flag to use.
  2. Choose awk -f
  3. Choose awk -E

The clever bit behind this awk gist is that it is valid for both awk and bash.

The bash handles two important operations:

  1. Find the awk binary
  2. Determine if E can be used with that awk binary

If you have that information already, it is much quicker to just pick the appropriate awk shebang. Otherwise, you should copy the bash shebang to the top of the file. When you do that, you put up with the overhead from bash before your awk runs, but you gain the ability to run in more environments as long as bash is available.

@seamusdemora
Copy link

Brilliant! Thank you.

This gist should get wider distribution. Have you considered a "self-answered" question on Stack Exchange?

@mogando668
Copy link

mogando668 commented Jan 12, 2023

do you actually need either -f or -E ?


dash -c '________='\''BEGIN { for(_ in ENVIRON) { print _, length(ENVIRON[_]) } }'\''; 

printf "\n\n\tawk \47%s\47\n\n" "$________"; mawk2 -- "$________" | gcat -n ' 


	awk 'BEGIN { for(_ in ENVIRON) { print _, length(ENVIRON[_]) } }'

     1	m2aCMB 37
     2	mnetHnOdFn 32
     ….
    12	HOMEBREW_PREFIX 13
    13	NODE_REPL_HISTORY 30
    14	mmvre1 43
    15	NODE_REPL_HISTORY_SIZE 5
    16	SHLVL 1
    17	TMPDIR 49
    18	m3rOLD 44
    …….
   121	ZDOTDIR 16
   122	m3t 41

worked exactly the same for gawk ::::

   124		# gawk profile, created Thu Jan 12 10:53:31 2023
   125	
   126		# BEGIN rule(s)
   127	
   128		BEGIN {
   129	   123  	for (_ in ENVIRON) {
   130	   123  		print _, length(ENVIRON[_])
   131			}
   132		}
   133	

in fact, if your shell allows this, directly piping in commands via /dev/stdin also works :::

         dash -c ' __='\''BEGIN { for (_ in FUNCTAB) { print _, __ = FUNCTAB[_], length(_) } }'\''; 
                          
                         printf "%s" "$__" | gawk -p- -- "$( paste -)" | gcat -n ' 

     1	rand rand 4
     2	dcgettext dcgettext 9
     3	gsub gsub 4
     4	match match 5
     5	int int 3
     6	log log 3
     7	sprintf sprintf 7
     8	strftime strftime 8
     9	systime systime 7
    ...
    37	sub sub 3
    38	substr substr 6
    39	xor xor 3
    40	lshift lshift 6
    41	strtonum strtonum 8
    42	toupper toupper 7
    43		# gawk profile, created Thu Jan 12 11:15:06 2023
    44	
    45		# BEGIN rule(s)
    46	
    47		BEGIN {
    48	    42  	for (_ in FUNCTAB) {
    49	    42  		print _, __ = FUNCTAB[_], length(_)
    50			}
    51		}
    52	

but of course, if it supports -f ( which is just about every major awk variant actually being used) - you can even automate multi-awk code testing without needing eval or exec :

 dash -c '__='\''BEGIN { srand(); srand(); OFS = "\f"; 

                          print ENVIRON["_"], ARGV[+_], srand(), NR, FPAT, OFMT, CONVFMT, ARGC, srand() }'\''; 

            printf "\n\n\t \"\$awkvariant\" \47%s\47\n\n" "$__"; 

            for ___ in "nawk" "mawk" "mawk2" "gawk"; do 

                    printf "%s" "$__" | "$___" -f- | gcat -n; 
                    printf " %s\n\n"      "$___"

           done ' 


	 "$awkvariant" 'BEGIN { srand(); srand(); OFS = "\f"; print ENVIRON["_"], ARGV[+_], srand(), NR, FPAT, OFMT, CONVFMT, ARGC, srand() }'

     1	/bin/dash
                 nawk
                     1673540748
                               0

                                %.6g
                                    %.6g
                                        1
                                         1673540748
 nawk

     1	/bin/dash
                 mawk
                     1673540748
                               0

                                %.6g
                                    %.6g
                                        1
                                         1673540748
 mawk

     1	/bin/dash
                 mawk2
                      #srand1673540748.39820#
                                             0

                                              %.6g
                                                  %.6g
                                                      1
                                                       #srand1673540748.39824#
 mawk2

     1	/bin/dash
                 gawk
                     1673540748
                               0
                                [^[:space:]]+
                                             %.6g
                                                 %.6g
                                                     1
                                                      1673540748
 gawk

@tcely
Copy link
Author

tcely commented Jan 12, 2023

The goal is to have awk run the content of the file without introducing additional dependencies. (Using bash is a fallback that's not preferred.)

As the comments say, if you already know where awk will be and/or which variant of awk will be used, and you don't care about supporting any unknown systems, then just write the shebang you need using that knowledge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment