Instantly share code, notes, and snippets.

Embed
What would you like to do?
Story: Writing Scripts with Go

Story: Writing Scripts with Go

This is a story about how I tried to use Go for scripting. In this story, I’ll discuss the need for a Go script, how we would expect it to behave and the possible implementations; During the discussion I’ll deep dive to scripts, shells, and shebangs. Finally, we’ll discuss solutions that will make Go scripts work.

Why Go is good for scripting?

While python and bash are popular scripting languages, C, C++ and Java are not used for scripts at all, and some languages are somewhere in between.

Go is very good for a lot of purposes, from writing web servers, to process management, and some say even systems. In the following article, I argue, that in addition to all these, Go can be used, easily, to write scripts.

What makes Go good for scripts?

  • Go is simple,readable, and not too verbose. This makes the scripts easy to maintain, and relatively short.
  • Go has many libraries, for all sorts of uses. This makes the script short and robust, assuming the libraries are stable and tested.
  • If most of my code is written in Go, I prefer to use Go for my scripts as well. When a lot of people are collaborating code, it is easier if they all have full control over the languages, even for the scripts.

Go is 99% There Already

As a matter of fact, you can already write scripts in Go. Using Go’s run subcommand: if you have a script named my-script.go, you can simply run it with go run my-script.go.

I think that the go run command, needs a bit more attention in this stage. Let’s elaborate about it a bit more.

What makes Go different from bash or python is that bash and python are interpreters - they execute the script while they read it. On the other hand, when you type go run, Go compiles the Go program, and then runs it. The fact that the Go compile time is so short, makes it look like it was interpreted. it is worth mentioning “they” say “go run is just a toy", but if you want scripts, and you love Go, this toy is what you want.

So we are good, right?

We can write the script, and run it with the go run command! What’s the problem? The problem is that I'm lazy, and when I run my script I want to type ./my-script.go and not go run my-script.go.

Let’s discuss a simple script that has two interactions with the shell: it gets an input from the command line, and sets the exit code. Those are not all the possible interactions (you also have environment variables, signals, stdin, stdout and stderr), but two problematic ones with shell scripts.

The script writes “Hello”, and the first argument in the command line, and exits with the code 42:

package main

import (
    "fmt"
    "os"
)

func main() {
    fmt.Println("Hello", os.Args[1])
    os.Exit(42)
}

The go run command behaves a bit weird:

$ go run example.go world
Hello world
exit status 42
$ echo $?
1

We’ll discuss that later on.

The go build can be used. This is how you would run it using the go build command:

$ go build
$ ./example world
Hello world
$ echo $?
42

Current workflow with this script looks like this:

$ vim ./example.go
$ go build
$ ./example.go world
Hi world
$ vim ./example.go
$ go build
$ ./example.go world
Bye world

What I want to achieve, is to run the script like this:

$ chmod +x example.go
$ ./example.go world
Hello world
$ echo $?
42

And the workflow I would like to have is this:

$ vim ./example.go
$ ./example.go world
Hi world
$ vim ./example.go
$ ./example.go world
Bye world

Sounds easy, right?

The Shebang

Unix-like systems support the Shebang line. A shebang is a line that tells the shell what interpreter to use to run the script. You set the shebang line according to the language that you wrote your script in.

It is also common to use the env command as the script runner, and then an absolute path to the interpreter command is not necessary. For example: #! /usr/bin/env python to run the python interpreter with the script. For example: if a script named example.py has the above shebang line, and it is executable (you executed chmod +x example.py), then by running it in the shell with the command ./example.py arg1 arg2, the shell will see the shebang line, and starts this chain reaction:

The shell runs /usr/bin/env python example.py arg1 arg2. This is actually the shebang line plus the script name plus the extra arguments. The command invokes /usr/bin/env with the arguments: /usr/bin/env python example.py arg1 arg2. The env command invokes python with python example.py arg1 arg2 arguments python runs the example.py script with example.py arg1 arg2 arguments.

Let’s start by trying to add a shebang to our go script.

1. First Naive Attempt:

Let's start with a naive shebang that tries to run go run on that script. After adding the shebang line, our script will look like this:

#! /usr/bin/env go run
package main

import (
    "fmt"
    "os"
)

func main() {
    fmt.Println("Hello", os.Args[1])
    os.Exit(42)
}

Trying to run it results in:

Output:

$ ./example.go
/usr/bin/env: ‘go run’: No such file or directory

What happened?

The shebang mechanism sends "go run" as one argument to the env command as one argument, and there is no such command, typing which “go run” will result in a similar error.

2. Second Attempt:

A possible solution could be to put #! /usr/local/go/bin/go run as the shebang line. Before we try it out, you can already spot a problem: the go binary is not located in this location in all environments, so our script will be less compatible with different go installations. Another solution is to use alias gorun="go run", and then change the shebang to #! /usr/bin/env gorun, in this case we will need to put the alias in every system that we run this script.

Output:

$ ./example.go
package main:
example.go:1:1: illegal character U+0023 '#'

Explanation:

OK, I have good news and bad news, what do you want to hear first? We’ll start with the good news :-)

  • The good news are that it worked, go run command was invoked with our script
  • The bad news: the hash sign. In a lot of languages the shebang line is ignored as it starts with a comment line indicator. Go compiler fails to read the file, since the line starts with an "illegal character"

3. The Workaround:

When no shebang line is present, different shells will fallback to different interpreters. Bash will fallback to run the script with itself, zsh for example, will fallback to sh. This leaves us with a workaround, as also mentioned in StackOverflow.

Since // is a comment in Go, and since we can run /usr/bin/env with //usr/bin/env (// == / in a path string), we could set the first line to:

//usr/bin/env go run "$0" "$@"

Result:

$ ./example.go world
Hi world
exit status 42
./test.go: line 2: package: command not found
./test.go: line 4: syntax error near unexpected token `newline'
./test.go: line 4: `import ('
$ echo $?
2

Explanation:

We are getting close: we see the output but we have some errors and the status code is not correct. Let's see what happened here. As we said, bash did not meet any shebang, and chose to run the script as bash ./example.go world (this will result in the same output if you'll try it). That's interesting - running a go file with bash :-) Next, bash reads the first line of the script, and ran the command: /usr/bin/env go run ./example.go world. "$0" Stands for the first argument and is always the name of the file that we ran. "$@" stands for all the command line arguments. In this case they were translated to world, to make: ./example.go world. That's great: the script ran with the right command line arguments, and gave the right output.

We also see a weird line that reads: "exit status 42". What is this? If we would try the command ourselves we will understand:

$ go run ./example.go world
Hello world
exit status 42
$ echo $?
1

It is stderr written by the go run command. Go run masks the exit code of the script and returns code 1. For further discussion about this behavior read here Github issue.

OK, so what are the other lines? This is bash trying to understand go, and it isn’t doing very well.

4. Workaround Improvement:

This StackOverflow page suggests to add `;exit "$?" to the shebang line. this will tell the bash interpreter not to continue to the following lines.

Using the shebang line:

//usr/bin/env go run "$0" "$@"; exit "$?"

Result:

$ ./test.go world
Hi world
exit status 42
$ echo $?
1

Almost there: what happened here is that bash ran the script using the go run command, and immediately after, exited with the go run exit code.

Further bash scripting in the shebang line, for sure can remove the stderr "exit status" message, even parse it, and return it as the program exit code.

However:

  • Further bash scripting means longer, and exhausting shebang line, which is supposed to look as simple as #! /usr/bin/env go.
  • Lets remember that this is a hack, and I don't like that this is a hack. After all, we wanted to use the shebang mechanism - Why? Because it's simple, standard and elegant!
  • That’s more or less the point where I stop using bash, and start using more comfortable languages as my scripting languages (such as Go :-) ).

Lucky Us, We Have gorun

gorun does exactly what we wanted. You put it in the shebang line as #! /usr/bin/env gorun, and make the script executable. That’s it, You can run it from your shell, just as we wanted!

$ ./example.go world
Hello world
$ echo $?
42

Sweet!

The Caveat: Compilability

Go fails compilation when it meets the shebang line (as we saw before).

$ go run example.go
package main:
example.go:1:1: illegal character U+0023 '#'

Those two options can’t live together. We must choose:

  • Put the shebang and run the script with ./example.go.
  • Or, remove the shebang and run the script with go run ./example.go.

You can’t have both!

Another issue, is that when the script lies in a go package that you compile. The compiler will meet this go file, even though it is not part of the files that are needed to be loaded by the program, and will fail the compilation. A workaround for that problem is to remove the .go suffix, but then you can’t enjoy tools such as go fmt.

Final Thoughts

We’ve seen the importance of enabling writing scripts in Go, and we’ve found different ways to run them. Here is a summary of the findings:

Type Exit Code Executable Compilable Standard
go run
gorun
// Workaround

Explanation: Type: how we chose to run the script. Exit code: after running the script it will exit with the script’s exit code. Executable: the script can be chmod +x. Compilable: the script passes go build Standard: the script doesn’t need anything beside the standard library.

As it seems, there is no perfect solution, and I don’t see why we shouldn’t have one. It seems like the easiest, and least problematic way to run Go scripts is by using the go run command. It is still too ‘verbose’ to my opinion, and can’t be “executable”, and the exit code is incorrect, which makes it hard to tell if the script was completed successfully.

This is why I think there is still work do be done in this area of the language. I don’t see any harm in changing the language to ignore the shebang line. This will solve the execution issue, but a change like this probably won't be accepted by the Go community.

My colleague brought to my attention the fact that the shebang line is also illegal in javascript. But, in node JS, they added a strip shebang function which enables running node scripts from the shell.

It would be even nicer, if gorun could come as part of the standard tooling, such as gofmt and godoc.

Thanks for reading,

If you find this interesting, please, ✎ comment below or ★ star above ☺

For more stuff: gist.github.com/posener.

Cheers, Eyal

@jucie

This comment has been minimized.

Show comment
Hide comment
@jucie

jucie Sep 8, 2017

Nice story, Posener. Easy reading and enjoyable. Thanks.

jucie commented Sep 8, 2017

Nice story, Posener. Easy reading and enjoyable. Thanks.

@carlmjohnson

This comment has been minimized.

Show comment
Hide comment
@carlmjohnson

carlmjohnson Sep 8, 2017

//usr/bin/env go run "$0" "$@" ; exit "$?" works to convey the exit code.

carlmjohnson commented Sep 8, 2017

//usr/bin/env go run "$0" "$@" ; exit "$?" works to convey the exit code.

@tiborvass

This comment has been minimized.

Show comment
Hide comment
@tiborvass

tiborvass Sep 9, 2017

The problem you're solving is to reduce go run example.go to something at least as short as ./example.go.

$ alias ,='go run'
$ , example.go       # takes as long to type as ./example.go

You can of course change the aliased character, but wanted to show that you can replace ./ with , both being punctuation and close to each other on the keyboard.

Pros:

  • does not require chmod +x
  • does not require shebang, especially unusual shebang, so works with all Go code parsers

Con:

  • not the usual ./
  • needs an alias setup

tiborvass commented Sep 9, 2017

The problem you're solving is to reduce go run example.go to something at least as short as ./example.go.

$ alias ,='go run'
$ , example.go       # takes as long to type as ./example.go

You can of course change the aliased character, but wanted to show that you can replace ./ with , both being punctuation and close to each other on the keyboard.

Pros:

  • does not require chmod +x
  • does not require shebang, especially unusual shebang, so works with all Go code parsers

Con:

  • not the usual ./
  • needs an alias setup
@ishanjain28

This comment has been minimized.

Show comment
Hide comment
@ishanjain28

ishanjain28 Sep 9, 2017

It is better than say using python because it takes some time for the python interpreter to start and then execute the script whereas go has no runtime so it starts instantaneously.

ishanjain28 commented Sep 9, 2017

It is better than say using python because it takes some time for the python interpreter to start and then execute the script whereas go has no runtime so it starts instantaneously.

@posener

This comment has been minimized.

Show comment
Hide comment
@posener

posener Sep 9, 2017

@carlmjohnson, it doesn't. the exit code is passed from go run command in an stderr line "exit status 123"

Owner

posener commented Sep 9, 2017

@carlmjohnson, it doesn't. the exit code is passed from go run command in an stderr line "exit status 123"

@posener

This comment has been minimized.

Show comment
Hide comment
@posener

posener Sep 9, 2017

@tiborvass, my main problem is that it is not standard, it is a "workaround" and I don't see the reason to have workarounds here. Another problem is the exit code, see the table in the bottom of the article.

Owner

posener commented Sep 9, 2017

@tiborvass, my main problem is that it is not standard, it is a "workaround" and I don't see the reason to have workarounds here. Another problem is the exit code, see the table in the bottom of the article.

@posener

This comment has been minimized.

Show comment
Hide comment
@posener

posener Sep 9, 2017

@ishanjain28, go run is very fast, but it doesn't "starts instantaneously". As I wrote in this article, the go run command first builds the program to a temporary executable, and then run it. So you have the build time, which will be usually very short for scripts.

Owner

posener commented Sep 9, 2017

@ishanjain28, go run is very fast, but it doesn't "starts instantaneously". As I wrote in this article, the go run command first builds the program to a temporary executable, and then run it. So you have the build time, which will be usually very short for scripts.

@muesli

This comment has been minimized.

Show comment
Hide comment
@muesli

muesli Sep 9, 2017

You can combine the // workaround with gorun, of course:

//usr/bin/env gorun "$0" "$@"; exit "$?"
package main
...

This lets you run the go script directly, as well as compile it with go build.

muesli commented Sep 9, 2017

You can combine the // workaround with gorun, of course:

//usr/bin/env gorun "$0" "$@"; exit "$?"
package main
...

This lets you run the go script directly, as well as compile it with go build.

@posener

This comment has been minimized.

Show comment
Hide comment
@posener

posener Sep 9, 2017

@muesli, like it! Thanks!
Still, that's leaving us with "non standard" solution.
Great solution!

Owner

posener commented Sep 9, 2017

@muesli, like it! Thanks!
Still, that's leaving us with "non standard" solution.
Great solution!

@vhodges

This comment has been minimized.

Show comment
Hide comment
@vhodges

vhodges Sep 9, 2017

On Linux you can use binfmt misc for this kind of thing, ie

echo ':Go:E::go::/path/to/gorun:' > /proc/sys/fs/binfmt_misc/register

More info: https://web.archive.org/web/20100118183358/http://www.tat.physik.uni-tuebingen.de/~rguenth/linux/binfmt_misc.html

EDIT: For @suntong (it's the same thing really.)

vhodges commented Sep 9, 2017

On Linux you can use binfmt misc for this kind of thing, ie

echo ':Go:E::go::/path/to/gorun:' > /proc/sys/fs/binfmt_misc/register

More info: https://web.archive.org/web/20100118183358/http://www.tat.physik.uni-tuebingen.de/~rguenth/linux/binfmt_misc.html

EDIT: For @suntong (it's the same thing really.)

@suntong

This comment has been minimized.

Show comment
Hide comment
@suntong

suntong Sep 9, 2017

@vhodges, can you completely rewrite your answer using gorun instead of your shell script please?

suntong commented Sep 9, 2017

@vhodges, can you completely rewrite your answer using gorun instead of your shell script please?

@yinchuan

This comment has been minimized.

Show comment
Hide comment
@yinchuan

yinchuan Sep 11, 2017

great to run go as script.
need a feature like .py => .pyc for performance

yinchuan commented Sep 11, 2017

great to run go as script.
need a feature like .py => .pyc for performance

@hanbang-wang

This comment has been minimized.

Show comment
Hide comment
@hanbang-wang

hanbang-wang Sep 11, 2017

Compile language and script language seem to be contradictory at the first place... But nice work 😄

hanbang-wang commented Sep 11, 2017

Compile language and script language seem to be contradictory at the first place... But nice work 😄

@johnrs

This comment has been minimized.

Show comment
Hide comment
@johnrs

johnrs Sep 11, 2017

Nice article. If you would like to cover Windows, too, this is how. From an Admin command prompt:
assoc .go=GoProgram
ftype GoProgram=C:\Go\bin\go.exe run "%1" %*
That's it. :)

johnrs commented Sep 11, 2017

Nice article. If you would like to cover Windows, too, this is how. From an Admin command prompt:
assoc .go=GoProgram
ftype GoProgram=C:\Go\bin\go.exe run "%1" %*
That's it. :)

@kostix

This comment has been minimized.

Show comment
Hide comment
@kostix

kostix Sep 11, 2017

@yinchan, that's what go install does.

No, really ;-)
Python is a scripting language and so one particular implemetation of it
you're referring to (CPython) implements behind-the-scenes "caching"
of the precompiled implementation to spare the parser's run on the next
invocation.

Contrary to this, go run does go build producing a temporary file
then execute that file and then throws it away.

If you'd like to keep the compiled form, just keep it ;-)

kostix commented Sep 11, 2017

@yinchan, that's what go install does.

No, really ;-)
Python is a scripting language and so one particular implemetation of it
you're referring to (CPython) implements behind-the-scenes "caching"
of the precompiled implementation to spare the parser's run on the next
invocation.

Contrary to this, go run does go build producing a temporary file
then execute that file and then throws it away.

If you'd like to keep the compiled form, just keep it ;-)

@posener

This comment has been minimized.

Show comment
Hide comment
@posener

posener Sep 11, 2017

@kostix, I think that @yinchuan raised a good point.
I think that he is saying that he would expect go run to skip the compilation of script.go if script.go and all it dependencies where not changed after the last go run command. This could be done by time stamping a cached binary of the form .goc :-)
Actually, this could be an interesting project to work on!

Owner

posener commented Sep 11, 2017

@kostix, I think that @yinchuan raised a good point.
I think that he is saying that he would expect go run to skip the compilation of script.go if script.go and all it dependencies where not changed after the last go run command. This could be done by time stamping a cached binary of the form .goc :-)
Actually, this could be an interesting project to work on!

@OneEricJohnson

This comment has been minimized.

Show comment
Hide comment
@OneEricJohnson

OneEricJohnson Sep 14, 2017

This seems like a straightforward enhancement to Go - support #! as the first line of a source file, and add a program to the distribution (an equivalent of gorun) so that it works with the #! line. Perhaps submit an enhancement request for Go? I've thought of trying this as well, but not been able to invest the time in experimenting that you did. Thanks so much for putting in time.

OneEricJohnson commented Sep 14, 2017

This seems like a straightforward enhancement to Go - support #! as the first line of a source file, and add a program to the distribution (an equivalent of gorun) so that it works with the #! line. Perhaps submit an enhancement request for Go? I've thought of trying this as well, but not been able to invest the time in experimenting that you did. Thanks so much for putting in time.

@dsypniewski

This comment has been minimized.

Show comment
Hide comment
@dsypniewski

dsypniewski Sep 14, 2017

Here are some "shebang" lines that will work with all of the requirements mentioned in this story.
This one creates temporary .goc file in the same directory as source
//usr/bin/env test -x $0 && (go build -o "${0}c" "$0" && "${0}c" $@; r=$?; rm -f "${0}c"; exit "$r"); exit "$?"
And this one keeps the cached .goc file alongside source and re-compiles only when source is newer
//usr/bin/env test -x $0 && (([ ! -x "${0}c" ] || [ "$0" -nt "${0}c" ]) && go build -o "${0}c" "$0"; "${0}c" $@); exit "$?"

Those are not so readable and definitely not super easy to remember but get the job done, tested on bash, sh, csh, zsh, tcsh and ksh

dsypniewski commented Sep 14, 2017

Here are some "shebang" lines that will work with all of the requirements mentioned in this story.
This one creates temporary .goc file in the same directory as source
//usr/bin/env test -x $0 && (go build -o "${0}c" "$0" && "${0}c" $@; r=$?; rm -f "${0}c"; exit "$r"); exit "$?"
And this one keeps the cached .goc file alongside source and re-compiles only when source is newer
//usr/bin/env test -x $0 && (([ ! -x "${0}c" ] || [ "$0" -nt "${0}c" ]) && go build -o "${0}c" "$0"; "${0}c" $@); exit "$?"

Those are not so readable and definitely not super easy to remember but get the job done, tested on bash, sh, csh, zsh, tcsh and ksh

@sdwarwick

This comment has been minimized.

Show comment
Hide comment
@sdwarwick

sdwarwick Sep 15, 2017

goexec -
1 - check source & includes for changes compared to a compiled exec file in ~/.gotemp ( if exists)
2 - re-compile if necessary, excise first line of files if they are formatted like a shbang (temp build file if needed), store exec in ~/.gotemp
3 - replace goexec process with executable so that all further term-io directly interacts with executable & error code return is correct

sdwarwick commented Sep 15, 2017

goexec -
1 - check source & includes for changes compared to a compiled exec file in ~/.gotemp ( if exists)
2 - re-compile if necessary, excise first line of files if they are formatted like a shbang (temp build file if needed), store exec in ~/.gotemp
3 - replace goexec process with executable so that all further term-io directly interacts with executable & error code return is correct

@gbeveridge

This comment has been minimized.

Show comment
Hide comment
@gbeveridge

gbeveridge Sep 15, 2017

It seems like we could use a solution like Elixir where a script is .exs and a compiled file is .ex if we had something like .gos .go it might make it easier for a tool to determine how to run the file.

gbeveridge commented Sep 15, 2017

It seems like we could use a solution like Elixir where a script is .exs and a compiled file is .ex if we had something like .gos .go it might make it easier for a tool to determine how to run the file.

@rkulla

This comment has been minimized.

Show comment
Hide comment
@rkulla

rkulla Sep 16, 2017

I made a wrapper that lets me type "go script.go". No need to chmod +x my go files. https://gist.github.com/rkulla/bbe82f81fa1baa283a5fde2aec8fb5a9 I've been using this a year and haven't had any issues. It also lets me run "go run script.go" and other normal uses of go when I need to.

rkulla commented Sep 16, 2017

I made a wrapper that lets me type "go script.go". No need to chmod +x my go files. https://gist.github.com/rkulla/bbe82f81fa1baa283a5fde2aec8fb5a9 I've been using this a year and haven't had any issues. It also lets me run "go run script.go" and other normal uses of go when I need to.

@md2perpe

This comment has been minimized.

Show comment
Hide comment
@md2perpe

md2perpe Sep 19, 2017

One solution is to create an "interpreter" that makes a copy of the file without the shebang line, and then go runs that file.

go-script:

#!/bin/sh

tmpfile=$(mktemp --suffix=.go /tmp/go-script.XXXXXX)
tail --lines=+2 "$@" >$tmpfile
go run $tmpfile
rm $tmpfile

Then one can write

#!/path/to/go-script
package main

func main() {
        println("It works!")
}

Some possible improvements of go-script:

  • only remove first line if it really begins with #!
  • make sure the temporary file is deleted even if program fails
  • allow more scripty code by adding package main and wrapping non-declarations in func main() { ... }

md2perpe commented Sep 19, 2017

One solution is to create an "interpreter" that makes a copy of the file without the shebang line, and then go runs that file.

go-script:

#!/bin/sh

tmpfile=$(mktemp --suffix=.go /tmp/go-script.XXXXXX)
tail --lines=+2 "$@" >$tmpfile
go run $tmpfile
rm $tmpfile

Then one can write

#!/path/to/go-script
package main

func main() {
        println("It works!")
}

Some possible improvements of go-script:

  • only remove first line if it really begins with #!
  • make sure the temporary file is deleted even if program fails
  • allow more scripty code by adding package main and wrapping non-declarations in func main() { ... }
@devonartis

This comment has been minimized.

Show comment
Hide comment
@devonartis

devonartis Sep 20, 2017

Ok why not use Go build or go install. If the GOPATH is setup correctly you would not need to add a SHEBANG. Go allows you to compile to any platform. Your Go workspace would have /bin /pkg /src

Build your Go app/script with Go Build /src/srcdirectory and it will place an executable file in your /bin directory. You can then take the same Code and then Build it with the command GOOS=windows in front of your build command it will create an exe for windows.

Once you compile your code you can easily run the command without using the run command in GO.

Please accept my apologies If I am misunderstanding something here ..

devonartis commented Sep 20, 2017

Ok why not use Go build or go install. If the GOPATH is setup correctly you would not need to add a SHEBANG. Go allows you to compile to any platform. Your Go workspace would have /bin /pkg /src

Build your Go app/script with Go Build /src/srcdirectory and it will place an executable file in your /bin directory. You can then take the same Code and then Build it with the command GOOS=windows in front of your build command it will create an exe for windows.

Once you compile your code you can easily run the command without using the run command in GO.

Please accept my apologies If I am misunderstanding something here ..

@MarounMaroun

This comment has been minimized.

Show comment
Hide comment
@MarounMaroun

MarounMaroun Oct 18, 2017

Well written, I like it. 👍

MarounMaroun commented Oct 18, 2017

Well written, I like it. 👍

@chmike

This comment has been minimized.

Show comment
Hide comment
@chmike

chmike Oct 30, 2017

Suggesting that the compiles consider as comment a shebang line as first line of a package main file would avoid the need to copy the file to strip off the shebang line. The problem of scripts are the access to packages. The script needs to be in the GOPATH dir if we want to be able to use packages. We can't have go scripts in a random directory.

chmike commented Oct 30, 2017

Suggesting that the compiles consider as comment a shebang line as first line of a package main file would avoid the need to copy the file to strip off the shebang line. The problem of scripts are the access to packages. The script needs to be in the GOPATH dir if we want to be able to use packages. We can't have go scripts in a random directory.

@metacritical

This comment has been minimized.

Show comment
Hide comment
@metacritical

metacritical Oct 30, 2017

I have been using go for writing scripts for some time.

metacritical commented Oct 30, 2017

I have been using go for writing scripts for some time.

@Grauwolf

This comment has been minimized.

Show comment
Hide comment
@Grauwolf

Grauwolf Nov 21, 2017

To make this POSIX compliant @thommey and me figured out, that the following would work and check all the boxes.

// 2>/dev/null;/usr/bin/go run $0 $@; exit $?

Contrary to the assumption in the beginning, // is not necessarily equal to / in all spots of a path. POSIX allows the root of // to be interpreted in an implementation-defined manner (cygwin uses this for example). Three or more slashes would be considered equal to / again, though.

Grauwolf commented Nov 21, 2017

To make this POSIX compliant @thommey and me figured out, that the following would work and check all the boxes.

// 2>/dev/null;/usr/bin/go run $0 $@; exit $?

Contrary to the assumption in the beginning, // is not necessarily equal to / in all spots of a path. POSIX allows the root of // to be interpreted in an implementation-defined manner (cygwin uses this for example). Three or more slashes would be considered equal to / again, though.

@vsoch

This comment has been minimized.

Show comment
Hide comment
@vsoch

vsoch Jan 24, 2018

Thank you I liked this a lot!! 💃

vsoch commented Jan 24, 2018

Thank you I liked this a lot!! 💃

@secumod

This comment has been minimized.

Show comment
Hide comment
@secumod

secumod commented Feb 21, 2018

https://blog.cloudflare.com/using-go-as-a-scripting-language-in-linux/

which is an extended and amended answer, that @vhodges provided

@doug-numetric

This comment has been minimized.

Show comment
Hide comment
@doug-numetric

doug-numetric Jul 14, 2018

hello.go

package main

import `fmt`

func main() {
    fmt.Println(`hello, world!`)
}

copy these contents to a file named after the go entry point.. without the .go extension
hello

#!/usr/bin/env bash

cat > $BASH_SOURCE.makefile << EOF
$BASH_SOURCE.goc : $BASH_SOURCE.go
	go build -o $BASH_SOURCE.goc $BASH_SOURCE.go

EOF

make -s -f $BASH_SOURCE.makefile
unlink $BASH_SOURCE.makefile
$BASH_SOURCE.goc
$ chmod 755 ./hello
$./hello
hello there, world!

doug-numetric commented Jul 14, 2018

hello.go

package main

import `fmt`

func main() {
    fmt.Println(`hello, world!`)
}

copy these contents to a file named after the go entry point.. without the .go extension
hello

#!/usr/bin/env bash

cat > $BASH_SOURCE.makefile << EOF
$BASH_SOURCE.goc : $BASH_SOURCE.go
	go build -o $BASH_SOURCE.goc $BASH_SOURCE.go

EOF

make -s -f $BASH_SOURCE.makefile
unlink $BASH_SOURCE.makefile
$BASH_SOURCE.goc
$ chmod 755 ./hello
$./hello
hello there, world!
@docwhat

This comment has been minimized.

Show comment
Hide comment
@docwhat

docwhat Sep 5, 2018

Typo:

Your Current workflow with this script looks like this: section incorrectly has the commands with a .go suffix.

docwhat commented Sep 5, 2018

Typo:

Your Current workflow with this script looks like this: section incorrectly has the commands with a .go suffix.

@docwhat

This comment has been minimized.

Show comment
Hide comment
@docwhat

docwhat Sep 5, 2018

This article has an explaination of using the binfmt stuff in Linux to do what you want: Using Go as a scripting language in Linux.

I see some problems, though:

  • How do you ensure the Go build environment is on all the boxes you want to run scripts on?

    It's over 300MiB which is big for some use ecases.

  • How do you ensure your libraries work and are the right version?

    gopkg.in helps somewhat, but not all libraries use this system for version management.

    vgo will eventually help a bunch with this.

  • Is it acceptable that your go scripts fail when the internet or github.com are down?

    bash scripts will run whether github.com is up or down. Python and Ruby scripts as well, assuming you pulled all the libraries down before hand.

  • How do you handle Go versions?

    As go evolves, you could end up using a feature that only works in older or newer versions of Go.

    This would cause more runtime versioning problems, such as we have with Ruby and Python scripts. There was a bunch of problems when macOs switched from Ruby 1.9.x to 2.x.

Anyway, I think it is a neat idea but I think the extra step of compiling and pushing it out to a directory in the PATH makes more sense at this time. 😃

docwhat commented Sep 5, 2018

This article has an explaination of using the binfmt stuff in Linux to do what you want: Using Go as a scripting language in Linux.

I see some problems, though:

  • How do you ensure the Go build environment is on all the boxes you want to run scripts on?

    It's over 300MiB which is big for some use ecases.

  • How do you ensure your libraries work and are the right version?

    gopkg.in helps somewhat, but not all libraries use this system for version management.

    vgo will eventually help a bunch with this.

  • Is it acceptable that your go scripts fail when the internet or github.com are down?

    bash scripts will run whether github.com is up or down. Python and Ruby scripts as well, assuming you pulled all the libraries down before hand.

  • How do you handle Go versions?

    As go evolves, you could end up using a feature that only works in older or newer versions of Go.

    This would cause more runtime versioning problems, such as we have with Ruby and Python scripts. There was a bunch of problems when macOs switched from Ruby 1.9.x to 2.x.

Anyway, I think it is a neat idea but I think the extra step of compiling and pushing it out to a directory in the PATH makes more sense at this time. 😃

@willurd

This comment has been minimized.

Show comment
Hide comment
@willurd

willurd Oct 10, 2018

You can also do this:

$ go build hello.go && ./hello world
Hello world

This gives the right status code and it's a "standard" solution:

$ go build hello.go && ./hello world
Hello world
$ echo $?
42

It's a bit more to type out, but you really only have to do it every once in a while. It also 1) doesn't print out the exit status 42 line, and 2) doesn't require a "shebang" line in your file.

It feels like this is the ideal solution for development. Once you're done writing/modifying your script you can add $GOPATH/bin to your PATH and just go install it:

$ go install
$ cd ~
$ hello world; echo $?
Hello world
42

willurd commented Oct 10, 2018

You can also do this:

$ go build hello.go && ./hello world
Hello world

This gives the right status code and it's a "standard" solution:

$ go build hello.go && ./hello world
Hello world
$ echo $?
42

It's a bit more to type out, but you really only have to do it every once in a while. It also 1) doesn't print out the exit status 42 line, and 2) doesn't require a "shebang" line in your file.

It feels like this is the ideal solution for development. Once you're done writing/modifying your script you can add $GOPATH/bin to your PATH and just go install it:

$ go install
$ cd ~
$ hello world; echo $?
Hello world
42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment