Very excited to identify how to write exec tests in golang

An Introduction to Testing External Utilities in Golang.md

How to Unit Test Code Relying on External Utilities in Golang

This gist illustrates the basic concept of how to unit test code that executes external scripts and utilities by mocking.

It combines several blog posts to create the necessary insights:

• http://cs-guy.com/blog/2015/01/test-main/ for a modern (golang 1.4+) approach to test chicanery.
• https://npf.io/2015/06/testing-exec-command/ for more explanation of the technique involved.
• http://cs-guy.com/blog/2015/01/test-main/ for broader context on how to use TestMain
• https://medium.com/@povilasve/go-advanced-tips-tricks-a872503ac859 for deeper insight into testing mindset.
• https://golang.org/pkg/testing for details of the core testing API.

In reading this content on Day 1 of my Golang Testing journey, I struggled to understand my goal: how to effectively test functionality related to remotely executed functionality without actually leveraging those utilities.

The connection between a basic understanding of Golang testing and some of the techniques recommended is really unclear, so this write-up seeks to connect the dots a bit more slowly while illustrating what's involved in my interpretation.

The key insights you will be wrapping your head around:

1. You can store a function, even a function attached to another package, in a variable in your local package, then use that variable as a function.
2. Inside a test function, you can swap in a different function for that variable, monkey patching it to use some test-only code.
3. When you run go test it compiles and generates a binary which is immediately executed, we are able to reference and execute this binary with os.Args[0] and the rest of the arguments passed to it.
4. Using TestMain we can tailor what the generated test binary does, using environment variables to provide context.
5. We can call our own test binary with the exec library, and using our trick from #2, we can manipulate our external executions to instead target our handy, immediately managed test binary.

Review the code and the inline comments for more insights into how this works.

But first, an example of the code under test for context:

// execCommand is a simple variable reference to the function exec.Command. We will use this in our test code to swap out the value of execCommand,
// creating an opportunity to manipulate the exec.Command struct and therefore the details of what will be executed.
var execCommand = exec.Command

// This is a function from github.com/phase2/rig/util to parse out the version number from a docker --version command.
func GetRawCurrentDockerVersion() string {
	output, _ := execCommand("docker", "--version").Output()
	re := regexp.MustCompile("Docker version (.*),")
	return re.FindAllStringSubmatch(string(output), -1)[0][1]
}

myexec_test.go

package myexec

import (
    "fmt"
    "os"
    "os/exec"
    "strings"
    "testing"
)

// mock provides mock values to use as lookup responses to functions we will execute in our production code.
// The idea is to use the command as a lookup key to the result it might generate.
// Currently it only supports a single value, in the future this may be split into multiple maps for different,
// generic classes of success and failure. We cannot use multiple values for entries in this map because each response
// is expected to be a string that an executed command would return to Stdout.
var mock = map[string]string{
	"docker --version": "Docker version 17.09.0-ce, build afdb6d4",
}

// TestMain is a special function that takes over handling the behavior of the the test runner `go test`
// generates to execute your code. I do not know if you can have one per file or one for a project's entire
// collection of tests.
//
// In the example below, we rely on an environment variable: `GO_TEST_MODE` to determine whether the
// testing process will behave normally (running all test and handling the result, done by default) or
// will behavior in a special manner because we have tailored the way an exec.Command() will execute
// to flow through this logic instead of what was originally intended.
//
// You may be wondering, why would we go to such an elaborate length to mock the result of the a shell
// execution? Well, if we directly interpolated the mocked value for the command, the resulting object
// would be a string, and not the expected structure the code might be looking for as a result of executing
// a remote command.
func TestMain(m *testing.M) {
    switch os.Getenv("GO_TEST_MODE") {
    case "":
        // Normal test mode.
        os.Exit(m.Run())

    case "echo":
	// Outputs the arguments passed to the test runner.
	// This will be the command that would have executed under normal runtime.
	// This mode can be used to test that we can predict programmatically assembled command that would be executed.
        iargs := []interface{}{}
        for _, s := range os.Args[1:] {
            iargs = append(iargs, s)
        }
        fmt.Println(iargs...)

    case "mock":
        // Used the command that would be executed under normal runtime as the key to our mock value map and outputs the value.
        // I am still researching how to adjust this overall pattern to centralize the code as test helpers but allow individual
	// test files to supply their own mock.
        fmt.Printf("%s", mock[strings.Join(os.Args[1:], " ")])
    }
}

// mockExecCommand uses fakeExecCommand to transform the intended remote executation
// into something controlled by the test runner, then adds an environment variable to
// the command so TestMain routes it to the mocking functionality.
func mockExecCommand(command string, args...string) *exec.Cmd {
  cmd := fakeExecCommand(command, args...)
  cmd.Env = []string{"GO_TEST_MODE=mock"}
  return cmd
}

// echoExecCommand uses fakeExecCommand to transform the intended remote executation
// into something controlled by the test runner, then adds an environment variable to
// the command so TestMain routes it to the command echo functionality.
func echoExecCommand(command string, args...string) *exec.Cmd {
  cmd := fakeExecCommand(command, args...)
  cmd.Env = []string{"GO_TEST_MODE=echo"}
  return cmd
}

// fakeExecCommand creates a new reference to an exec.Cmd object which has been transformed
// to use the supplied parameters as arguments to be submitted to our test runner binary.
// It should never be used directly.
func fakeExecCommand(command string, args...string) *exec.Cmd {
  testArgs := []string{command}
  testArgs = append(testArgs, args...)
  return exec.Command(os.Args[0], testArgs...)
}

// TestEcho is an example Test from the various blog posts to illustrate the echo functionality.
// I will be swapping it out for a real use case, as it is it's difficult to understand because it's not
// actually testing real code, just demonstrating the concept inline to the test.
func TestEcho(t *testing.T) {
    cmd := exec.Command(os.Args[0], "hello", "world")
    cmd.Env = []string{"GO_TEST_MODE=echo"}
    output, err := cmd.Output()

    if err != nil {
        t.Errorf("echo: %v", err)
    }
    if g, e := string(output), "hello world\n"; g != e {
        t.Errorf("echo: want %q, got %q", e, g)
    }
}

// TestGetRawCurrentDockerVersion is a real first test implementation using the test helpers above.
func TestGetRawCurrentDockerVersion(t *testing.T) {
  // Re-define execCommand so our runtime code executes using the mocking functionality.
  // I thought execCommand would be a private variable in file scope, apparently sharing the package
  // is enough to access and manipulate it. Or perhaps test functions have special scope rules?
  execCommand = mockExecCommand
  // Put back the original behavior after we are done with this test function.
  defer func(){ execCommand = exec.Command }()
  // Run the code under test.
  out := GetRawCurrentDockerVersion()
	
  // Implement our assertion.
  expected := "17.09.0-ce"
  if out != expected {
	  t.Errorf("GetRawCurrentDockerVersion: Expected %q, Actual %q", expected, out)
  }
}

TODOS.md

Future Actions

• Put together a real test of echo mode to confirm it's utility.
• Identify if this is the right approach to behavior swapping. A dependency injection container feels more natural coming from PHP trends.
  • http://ewanvalentine.io/writing-a-service-container-in-go/
  • https://github.com/euskadi31/go-service
  • etc
• Research more about how TestMain works in practice. Per the golang docs for the testing package, each testing file should have it's own implementation.
• Create github.com/phase2/rig/testing/testing.go for test helpers, and use this to home the reusable pieces above.