Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Golang test runner (not very mature, prob lots of cases I don't yet know about, that it doesn't handle correctly yet)
#!/usr/bin/env ruby
require 'json'
require 'time'
require 'pp'
# Run from the root of the app
Dir.chdir File.dirname __dir__
# Find package information for handling args
ROOT_PACKAGE = File.foreach("go.mod").first.split.last
valid_directories = Dir['**/*_test.go'].map { File.dirname _1 }.uniq
valid_directories << "..." # all packages, this seems to be a golang convention
valid_directories.sort!
# Help screen
if ARGV.include?("--help") || ARGV.include?("-h")
puts "#$0 [flags] [directory-name]"
puts
puts "Wraps `go test`. Args are directories with tests in them."
puts "Note that directory paths should be relative to the root, not the CWD."
puts "To run all tests, either pass no arguments or pass `...`"
puts "To run the main test suite, pass `.` (the directory containing"
puts "the main package). Directories may begin with a leading `./` or a"
puts "trailing slash, (eg valid paths)."
puts
puts "Valid directory names:"
puts valid_directories.pretty_inspect.gsub(/^/, ' ')
puts
puts "Flags:"
puts " -h | --help # show this help screen"
puts
puts "Examples:"
puts " #$0 # implicitly test everything"
puts " #$0 ... # explicitly test everything"
puts " #$0 ./... # same as above, the leading `./` is optional"
puts " #$0 . # test main"
puts " #$0 evaluator # run these tests: ./evaluator/*_test.go"
puts " #$0 ./evaluator/ # same as above"
puts " #$0 api/router # test the router package within the api namespace"
puts " #$0 . api/router # test both main and api/router"
exit 0
end
# Parse and validate args
directories_to_run = ARGV.map { _1.gsub(%r(\A\./|/\z), "") } # remove leading "./" (for normalization) and trailing "/" (so you can pass paths the shell completed)
invalid_packages = directories_to_run - valid_directories
if invalid_packages.include? "main"
$stderr.puts "To test main, instead pass '.'"
exit 1
elsif invalid_packages.any?
$stderr.puts "Invalid directories: #{invalid_packages.inspect}"
$stderr.puts "All provided directories should be in this list:"
$stderr.puts valid_directories.inspect
exit 1
end
directories_to_run << "..." if directories_to_run.empty?
# Honestly, it's pretty confusing what the difference between a dir and a package is.
# In their help screen (`go help test`) they say they take packages, but, eg,
# `go test .` works where `go test main` does not... so do they take dirs or packages? ¯\_(ツ)_/¯
packages_to_run = directories_to_run.map { "./#{_1}" }
# Find known test names so we can figure out what the correct test name is for
# the mangled test names that their runner emits.
# For fixing incorrect results when humanizing the test names. (the algorithm
# will put spaces on camelCase, but this isn't always correct, so add exceptions here).
NAME_FIXES = {
"Graph QL" => "GraphQL",
"Run Tests" => "RunTests", # this is the name of a GraphQL mutation resolver
}
HUMAN_READABLE_TEST_NAMES = Dir["**/*_test.go"]
.flat_map { File.readlines _1 }
.filter_map { |line|
case line
# Find the human string passed to `testing.T.Run`
# eg if this line was in main_test.go:
# `t.Run("it returns the command's output", func(t *testing.T) {`
# Then we would have this hash entry:
# `{ "it_returns_the_command's_output" => "it returns the command's output" }`
when /\bt.Run\("(.*)"/
name = $1 # At some point, this might need to unescape the test names (they're pulled from a string literal)
[name.gsub(/\s/, "_"), name] # I don't actually know their mangling algorithm, so far replacing whitespace with underscores has found everything
# They find tests by looking for a function whose name begins with `Test`
# (they may also reflect on the arg type, not sure, but this heuristic is probably sufficient)
# eg if this line was in main_test.go
# `func TestGraphQLServer(t *testing.T) {`
# Then we would have this hash entry:
# `{ "TestGraphQLServer" => "Test GraphQL Server" }`
when /\bfunc Test(\w+)/
[ "Test#{$1}",
$1.gsub(/(?<=[a-z])(?=[A-Z])/, " ") # add spaces between camel case word boundaries
.gsub(Regexp.union(NAME_FIXES.keys), NAME_FIXES) # replace incorrect names with the correct ones
.gsub(/#{Regexp.union(NAME_FIXES.values)}(?=[A-Z])/, '\& ') # potentially add a space after the corrected name
]
end
}
.to_h
# could maybe put this in a module and include it, but this seems good enough for now
def humanize(mangled_test_name)
HUMAN_READABLE_TEST_NAMES.fetch mangled_test_name, mangled_test_name
end
# Their runner isn't very good, it still prints out updates for humans,
# even though we run it with a JSON output format. So we need to filter
# out the human runner's messages.
def self.spam?(output)
case output
when /\A\s*(===|---) (RUN|PASS|FAIL)\b/
true # eg filter out "=== RUN TestName", and "--- PASS:"
when "PASS\n", "FAIL\n"
true # this one is for like the full suite's status
when /\A(ok|FAIL|\?) *\t#{ROOT_PACKAGE}/
true # this is like a summary line or something, IDK
else
false
end
end
# ANSI escape codes for colourizing the output
module ANSI
RED = "\e[31m".freeze
GREEN = "\e[32m".freeze
YELLOW = "\e[33m".freeze
BLUE = "\e[34m".freeze
MAGENTA = "\e[35m".freeze
CYAN = "\e[36m".freeze
WHITE = "\e[37m".freeze
OFF = "\e[39m".freeze
end
# Structure to handle the test results
class TestResults
module Status
INITIALIZED = :initialized
RUNNING = :running
PASSED = :pass
FAILED = :fail
SKIPPED = :skip
end
attr_reader :name, :status, :start, :stop, :output, :parent, :children
def initialize(name: nil, hide: false, parent: nil)
self.status = Status::INITIALIZED
self.output = []
self.children = {}
self.name = name
self.hide = hide
self.parent = parent
@already_displayed = false
end
def [](child_path, hide: false)
child_path = split_path child_path
return self if child_path.empty?
child_name = child_path.shift
children[child_name] ||= TestResults.new(name: child_name, hide: hide, parent: self)
children[child_name][child_path, hide: hide]
end
def add_output(text)
output << text
self
end
def run(time)
self.start = to_time time
self.status = Status::RUNNING
self
end
[Status::PASSED, Status::FAILED, Status::SKIPPED].each do |status|
define_method status do |time|
# It sometimes says the parent fails without saying the children failed
# so have the parent send its status update to its children, and have
# each test only remember the first status update
self.stop ||= to_time time
self.status = status unless complete?
children.each do |_name, child|
child.send status, time
end
self
end
end
def already_displayed?
@already_displayed
end
def to_ansi
return "" if hide?
@already_displayed = true
ansi = ""
ansi << parent.ansi_begin_group if parent
indicator, color =
case status
when Status::INITIALIZED then ["i", ANSI::YELLOW]
when Status::RUNNING then ["r", ANSI::YELLOW]
when Status::PASSED then ["√", ANSI::GREEN]
when Status::FAILED then ["X", ANSI::RED]
when Status::SKIPPED then ["-", ANSI::YELLOW]
else raise "UNEXPECTED STATUS: #{status.inspect}"
end
if children.none? && name
ansi << "#{indentation}#{color}#{indicator}#{ANSI::OFF} #{humanize(name)}\n"
end
if children.any? && output.any?
ansi << "#{indentation}#{humanize(name)}\n"
end
# it doesn't always send the message that the child completed, so we may
# need to print the child's information when the parent prints
children.each do |_name, child|
ansi << child.to_ansi unless child.already_displayed?
end
if output.any?
output.each do |lines|
lines.each_line(chomp: true) do |line|
ansi << "#{indentation}#{color}|#{ANSI::OFF} #{line}\n"
end
end
end
ansi
end
def hide?
@hide
end
protected
def depth
@depth ||=
if !parent
-1
else
depth = 0
depth += 1 unless hide?
depth += parent.depth if parent
depth
end
end
def indentation
@indentation ||=
if depth < 0
""
else
" " * depth
end.freeze
end
def ansi_begin_group
return "" if hide?
return "" if @ansi_begin_group_printed # don't begin the group multiple times
@ansi_begin_group_printed = true
ansi = ""
ansi << parent.ansi_begin_group if parent
ansi << "#{indentation}#{humanize(name)}\n"
end
private
attr_writer :name, :hide, :status, :start, :stop, :output, :parent, :children
def to_time(time)
time.is_a?(Time) ? time : Time.iso8601(time)
end
def split_path(path)
return path.dup if path.is_a? Array
dirname, basename = File.split path
return [basename] if basename == path
split_path(dirname) << basename
end
# sometimes tests pass/fail even though it never emitted a run event :rolls-eyes:
def complete?
status != Status::RUNNING && status != Status::INITIALIZED
end
end
tests = TestResults.new(hide: true)
tests[ROOT_PACKAGE, hide: true] # don't display the full path to the package, which would add lots of pointless nesting
# Run the tests
read, write = IO.pipe
pid = spawn "go", "test", "-json", *packages_to_run, out: write, err: write
write.close
read.each_line do |line|
begin
msg = JSON.parse(line, symbolize_names: true)
rescue JSON::ParserError => err
# *sigh*, some problems cause it to fail without printing JSON output,
# I think this is like if the build fails or something, IDK, but we have
# to handle the case where it gives us back random fkn non JSON nonsense
# ...I know what you're thinking, you're thinking it's because we're printing
# stderr into stdout, which is true, but I'm only doing that b/c it's printing
# non JSON error lines to both streams, so since I have to deal with it in
# stderr anyway, I might as well only have to read it out of one of them,
# rather than adding threads and another handler for stderr and having to
# do this, here, anyway.
puts "#{ANSI::RED}#{line}"
next
end
# The tests come in like "TestEvaluate/it_returns_an_error_and_the_output_when_the_command_returns_a_non-zero_status_code"
# for whatever dumb reason (I assume it's because they're converting tests to
# named methods or something), so we'll just join them to the package, which
# comes in like "github.com/QuickbeamPBC/quickbeam-server/evaluator"
test = tests[File.join(msg.fetch(:Package), msg.fetch(:Test, ""))]
case msg[:Action]
when "run"
test.run msg[:Time]
when "output"
test.add_output msg[:Output] unless spam? msg[:Output]
when "pass"
test.pass msg[:Time]
print test.to_ansi
when "fail"
test.fail msg[:Time]
print test.to_ansi
when "skip"
test.skip msg[:Time]
else
raise "Unknown message type: #{msg[:Action]}\n\n#{msg.inspect}"
end
end
# Cleanup
Process.wait pid
exit $?.exitstatus
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment