BATS Assertion Libraries comparison

README.md

"Who would dare assert that we know all there is to be known?" ~ Galileo Galilei

Introduction

At the time of this writing there are 3 separate libraries that offer assertions for the Bash Automated Testing System (otherwise known as BATS).

These are:

• jasonkarns/bats-assert
• thingsym/bats-assertion
• ztombol/bats-assert

This page shows a comparison of the functions each projects offers.

Legend

Symbol	Meaning
JBA	jasonkarns/bats-assert
TBA	thingsym/bats-assertion
ZBA	ztombol/bats-assert
foo	Affirmatory function
foo	Negatory function
👍	Function present
☝️	Other function can be used
✌️	Alias present

Assert comparison Table

Function Name	Also Known As	JBA	TBA	ZBA
fail	flunk	✌️		☝️ 1
assert		👍		👍
refute		👍		👍
assert_contains	assert_output_contains	👍		☝️ 2
refute_contains	refute_output_contains	👍		☝️ 2
assert_equal		👍	👍	👍
refute_equal	assert_fail_equal		✌️
assert_line		👍		👍
refute_line		👍		👍
assert_lines_equal			👍
refute_lines_equal	assert_fail_lines_equal		✌️
assert_lines_match			👍
refute_lines_match	assert_fail_lines_match		✌️
assert_match			👍
refute_match	assert_fail_match		✌️
assert_output		👍		👍
refute_output				👍
assert_success		👍	👍	👍
refute_success	assert_failure	✌️	✌️	✌️
assert_starts_with		👍		☝️ 3
refute_starts_with				☝️ 3
assert_status			👍	☝️ 4
TOTAL (Per project)		12	11	15

• ¹ ZBA supports fail through its dependency on bats-support
• ² ZBA supports assert_contains by using assert_output --partial or assert_output --regex options
• ³ ZBA supports assert_starts_with by using assert_contains with a /^.../ regex. (same with refute_starts_with)
• ⁴ ZBA supports asserting an explicit status "N" using assert_failure N

Summary

22 Function present across all projects

• ZBA does support assert_contains by using assert_output with the --partial or --regex options
• JBA's and ZBA's assert_success are not equivalent because JBA's assert_success checks both status and output. ZBA's only checks status. (uncertain about TBA)
• assert_contains and assert_output_contains (from JBA) should be considered aliases. only difference is stdin handling which ought to be unified
• ZBA's does support assert_starts_with by using assert_contains with a /^.../ regex. (same with refute_starts_with)
• TBA's assert_status is the same as combining JBA's and ZBA's assert_success and assert_failure (assert_success is the same as assert_status 0)
• i'm not sure what a pass helper would do, since a test passes by default. how would you expect that to be used?
• in most cases, refute_status would primarily be used to assert that the exit code is != 0, which assert_failure does, so I'd consider that nearly covered.

@jasonkarns Thanks for the review! I've updated the comparison based on your feedback.

• Both refute_status and pass have been removed, as I agree that they don't serve much purpose.
• I'm leaving assert_status in, as I should be able to check for a status other than 0 or 1.
• Support for assert_contains and assert_starts_with have been added to ZBA.

asserting an explicit status is possible with ZBA using assert_failure: https://github.com/ztombol/bats-assert#expected-status

Cool! Added. I've also added fail for ZBA (through bats-support).