Skip to content

Instantly share code, notes, and snippets.

@AaradhyaSaxena
Last active February 27, 2023 11:31
Show Gist options
  • Save AaradhyaSaxena/e594d959aa4cff7918144c328111765e to your computer and use it in GitHub Desktop.
Save AaradhyaSaxena/e594d959aa4cff7918144c328111765e to your computer and use it in GitHub Desktop.
Cucumber BDD

BDD

BDD is a way for software teams to work that closes the gap between business and tech people by encouraging collaboration between roles to build shared understanding of desired behavior of the system.

Producing system documentation that guides development and is automatically checked against system behavior.

BDD requires 3 things to be done in small rapid iterations:

  1. Take a small upcoming change to. the system ( User Story ).
    • Have a conversation to decide system behavior with concrete examples.
  2. Document those examples in a way that could be automated, and check for agreement.
  3. Implement the behavior described by each documented example.

Discovery -> Formulation -> Automation

Cucumber facilitates BDD.

Living Documentation

Code reflects Documentation, Documetation reflects teams shared understanding. Doc automatically tells you when it is out of sync with the behavior of the app.

Gherkin

Gherkin is a simple syntax that allows teams to write business readable, executable specifications.

Those concrete examples mentioned above are formulated into Gherkin scenarios. Cucumber turns the Gherkin specification into automated tests.

Feature: Guess the word

  # The first example has two steps
  Scenario: Maker starts a game
    When the Maker starts a game
    Then the Maker waits for a Breaker to join

  # The second example has three steps
  Scenario: Breaker joins a game
    Given the Maker has started a game with the word "silky"
    When the Breaker joins the Maker's game
    Then the Breaker must guess a word with 5 characters

Some of the Gherkin keywords are Given, When, and Then.

Shouty

Shouty is a social network that allows people who are physically close to communicate, just like people have always communicated with their voices. In the real world you can talk to someone in the same room, or across the street. Or even 100 m away from you in a park - if you shout.

That’s Shouty. What you say on this social network can only be “heard” by people who are nearby.

Let’s start with a very basic example of Shouty’s behaviour. Sean the shouter shouts "free bagels at Sean’s" and Lucy the listener who happens to be stood across the street from his store, 15 metres away, hears him. She walks into Sean’s Coffee and takes advantage of the offer.

We can translate this into a Gherkin scenario so that Cucumber can run it. Here’s how that would look.

Scenario: Listener is within range
  Given Lucy is located 15m from Sean
  When Sean shouts "free bagels at Sean's"
  Then Lucy hears Sean’s message

Scenario just tells Cucumber we’re about to describe an example that it can execute.

  • Given is the context for the scenario. We’re putting the system into a specific state, ready for the scenario to unfold.
  • When is an action. Something that happens to the system that will cause something else to happen: an outcome.
  • Then is the outcome. It’s the behaviour we expect from the system when this action happens in this context.

You’ll notice we’ve omitted from our outcome anything about Lucy walking into Sean’s store and making a purchase. Remember, Gherkin is supposed to describe the behaviour of the system, so it would be distracting to have it in our scenario.

Each scenario has these three ingredients: a context, an action, and one or more outcomes.

Together, they describe one single aspect of the behaviour of the system. An example.

Now that we’ve translated our example into Gherkin, we can automate it!

Setup

First we’ll create a package.json that describes the NPM packages we need for our project and add the @cucumber/cucumber package to it.

$ npm init -y
$ npm install -D @cucumber/cucumber

We can now run the cucumber-js command to see if everything works:

$ ./node_modules/.bin/cucumber-js

There’s no output, because we haven’t written any scenarios yet, but there are no errors either! Having to remember the full path to Cucumber is not very user friendly, but luckily there’s a nicer way.

Edit package.json and change the test line under the scripts from it’s default to cucumber-js

  "scripts": {
    "test": "cucumber-js"
  },

Now we’re ready! If we run npm test at this point, we’ll again see cucumber pass with no errors.

>> npm test
Failure
1 scenario (1 undefined)

Undefined means Cucumber doesn’t know what to do for any of the three steps we wrote in our Gherkin scenario. It needs us to provide some step definitions.

Step definitions translate from the plain language you use in Gherkin into JavaScript code.

When Cucumber runs a step, it looks for a step definition that matches the text in the step. If it finds one, then it executes the code in the step definition.

If it doesn’t find one… well, you’ve just seen what happens. Cucumber helpfully prints out some code snippets that we can use as a basis for new step definitions.

Create a new directory called step_definitions underneath the features directory, Say: steps.js.

steps.js

const { Given, When, Then } = require('@cucumber/cucumber')

Given('Lucy is located {int} metres from Sean', function (int) {
  // Write code here that turns the phrase above into concrete actions
  return 'pending'
})

When('Sean shouts {string}', function (string) {
  // Write code here that turns the phrase above into concrete actions
  return 'pending'
})

Then('Lucy hears Sean\'s message', function () {
  // Write code here that turns the phrase above into concrete actions
  return 'pending'
})

Q. What does it mean when Cucumber says a step is Pending?

  • Cucumber tells us that a step (and by inference the Scenario that contains it) is Pending when the automation code throws a Pending error.
  • This allows the development team to signal that automation for a step is a work in progress. This makes it possible to tell the difference between steps that are still being worked on and steps that are failing due to a defect in the system.
  • For example, when we run our tests in a Continuous Integration (CI) environment, we can choose to ignore pending scenarios.

Update Step Definitions

steps.js

Given('Lucy is located {int}m from Sean', function (distance) {
  this.lucy = new Person
  this.sean = new Person
  this.lucy.moveTo(distance)
})

When('Sean shouts {string}', function (message) {
  this.sean.shout(message)
  this.message = message
})

Then('Lucy hears Sean’s message', function () {
  assert.deepEqual(this.lucy.messagesHeard(), [this.message])
})

shouty.js

class Person {
    moveTo() {
    }
    shout(message) {
    }
    messagesHeard() {
        return ["free bagels at Sean's"]
    }
}
  • First we specified the behaviour we wanted, using a Gherkin scenario in a feature file.
  • Then we wrote step definitions to translate the plain english from our scenario into concrete actions in code.
  • Finally, we used the step definitions to guide us in building out our very basic domain model for the Shouty application.

Cucumber Expressions

Step Definitions

Good step definitions are important because they enable the readability of your scenarios. The better we are at matching plain language phrases from Gherkin, the more expressive you can be when writing scenarios.

When Cucumber first started, they used to use regular expressions to match plain language phrases from Gherkin steps.

As the regular expressions have quite an intimidating reputation, they replaced them with Cucumber expressions. Cucumber is backwards compatible so we can still use regular expressions.

Parameters

To capture interesting values from our step definitions, we can use a feature of Cucumber Expressions called parameters.

For example: To capture the number of metres, we can use the {int} parameter: which is passed as an argument to our step definition

steps.js

Given('Lucy is located {int} metres from Sean', function (distance) {

Now we’re capturing that value as an argument. The value 100 will be passed to our code automatically by Cucumber.

Flexibility

  1. One common example is the problem of plurals. Suppose we want to place Lucy and Sean just 1 metre apart:
Given Lucy is located 1 metre from Sean

Because we’ve used the singular metre instead of the plural metres we don’t get a match.

  1. Another is to allow alternates - different ways of saying the same thing. For example, to accept this step:
Given Lucy is standing 1 metre from Sean

…we can use this Cucumber Expression:

Given('Lucy is located/standing {int} metre(s) from Sean', function (distance) {

Now we can use either 'standing' or 'located' in our scenarios, and both will match just fine:

Custom Parameters

This allows us to transform the text captured from the Gherkin into any object you like before it’s passed into your step definition.

For example, let’s define our own {person} custom parameter type that will convert the string Lucy into an instance of Person automatically.

steps.js

Given('{person} is located/standing {int} metre(s) from Sean', function (lucy, distance) {

person_parameter.rb

const { defineParameterType } = require('@cucumber/cucumber')

const Person = require('../../src/shouty')

defineParameterType({
  name: 'person',
  regexp: /Lucy|Sean/,
  transformer: name => new Person(name)
})

We use the defineParameterType function from Cucumber to define our new parameter type. We need to give it a name which is the name we’ll use inside the curly brackets in our step definition expressions.

We also need to define a regular expression. This is necessary to tell Cucumber Expressions what text to match when searching for this parameter in a Gherkin step.

Finally, in transformer block, which takes the text captured from the Gherkin step that matched the regular expression pattern, and runs some code. The return value of this block is what will be passed to the step definition. In this case, the block is passed the name of the person (as a string) which we can then pass to the Person class’s constructor.

Custom parameters allow you to bring your domain model - the names of the classes and objects in your solution - and your domain language - the words you use in your scenarios and step definitions - closer together.

Code

steps.js

const { Given, When, Then, Before } = require('cucumber')
const { assertThat, is } = require('hamjest')

const { Person, Network } = require('../../src/shouty')

const default_range = 100

Before(function () {
  this.people = {}
  this.network = new Network(default_range)
})

Given('a person named {person}', function (name) {
  this.people[name] = new Person(this.network)
})

Given('Lucy is {int} metres from Sean', function (distance) {
  this.network = new Network()
  this.lucy    = new Person(this.network, 0)
  this.sean    = new Person(this.network, 0)
})

When('Sean shouts', function () {
  this.people['Sean'].shout('Hello, world')
})

When('Sean shouts {string}', function (message) {
  this.sean.shout(message)
  this.messageFromSean = message
})

Then('Lucy should hear a shout', function () {
  assertThat(this.people['Lucy'].messagesHeard().length, is(1))
})

Then('Lucy should hear Sean\'s message', function () {
  assertThat(this.people['Lucy'].messagesHeard(), contains(this.messageFromSean))
})

Then('Larry should not hear Sean\'s message', function () {
  assertThat(this.people['Larry'].messagesHeard(), not(contains(this.messageFromSean)))
})

Then('Larry should not hear a shout', function () {
  assertThat(this.people['Larry'].messagesHeard(), not(contains(this.messageFromSean)))
})

In the step definition layer, we can see that a new class has been defined, the Network. We’re creating an instance of the network here. Then we pass that network instance to each of the Person instances we create here. So both instances of Person depend on the same instance of network. The Network is what allows people to send messages to one another.

shout.feature

Feature: Hear shout

  Shouty allows users to "hear" other users "shouts" as long as they are close enough to each other.

  Rule: Shouts can be heard by other users

    Scenario: Listener hears a message
      Given a person named Sean
      And a person named Lucy
      When Sean shouts "free bagels at Sean's"
      Then Lucy should hear Sean's message

    Scenario: Listener hears a different message
      Given a person named Sean
      And a person named Lucy
      When Sean shouts "Free coffee!"
      Then Lucy should hear Sean's message

  Rule: Shouts should only be heard if listener is within range

    Scenario: Listener is within range
      Given the range is 100
      And a person named Sean is located at 0
      And a person named Lucy is located at 50
      When Sean shouts
      Then Lucy should hear a shout

    Scenario: Listener is out of range
      Given the range is 100
      And a person named Sean is located at 0
      And a person named Larry is located at 150
      When Sean shouts
      Then Larry should not hear a shout
  • The first scenario exists to illustrate that a listener hears the message exactly as the shouter shouted it. All the additional details (range, location) are incidental and make the scenario harder to read. So we added default values for them to remove them from the specifications.
  • Similarly for second scenario there’s no need to actually document the content of the shout, so created default content for the shout.

shouty.js

class Person {
  constructor(network) {
    this.messages = []
    this.network  = network

    this.network.subscribe(this)
  }

  shout(message) {
    this.network.broadcast(message)
  }

  hear(message) {
    this.messages.push(message)
  }

  messagesHeard() {
    return this.messages
  }
}
class Network {
    constructor(range) {
        this.listeners = []
        this.range = range
    }
    subscribe(person) {
        this.listeners.push(person)
    }
    broadcast(message, shouter_location) {
        this.listeners.forEach(listener => {
            if(Math.abs(listener.location - shouter_location) <= this.range) {
                listener.hear(message)
            }
        })
    }
}

DataTable

Gherkin has a special syntax called Data Tables, that allows you to specify tabular data for a step, using pipe characters to mark the boundary between cells.

Feature: hear_shout

    Scenario: Listener is out of range
      Given the range is 100
      And people are located at
        | name  | location |
        | Sean  | 0        |
        | Larry | 150      |
      When Sean shouts
      Then Larry should not hear a shout

At its most basic, the table is just a two-dimensional array. So, Larry’s location can be accessed by getting the value from array cell (2, 1)

steps.js

Given('people are located at', function (dataTable) {
  console.log(dataTable.raw())
})

You can also turn the table into a array of objects , where the first row is used for the property names, and each following row is used for the property values.

Given('people are located at', function (dataTable) {
  console.log(dataTable.hashes())
})

Now we can easily iterate over these objects and turn them into instances of Person:

Given('people are located at', function (dataTable) {
  dataTable.hashes().map((person) => {
    this.people[person.name] = new Person(this.network, person.location)
  })
})

Working with Cucumber

Filtering Tests

  1. Name We can pass arguments to Cucumber using the --name option, which tells Cucumber to only run scenarios with a name that matches the string provided, in this case "Message is too long".
npm test -- --name "Message is too long"

The value of the --name option is a regular expression.

  1. Line Number To run a specific scenario, specify the line number of the scenario within a feature file. The 'Message too long' scenario starts on line 44 in the feature file.
npm test -- features/hear_shout.feature:44
  1. Tags To runs scenarios with particular tags attached.

hear-shout.feature

    @focus
    Scenario: Listener is out of range
      Given the range is 100
      And people are located at
        | name     | Sean | Larry |
        | location |  0   | 150   |
      When Sean shouts
      Then Larry should not hear a shout

cucumber.js

module.exports = { default: '--publish-quiet --tags @focus' }

Now we can run only the scenarios tagged with focus.

Random Order

Each scenario should be isolated - it’s result should not depend on the outcome of any other scenario. To help you catch any dependencies between your scenarios, Cucumber can be told to run your scenarios in a random order.

To do this, we can use the random flag in the profile file.

cucumber.js

module.exports = { default: '--publish-quiet --order random' }

Result Reports

npm test -- --format html:report.html
npm test -- --format json:report.json

The rerun formatter

We choose the rerun formatter and send the output to a file whose name is preceded by the @ sign. Let’s call it @rerun.txt.

npm test -- -f rerun:@rerun.txt

What’s in that @rerun.txt file? It’s a list of the scenarios that failed! It’s using the line number filtering format.

This is really useful when we have a few failing scenarios and you want to re-run only ones that failed. We tell Cucumber to run only the failed scenarios by pointing it at the rerun file.

npm test -- @rerun.txt

Publishing Result

The HTML output can be shared, automatically publish the results online to Cucumber Reports.

  • Remove publish-quiet from cucumber.js.
  • Update cucumber.js, module.exports = { default: '--publish --order random' }

Now when we run Cucumber again and we get a banner output, the unique URL in this banner is the address of your published report.

Test scenarios should be BRIEF:

  • "B" is for Business language
  • "R" is for Real data
  • "I" is for Intention revealing
  • "E" is for Essential
  • "F" is for Focused

Step Results

  • Success When Cucumber finds a matching step definition it will execute it. If the block in the step definition doesn’t raise an error, the step is marked as successful (green). Anything you return from a step definition has no significance whatsoever.
  • Undefined When Cucumber can’t find a matching step definition, the step gets marked as undefined (yellow), and all subsequent steps in the scenario are skipped.
  • Pending When a step definition’s method or function invokes the pending method, the step is marked as pending (yellow, as with undefined ones), indicating that you have work to do.
  • Failed Steps When a step definition’s method or function is executed and raises an error, the step is marked as failed (red). What you return from a step definition has no significance whatsoever. Returning null or false will not cause a step definition to fail.
  • Skipped Steps that follow undefined, pending, or failed steps are never executed, even if there is a matching step definition. These steps are marked as skipped (cyan).
  • Ambiguous Step definitions have to be unique for Cucumber to know what to execute. If you use ambiguous step definitions, the step / scenario will get an “Ambiguous” result, telling you to fix the ambiguity.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment