Mix It Up
Experiences Combining BDD Behavioural Specifications with UI e2e Testing
Sometimes in life, two unexpected elements can combine together to form something better. Quite often we talk about two minds being better than one. But this applies to other things too.
One key example from music is the concept of a mashup. For those unfamiliar, a mashup is where at least two songs are merged together to produce another track. In 2020 one mashup that caused a storm in the UK was the merging of Dua Lipa’s track Hallucinate with the BBC News theme. Ben Howell’s remix actually won the Remix of the Year in the BBC Radio 1 Lockdown Awards.
Inspired by Ben’s achievement, myself and colleagues have been playing around with combining Behavioural Specifications with e2e testing to improve our testing footprint. In this piece I discuss the justification for combining these practices, the lessons we have learned along the way, and provide a simple example showcasing these benefits.
My Name Is
Before delving into why these techniques combine to be a match made in heaven, let me explain what each of these concepts are for anyone new to these concepts.
Behaviour Driven Development
BDD, or Behaviour Driven Development is a practice where new features are discovered by collaborating with business partners and specified as a set of examples that can drive implementation and testing of said feature. It evolved from the Test Driven Development movement.
I refer purposefully to specifications being used as part of this mashup very purposefully. What we intend to use in conjunction with an e2e testing framework is the examples, most likely specified in a format such as the Gherkin specification used along with the Cucumber framework. If you are following BDD as a practice, these specifications shall be outlined as part of the formulation stage. There are numerous resources available online that explain BDD as a concept, along with the benefits. Do check out two of my favourite sources from Seb Rose:
- Seb Rose @ Lean Agile Exchange 2020: Our BDDs are Broken
- Smartbear: Understanding the Benefits of Test Automation and BDD
End to End Testing
e2e, or End to End testing, refers to the ability to test the end user workflow by recreating the product experience from the user perspective. You’re basically testing the entire product from beginning to end.
Of course with e2e testing there is an argument to be made as to when it is appropriate to connect to the live running services compared to mocked services. I could spend an entire blog talking about my thoughts on when to connect to live running services compared to mocked. Since we are focusing on the basic setup today, the example covered later in this piece uses a hardcoded dataset.
Put Yourself In My Shoes
The user perspective aspect of e2e testing is exactly why it can merge with BDD practices so well. Behavioural specifications, such as those we shall write in Cucumber using the Gherkin format, are the examples that expose their view of the product.
We live in the age where technology and software are heavily ingrained into our everyday lives. Users of our software are not faceless individuals anymore. If we are not the direct end consumer, we often work within the same organisation with them, as I do, or we are competing with many products to gain attention and revenue.
You never really understand a person until you consider things from his point of view… until you climb into his skin and walk around in it.
Harper Lee, To Kill a Mocking Bird
To build good products, we need to embrace the comments of Atticus Finch and walk in our user’s skin. By collaborating to form examples using BDD, the documented examples can then be used in your tests. This makes the intended workflow understandable for developers and stakeholders alike.
How?
There are several different technologies that can be employed to achieve this mashup. Primarily, you need three frameworks:
- BDD Specification Framework
- Assertion Library
- e2e Testing Framework
This example, available on my personal GitHub, covers usage of Cucumber, Chai and Protractor. The main justification for using Protractor is that it is available with any Angular project created using the Angular CLI. I am very intrigued by Cypress as a potential replacement for Protractor as the expressive language it uses instead of the Page Object pattern looks to be more inline with user workflow than Protractor’s element based syntax. I will happily report my experiences in a future post.
Good Example
There are many resources that explain in detail how to setup Cucumber, Chai and Protractor. This guide by Amadou Sall is particularly helpful. However, at time of writing with the release of Cucumber 7 and the bundling of the cucumber types, you need to explicitly state install of Cucumber 6. Note that this will be required until Protractor provides support, which was not the case at time of writing. The command is as follows:
npm install --save-dev @types/{chai,cucumber} chai cucumber protractor-cucumber-framework
Focusing on the tests themselves, there are three file types for a given test that we require:
- The feature file containing our examples, or the behavioural specification. This is identifiable from the .feature extension.
- The test steps, postfixed by .steps, which lists the implementation of a given scenario step.
- Since we are using Protractor, we are adopting the Page Object pattern. This means we have an additional file, containing .po within the filename. It is here that we place our logic for accessing elements on the page.
Based on my experience of writing these tests in enterprise software, I’ve found it useful to use the folder structure depicted below. I advise adopting the below folder structure separating feature, test steps and page object files to stop your e2e folder from ballooning out of control. Adopting a component-based folder strategy similar to the Angular implementation is a valid option. Nevertheless, since we are testing the user perspective of the application, we will most likely have any given scenario touching multiple components, meaning this structure is less clear by comparison.
American English
Using Cucumber exposes the Gherkin syntax. The power of this syntax is that it allows you to express the user workflow in english-like language that users will understand. Often you should be able to collaborate with stakeholders using the BDD process to formulate their perspective as a series of examples.
via user collaboration as the scenarios to test against
As you can see from this example each scenario will specify a particular scenario outlining how the user expects the system to behave. Within each scenario:
- We can set the initial state using the Given line.
- The When segment outlines the precise action that you are expecting the user to perform.
- Our Then statements show the expected behaviour. Here you can see I have two checking on not just the title, but the list of artists I expect to see as a data table. These data tables are particularly useful for data-driven products such as financial systems.
Step by Step
The implementation of these steps is carried out in the steps file, depicted below. You will see for each Gherkin keyword, we specify the expression matching the precise step in the aforementioned feature file. We also specify a callback function dictating the steps to execute for that precise step.
actions corresponding to each scenario step
Focusing on the Then statement, you will see we are using parameters to promote reuse of these steps for potentially multiple scenarios. In addition to simple textual parameters, data tables such as the artists table shown in our example can support specification of complex data types and collections. Data tables are particularly useful for data heavy applications such as those I see working in financial services.
Being a web application, we often encounter asynchronous situations in obtaining the page results. It’s important to ensure you use async and await correctly to handle the time taken to capture the elements or perform service calls. Often when starting you may be tempted to start including timeout waits, but these should be used sparingly. In this trivial example it is not required, but I have often seen them be gradually increased over time when pinging backend services directly.
You Find Me
Looking at the steps you may see we are performing particular operations to get elements on the page. With Protractor we use the Page Object pattern to seek out elements to inspect within the tests. It has the advantage of decoupling your test steps from the page layout. Be mindful that other frameworks such as Cypress do not advocate that pattern.
All of the examples here search by CSS paths, including the $$ and $ Protractor shorthands. Other selectors, including by id, name, ng-model and binding are provided.
Page Object pattern to separate DOM-based logic from the testing framework
All of the examples here search by CSS paths, including the $$ and $ Protractor shorthands. Other selectors, including by id, name, ng-model and binding are provided.
Coda
As we have reached the end of this track, I hope you have seen how combining BDD specifications and e2e testing helps provide expressive and understandable testing. By using examples produced in conjunction with key stakeholders, we can develop features in collaboration with users to showcase the expected product workflow.
Feel free to check out the code on my personal GitHub at the listed link if you’re interested in digging more into the code. I hope this also encourages you to think about how other technologies and techniques you either use or research can be combined to make a more effective combination. Be inspired by music and mashup your technologies today!
Thanks for reading!
Originally published at http://carlyrichmond.com on February 15, 2021.
