BDD and Automated UI Testing in .NET Part 1: Tools and Architecture

Introduction

This post aims to cover the practical and architectural aspects of setting up Behavioural Driven Development (BDD) in a .NET-based web application project which is using an agile methodology.

To put it in oversimplified terms, BDD starts with business requirements defined as features and their associated acceptance criteria. In the popular Cucumber framework, these are usually defined in the Gherkin language and stored in .feature files. Due to the business-readability of Gherkin, these feature files can serve a double duty as both the documentation of business requirements as well as the tests which the developers' software must pass. Instead of developing to tests in a technical language (eg. "when the LoginController receives a request, then the Login view should be served") which offer little business value and really should be the purview of technical staff anyway, software must pass business-readable tests (eg. "when I click the login button, then I am navigated to the login page and a username field is present and a password field is present"). These tests can theoretically be written by business users, although in practice this is still in the domain of the testers. At the very least, this gives a common, easily understandable and common language (or a Domain Specific Language) between developers, testers and business users.

There is a lot of material out there which covers this topic in-depth, here are some excellent posts covering the overall process itself:

• https://github.com/cucumber/cucumber/wiki/Gherkin

Issues

In an agile methodology, software features are delivered in iterations or sprints. In order to assure that new features being introduced do not break any previously developed and verified features, it is important to re-test the software as a whole at each sprint delivery point.

At the bare minimum, the acceptance criteria must be executed manually by testers. We can already see that, in this agile environment, the job of regression testing previously-implemented features gets larger and larger for each sprint. Automation of this regression testing yields massive rewards to catch defects early and prevent the defect list from mounting towards the end of a project. Fortunately, the structured-yet-business-readable Gherkin language lends itself quite well to be converted into executable unit tests, which is where the popular Cucumber framework has come into play over the last decade or so.

Additionally, the aims of BDD are such that the deliverable software is specified, developed and tested against its expected behaviour. In the context of a custom web application, the software's behaviour is primarily driven by user input into a web browser. There are clunky ways to do this with the standard .NET framework, but the most accurate and robust test would be to emulate user actions in an actual web browser. To this end, a robust web driver framework is required such that unit test code can programmatically automate/drive web browser actions.

Architecture

In order to overcome these issues, an automated UI testing framework can be developed with the following components:

• SpecFlow - to consume Gherkin feature files and produce corresponding executable NUnit test methods.
• Selenium - to provide API to automate and drive web browsers when executing NUnit tests.
• Team City - to provide a continuous build environment to build and execute tests. Tests can also be set to execute on a scheduler for regular smoke tests.
• Octopus - to provide deployment component in continuous delivery pipeline. Automated UI tests can be set to execute as a post-deployment process to catch regression defects in a test environment.

Here is a summary of the proposed architecture:

Benefits

The benefits of this solution are manifold:

• With good test coverage, regression testing is performed on the entire software product with minimal effort
• Testers can have more capacity to perform higher-value exploratory testing rather than getting bogged down in things like field-level checks, which are mundane yet repeatable (and thus ideal candidates to be automated) tasks
• Defects are caught much earlier in the development cycle, reducing costs of bug fixing later in the project

Note that this post is not advocating this solution as a one-size-fits-all approach, nor is this solution and tooling specifically tied to BDD; BDD is a more of a conceptual practice rather than a technical one. This solution addresses the issues of regression testing a web application project and fitting in with a BDD project.

Below, we will explore the specific tools.

SpecFlow

http://www.specflow.org/

SpecFlow the official .NET implementation of Cucumber and is able to convert Gherkin language acceptance criteria into executable unit tests. This is the main technical component which bridges the gap between the testers and developers. 

SpecFlow can be added as a NuGet package (source) to a Visual Studio project. The project will hold the acceptance criteria feature files written by testers as *.feature files. SpecFlow will then generate corresponding empty unit tests for the test engineer to complete. These unit tests can then be executed by a supported unit test provider just like any other unit tests, with the difference being mainly that these tests are specified around the Given, When, Then framework of the Gherkin features.

SpecFlow has built-in functionality to produce a comprehensive test execution report on each run. This can be scripted/automated to be sent to key stakeholders after each run.

Excel Integration

SpecFlow comes with a plugin called SpecFlow+ Excel, which allows integration with Excel such that it can pull acceptance criteria and acceptance criteria input data (ie. Examples in the Gherkin language) into feature files and thus generate corresponding test cases. 

This becomes extremely handy in a software project which contains an integration component, where many fields may need to be mapped as inputs and outputs across multiple systems. These field properties and mappings may be stored in a data dictionary which forms the source of truth of field-level requirements for the entire team and is stored in a centralised location (perhaps on a SharePoint installation). 

As NUnit generates and detects test cases on build of the Visual Studio project, it becomes important to update the referenced Excel file when the centralised data dictionary is updated. This can become cumbersome and error-prone, however it is possible to catch such errors using a pre-build event call to a script to download the latest copy of the data dictionary. SpecFlow will generate the unit test code with a default body of 

ScenarioContext.Current.Pending();