The (hidden) holy grail
In theory, BDD seems to be the holy grail, while in practice, BDD proves to be a lot more difficult to adopt. One of the main causes on this account is that BDD actually alters the traditional approach to software development1. BDD introduces high involvement of every role in the team, including the product owner, to the process, which poses another challenge2. Adopting a new software development method on its own already requires a lot of effort3, which is not any different for BDD. All team members should be able, and willing, to collaborate in order to make BDD worthwhile.
In certain cases, for example, testers and developers are not used to working closely together, but rather reside in separate teams, like the V-Shaped model4 proposes. When developers and testers do work closely together, there might still be conflicts that are in the way of proper collaboration5. In order to adopt BDD properly, the entire team must therefore not only master the new practices that come with BDD, but in most cases also drastically alter the way they collaborate with each other. Therefore, a development team should require everyone’s support and the time and resources in order to accomplish the altered approach to software development. The biggest advantage of BDD is that all those who are involved are able to collaboratively contribute2. Not willing to collaborate evades the goal of BDD, making it difficult to successfully adopt its practices.
Even in case there is optimal collaboration within the development team, BDD still requires an initial learning curve1, which organisations need to overcome to benefit from the advantages of BDD2. In order to avoid adopting BDD improperly, the team should deep-dive into BDD before they decide to suddenly adopt BDD in its entirety for one of their projects. If BDD is not properly adopted from the beginning, chances are it will alienate certain team members from following the practices of BDD, making it difficult to uphold and maintain BDD in the future. Important is that the team is aware of the misconceptions of BDD and that BDD is not being adopted for the wrong reason (e.g., end-to-end testing). Mastering all three phases of BDD, discovery, formulation and automation, requires extensive practice, as each phase has its own challenges, which is not always feasible due to a possible lack of time or resources.
A proposal for a custom approach based on maturity
Given the misconceptions and pitfalls that surround BDD, it proves difficult to adopt BDD correctly in its entirety. In order to facilitate the adoption of BDD, we therefore propose a custom approach to adopting BDD, based on a certain level of maturity. For each of the three phases in BDD, three maturity levels have been contrived. For each level of maturity, a minimum amount of effort is required in order to complete that specific phase and to be able to advance to the next, as presented in the matrix below.
BDD starts with the most important phase, i.e., discovery. In this phase it is important to deliberately try and discover what is still unknown, while creating a shared understanding by specifying acceptance criteria, or business rules, by examples. Ideally, at the highest level of maturity, it is suggested that at least the “three amigos” (i.e., Tester, Developer and Business Analyst) explore a user story together in a requirements workshop (e.g., in an example mapping session). By doing so, the three amigos can each bring their own experience to the table, broadening the exploration area of the user story and shortening feedback loops that would otherwise be required. However, as mentioned before, the simultaneous collaboration is not always possible. Therefore, on a lower level, it might be easier at first to only involve those that were initially interested in adopting BDD and to adopt BDD in an “undercover” manner. Instead of organising specific three amigos meetings for a requirements workshop, try to bring the requirements workshop to regular meetings in which at least the three amigos are present, without mentioning the goal of the meeting. If it is not possible to bring at least the three amigos together for a requirements workshop, or if the “undercover BDD” is not working due to the three amigos not actively looking for examples to specify the acceptance criteria, on the lowest level, you could still perform a requirements workshop on your own, where you specify the acceptance criteria with examples on your own. After doing so, you can try and discuss the examples with the three amigos. The lowest level is, however, far from ideal, as the examples are based on only one person’s point of view.
The second phase of BDD encompasses formulating the examples that have been gathered during a requirements workshop into business readable Gherkin (GWT) scenarios. The goal of this phase is to verify that the tester and developer both understand how the behaviour of the application should function. In an ideal situation, the scenarios are written in pair by the developer and tester and are then reviewed by at least the product owner. At this level of maturity, it is required that the product owner is interested in reading the formulated scenarios, in order for the scenarios to be reviewed thoroughly. This step thus requires a good understanding of how scenarios should be written, in order not to alienate the product owner from reviewing the scenarios. While formulating scenarios might show to seem very simple, it is a rather tedious task. Oliveira et al., even dedicated a research paper to propose a question-based checklist to assess the quality of a scenario6. At a lower level, you might therefore practice writing good scenarios first, ideally with the entire team. If there is not much interest in the scenarios yet by the team, it might still be beneficial to start with providing only the scenarios that are of interest to the team (level 2), or to even only share the titles of the examples that have been specified in the discovery phase (level 1). Even only the scenario titles could already serve as a guideline for automating tests, especially when already practicing TDD.
In the third phase, automation, the input of the formulation phase is used for automating tests. Ideally the reviewed scenarios are integrated in the full test stack, covering mainly Unit and integration tests, in addition to programmer tests. It’s also possible that certain tests will not be automated, but are at least marked as being tested manually. By integrating the scenarios in the full test stack, there is an automatic system that warns if certain behaviour does not work as intended (anymore), providing for the benefit of living documentation. When integration in the full test stack is not possible yet, on a second level, it might be a good idea to start using the scenarios only for end-to-end testing. However, when using the scenarios solely for end-to-end testing, don’t forget to keep in mind that the intent of formulating scenarios will always be enabling collaboration, not end-to-end testing. Executing end-to-end tests just doesn’t always require extensive technical skills and can avoid the need to alter the development environment. Instead, the tests can be automated in a separate environment, which also provides for a system that warns if tests are out of date, so that there still is the benefit of living documentation. On a lower level, if there are also no resources to automate the scenarios as end-to-end tests, it would still be of use to utilize the scenarios to execute manual tests. Giving the formulated scenarios a purpose ensures that the scenarios are decently written and that the practices of BDD will be properly upheld. As a minimal effort in the lowest level of maturity, the scenarios are all still best used to execute manual tests and kept as documentation in order to maximize its usefulness.
Sources:
1Binamungu LP, Embury SM, Konstantinou N: Maintaining behaviour driven development specifications: Challenges and opportunities. 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER) 2018; 175-184.
2Pereira L, Sharp H, de Souza C, Oliveira G, Marczak S, Bastos R: Behavior-driven development benefits and challenges: reports from an industrial study. Proceedings of the 19th International Conference on Agile Software Development: Companion 2018; 1-4.
3Nikitina N, Kajko-Mattsson M: Guiding the adoption of software development methods. ACM International Conference Proceeding Series, 2014.
4Ruparelia N: Software development lifecycle models. ACM SIGSOFT Software Engineering Notes 2010; 35:8–13.
5Zhang X, Stafford T, Dhaliwal J, Gillenson M, Moeller G: Sources of Conflict between Developers and Testers in Software Development. Information & Management 2014; 51:13-26.
6Oliveira G, Marczak S, Moralles C: How to Evaluate BDD Scenarios’ Quality? Proceedings of the XXXIII Brazilian Symposium on Software Engineering 2019; 481-490.