BDD Testing vs. API Testing
In the API economy, the stories for Minimum Viable Products (MVPs) are becoming more complex as business owners seek faster, and sometimes, more exotic differentiation. For example, a product team at a global bank may aggressively push for open banking to create competitive new features for the bank’s personal finance app. However, developers and testers may not be familiar with, for instance, distributed ledgers and blockchain technologies, resulting in a bottleneck that can significantly delay go-to-market or raise the risk of falling short of the business case.
Behavior-driven Development (BDD) has emerged as a proven methodology to narrow the gap between business owners and developers by improving collaboration throughout the development lifecycle. Test-driven Development (TDD) is closely related to BDD in that both methodologies support continuous testing to reduce software and API defects. However, it is important to know how the two methodologies work in harmony while testing for very different capabilities. BDD frameworks such as the popular Cucumber were never designed to stray into certain core elements of Test-driven Development, particularly, API testing.
In “Why You Shouldn’t Use Cucumber for API Testing” from StickyMinds, developer Byron Katz, writes:
It is not uncommon for people to use Cucumber (or other BDD framework) to test API endpoints…. [However], Cucumber is a BDD framework, and it should be used solely to support BDD. API testing focuses on coverage of the API endpoints and is more oriented to the technical solution, unlike BDD testing, which is oriented to business capabilities.
Essentially, BDD testing is focused on user acceptance tests that validate a user story. Cucumber offers Gherkin, an easy DSL (domain-specific scripting language) that allows testers or business stakeholders with zero coding background to easily formulate tests in natural language (“prose”) that approximates the user story in terms of “given, when, then” scenarios. These scenarios change slowly. A user story about remotely starting a car via a smart device on a cold night to warm it up before the driver gets into the vehicle does not involve a high number of dynamic requirements.
But that all changes when reimagining the user story in terms of function, integration, and performance requirements. These requirements involve a high rate of change that should be tested continuously. API testing “digs” behind the user story to validate API function with positive and negative coverage. Complex user stories almost always involve a complex array of API calls. Whereas Cucumber BDD may empower testers to build the remote vehicle app specifically to user stories, Cucumber BDD cannot verify code like API testing when that remote vehicle app must satisfy the user story while taking X amount of time to load. If it takes several minutes for the app to start the vehicle, it may create a terrible customer experience while satisfying the user story (false positive).
In a perfect world, Cucumber BDD testing and API testing are deployed together, and not discretely. Recently, we at API Fortress created guides to show customers how to melt the barriers between BDD and modern API testing. The results are smarter, more effective testing processes that reduce risk throughout the lifecycle, and help to erase fears of “false positives.”
Seamless Integration: Cucumber BDD + API Fortress
API Fortress was built from the ground up for continuous testing and TDD, which requires the sharing of critical API testing resources, test results, and reports between stakeholders such as developers, testers, and product. With API Fortress for Cucumber BDD, it is now possible for all stakeholders to close any gaps in understanding of what should be tested and why something doesn’t work. By giving business owners a little insight into how their user stories may impact API functionality, resilience, and performance, API Fortress for Cucumber BDD can transform the efficiency and effectiveness of BDD and TDD in three key areas:
|Improve Collaboration||Simultaneously validate user acceptance while verifying API function. API Fortress integrates seamlessly with Cucumber, allowing developers, testers, and business owners to view BDD and API test results and reports throughout the lifecycle.|
|Expand Coverage||Start by ensuring the testability of both business and technical capabilities. Verify positive and negative coverage of API endpoints, and run data-driven testing for technical capabilities with a high rate of change.|
|Accelerate TDD||Shift API testing left alongside BDD testing as early as the design stage. API Fortress unfies functional, integration, and load tests to extract more reliable, accurate, and usable testing data. Measure functional uptime to detect API flaws early, and diagnose API flaws quickly.|