Bloodhound: A Better Breed of API Debugging (Open-Source Microgateway)

Capture, Transform, Track, and Debug Live API Conversations

Today, API Fortress announces the generally available release of Bloodhound, a lightweight API debugging gateway that is free to download and open source. Watch the Bloodhound Demo video or visit the Bloodhound page to see how easily you can start using the most powerful tool for API transaction debugging available today. Download Bloodhound at our GitHub.

Before Bloodhound, it was hard to send API calls to a logger for the right kind of analysis to quickly solve difficult bugs (see examples below). Also, you may have been limited in how you could test APIs. If you’re trying to test an API but can’t leverage the data stuck in a database, your functional and integration tests aren’t going to be as good as they need to be. It’s just one of the many reasons why so many API errors go live. 

A New Best Friend for Developers and Engineers

Here’s how it works:

With Bloodhound, you can route API calls to any logger for comprehensive analysis to uncover solutions to difficult bugs, or test an API in ways not possible before. Now, you can get the right insights to make sure that microservices and APIs are behaving like they should in the real world.

Eliminate More False-Positives and False-Negatives

Simone Pezzano, CTO at API Fortress says, “There are a lot of powerful API gateways, but many are difficult to work with, and not built with the goal to help test and debug APIs. So we were driven to build Bloodhound, a microgateway that you can deploy and use more easily. Deploy Bloodhound locally or in the cloud via a Docker or K8s container.”


Patrick Poulin, CEO at API Fortress adds: “It’s never been easier to build new APIs. But the mindset of how we test and monitor them hasn’t evolved. Writing a handful of functional tests using a small subset of fake data against a staging environment is not enough. With Bloodhound, you can do more. Get more accurate test results by reproducing real world scenarios, and find clarity while trying to debug any problem.”

A FALSE SENSE OF SECURITY

Did you know that the API Fortress platform can reuse your data-driven, functional, integration, and load tests as holistic Functional Uptime Monitors? Use Bloodhound to build the right tests with better capabilities, and then convert those tests into monitors that precisely inform you about functional uptime in any environment before and after going live. For more information, view the eBook: API Monitors: A False Sense of Security – Why API monitors give so many false-negatives and fail to catch human error

USE CASES

Before the generally available release of Bloodhound, the gateway was deployed to several API Fortress customers that are among the world’s largest retail, financial services, healthcare, and telecom companies. While the API Fortress platform is flexible and makes it easy to create addons, customers implemented several out-of-the-box use cases including: 

  • Transform Databases into APIs: Solve the problem of creating data-driven functional tests when test data is locked in a database (PostGres, MySQL, MS SQL Server, MongoDB, Redis, and more).
  • Test APIs Beyond a Normal Functional Test: Extend what can be tested by transforming the API into unique scenarios such as throttling, broken or unexpected headers, invalid payloads, and status code changes, etc.
  • Detect Signals from Noise: Understand the interdependencies in complex API call arrays to help teams create or improve documentation for new API projects.
  • Implement an Echo Server: Understand what requests look like from an API server’s POV to capture issues not revealed during a send or receive.
  • Enforce Internal Policy: Add authorization layers to unsecured APIs – popular when exposing APIs to third parties or contractors.
  • Conduct Live Contract Validation: Compare Swagger/OpenAPI specs to live API transactions to detect potentially dangerous anomalies.

Download Bloodhound for Free

Visit our GitHub for more documentation about Bloodhound. And download Bloodhound.

Adopt Bloodhound

Bloodhound Press Release:

API Fortress Releases Open Source API Debugging Microgateway – Bloodhound
Capture, Transform, Track, and Debug Live API Conversations

New York, NY — June 9, 2020 — API Fortress, the leader in data-driven and functional API testing and monitoring, announces Bloodhound, a lightweight API debugging gateway that is free to download and open source. In less than 3 minutes, developers and engineers can begin using a powerful, purpose-built tool for API transaction debugging. Watch the Bloodhound Demo video

Bloodhound allows teams to route API calls to any logger for comprehensive analysis to uncover solutions to difficult bugs, or test an API in ways not possible before. This gives QA teams the insights to ensure that microservices and database-connected APIs behave as they should in real-world conditions.

Patrick Poulin, co-founder and CEO at API Fortress remarks: “It’s never been easier to build new APIs. But the mindset of how we test and monitor them hasn’t evolved. Writing a handful of functional tests using a small subset of fake data against a staging environment is not enough. With Bloodhound, you can do more. In capturing and transforming your APIs, you can reproduce real world scenarios, and find clarity while trying to debug any problem.”

Before the generally available release of Bloodhound, the gateway was deployed to several API Fortress customers that are among the world’s largest retail, financial services, healthcare, and telecom companies. While the platform is flexible and creating addons is simple, several out-of-the-box use cases included: 

Transforming Databases to APIs: Create data-driven functional tests when test data is locked in a database (PostGres, MySQL, MS SQL Server, MongoDB, Redis, and more).
Testing APIs Beyond a Normal Functional Test: Extend what can be tested by transforming the API into unique scenarios such as throttling, broken or unexpected headers, invalid payloads, and status code changes, etc.
Detecting Signals from Noise: Understand the interdependencies in complex API call arrays to help teams create or improve documentation for new API projects.
Acting as an Echo Server: Understand what requests look like from an API server’s POV to capture issues not revealed during a send or receive.
Internal Policy Enforcement: Add authorization layers to unsecured APIs – popular when exposing APIs to third parties or contractors.
Live Contract Validation: Compare Swagger/OpenAPI specs to live API transactions to detect potentially dangerous anomalies.


For more information about Bloodhound from API Fortress, please visit APIFortress.com.
Download Press Release

Send API Test Results to Elastic and Visualize in Kibana

API Fortress’ API-first architecture means that we can seamlessly integrate with any tool in your toolchain. One of the most popular tooling companies in the world is Elastic. Not only do they provide logging and search, but they have a great visualization tool for all the data they collect called Kibana. With the API Fortress data API, all of your test results can be exported in real-time to the data analysis platform of your choice.

Due to its popularity with our customers, we’ve decided to spend some time discussing the Elastic integration. The official doc is here. The connector is freely available to all customers, and it’s already preloaded in our cloud instance.

Kibana Dashboard

API Fortress test results are incredibly detailed, and that level of detail allows for two major advantages:

  1. Rapid Diagnosis
  2. Pattern Recognition

Our reporting when an assertion fails does much more than simply tell the user there is a single failure. We accelerate diagnosis by telling users which assertion failed, how it failed, what we expected, and even include the header information and the entire payload as well. This is particularly important when creating data-driven tests with hundreds of payloads and results. Our reports can help find the needle in the haystack.

The second advantage is an even more interesting differentiation that we make possible by unifying functional, integration, and load tests into “functional uptime” monitors that can run in any environment. This allows API Fortress to aggregate far more usable real-time data for deeper, more accurate insights. To really explain the utility of data aggregation and analysis, let me describe what happened recently with one of our e-commerce/e-retail customers: 

A large book publisher created two partner APIs, one for listing active ISBNs, another to get product details for ISBNs. The publisher’s internal API monitoring platform was suspiciously reporting 100% uptime when the publisher approached API Fortress about getting a second opinion. The following Monday morning, API Fortress detected hundreds of errors and 404 soft errors from 6-8 a.m. Some limited manual testing was conducted by the publisher, but did nothing to help diagnose the root problem.

The issue happened again the next two weeks. The QA team reported to the chief architect that they couldn’t understand what caused it. Fortunately, the architect had set up API Fortress’ data API to send the data to Elastic, and he had set up a dashboard in Kibana.

When the architect reviewed the data in Kibana, a lightbulb went off. He noticed that the failures always happened on Monday mornings from 6-8 a.m. That’s all he needed to see to know the exact cause. They were using an API gateway, and for performance purposes, they had set up the gateway to cache their listing API. The problem was that they were updating their ISBNs database (which feeds the listing API) every Monday morning at 6 a.m. The gateway only refreshed the cache every two hours. That meant every Monday, for months, the book publisher had exposed hundreds of bad ISBNs for two hours with partners to start their week, and they had no idea. A full analysis revealed that this single uncaught API flaw had caused thousands in lost sales, frustrating partners and causing damage to reputation.

The book publisher had no idea their partner API was failing their partners every Monday morning until they created proper integration tests and monitors. Even in taking those steps, they had trouble diagnosing a remediation to the issue until they had the holistic data to see that the issue happened at the same time every week. While this example was simple, it was indicative of the sort of benefit that can come from integrating with a platform like Kibana for your organization.

This sort of scenario happens far too often. We recommend that you don’t just test on release, but use those functional tests as monitors as well. Then use that rich test data in proper data analysis tools like Elastic + Kibana. Unified testing and monitoring allows you to transform your QA efficiency, effectiveness, and ROI overnight with both legacy and new services.

About Kibana:

Kibana lets you visualize your Elasticsearch data and navigate the Elastic Stack so you can do anything from tracking query load to understanding the way requests flow through your apps.

About API Fortress:

API Fortress is a continuous testing and monitoring platform for APIs that was built from the ground up for shift-left automation and simplified collaboration across teams. By unifying data-driven functional tests with monitoring that can run in any environment, API Fortress detects a much wider range of API issues early in the lifecycle, while significantly accelerating diagnosis with detailed reporting. Now, achieve unlimited quality-at-speed as you integrate API Fortress into any CI/CD platform or DevOps toolchain. Use API Fortress on our hosted cloud at APIFortress.com, or your cloud with a self-managed (on-premises) container.

API Fortress for Cucumber BDD: Add APIs To Your BDD Testing in a Unified Workflow

BDD Testing vs. API Testing

In the API economy, the stories for Minimum Viable Products (MVPs) are becoming more complex as business owners seek faster, and sometimes, more exotic differentiation. For example, a product team at a global bank may aggressively push for open banking to create competitive new features for the bank’s personal finance app. However, developers and testers may not be familiar with, for instance, distributed ledgers and blockchain technologies, resulting in a bottleneck that can significantly delay go-to-market or raise the risk of falling short of the business case.

Behavior-driven Development (BDD) has emerged as a proven methodology to narrow the gap between business owners and developers by improving collaboration throughout the development lifecycle. Test-driven Development (TDD) is closely related to BDD in that both methodologies support continuous testing to reduce software and API defects. However, it is important to know how the two methodologies work in harmony while testing for very different capabilities. BDD frameworks such as the popular Cucumber were never designed to stray into certain core elements of Test-driven Development, particularly, API testing.

In “Why You Shouldn’t Use Cucumber for API Testing” from StickyMinds, developer Byron Katz, writes: 

It is not uncommon for people to use Cucumber (or other BDD framework) to test API endpoints…. [However], Cucumber is a BDD framework, and it should be used solely to support BDD. API testing focuses on coverage of the API endpoints and is more oriented to the technical solution, unlike BDD testing, which is oriented to business capabilities.

Essentially, BDD testing is focused on user acceptance tests that validate a user story. Cucumber offers Gherkin, an easy DSL (domain-specific scripting language) that allows testers or business stakeholders with zero coding background to easily formulate tests in natural language (“prose”) that approximates the user story in terms of “given, when, then” scenarios. These scenarios change slowly. A user story about remotely starting a car via a smart device on a cold night to warm it up before the driver gets into the vehicle does not involve a high number of dynamic requirements. 

But that all changes when reimagining the user story in terms of function, integration, and performance requirements. These requirements involve a high rate of change that should be tested continuously. API testing “digs” behind the user story to validate API function with positive and negative coverage. Complex user stories almost always involve a complex array of API calls. Whereas Cucumber BDD may empower testers to build the remote vehicle app specifically to user stories, Cucumber BDD cannot verify code like API testing when that remote vehicle app must satisfy the user story while taking X amount of time to load. If it takes several minutes for the app to start the vehicle, it may create a terrible customer experience while satisfying the user story (false positive).

In a perfect world, Cucumber BDD testing and API testing are deployed together, and not discretely. Recently, we at API Fortress created guides to show customers how to melt the barriers between BDD and modern API testing. The results are smarter, more effective testing processes that reduce risk throughout the lifecycle, and help to erase fears of “false positives.”

Run API Fortress from Gherkin DSL
Read our Executing from Cucumber doc to learn how you can easily integrate modern API tests powered by API Fortress into your Gherkin DSL (Cucumber). View sample DSL scripts on Github.

Seamless Integration: Cucumber BDD + API Fortress

API Fortress was built from the ground up for continuous testing and TDD, which requires the sharing of critical API testing resources, test results, and reports between stakeholders such as developers, testers, and product. With API Fortress for Cucumber BDD, it is now possible for all stakeholders to close any gaps in understanding of what should be tested and why something doesn’t work. By giving business owners a little insight into how their user stories may impact API functionality, resilience, and performance, API Fortress for Cucumber BDD can transform the efficiency and effectiveness of BDD and TDD in three key areas: 

 

Improve Collaboration Simultaneously validate user acceptance while verifying API function. API Fortress integrates seamlessly with Cucumber, allowing developers, testers, and business owners to view BDD and API test results and reports throughout the lifecycle.
Expand Coverage Start by ensuring the testability of both business and technical capabilities. Verify positive and negative coverage of API endpoints, and run data-driven testing for technical capabilities with a high rate of change.
Accelerate TDD Shift API testing left alongside BDD testing as early as the design stage. API Fortress unfies functional, integration, and load tests to extract more reliable, accurate, and usable testing data. Measure functional uptime to detect API flaws early, and diagnose API flaws quickly.

Add API testing powered by API Fortress
to your Cucumber BDD

Sign up for a free trial of API Fortress for Cucumber BDD: optimize API functionality, resilience, and performance.


Connect API Fortress to Any Database

Easily Connect API Fortress to Any Database

Option 1: Leverage the JDBC component on the API Fortress platform to connect with any  JDBC-compliant database including PostGres, MySQL, and Microsoft SQL.

Option 2: Use an API Fortress helper app to convert most popular databases, CSV and other files into an API, which API Fortress can then use for data-driven testing.

Data-driven Testing (DDT) and Data-driven APIs

Data-driven testing is critical in validating that APIs meet your needs for reliability, resiliency, and performance.

Most mobile and web apps require APIs that connect to multiple databases that undergo constant, iterative changes. Variable data can be provided by the provider or consumer, and the only constant is how inconsistent those requests can be. For example, an ecommerce app may allow users to select different shoes with varying colors, sizes, and prices. 

What this means is that testing those APIs using static calls from something like a  CSV won’t properly reproduce real world conditions. Yet, that’s the most common method of testing APIs.

Fortunately, there is a new push towards data-driven tests (DDT). The obvious utility in this method is allowing APIs to be run in numerous unpredictable manners, and therefore properly tested for normal and edge cases. 

We have years of experience in seeing the difference between using CSVs and proper data-driven tests, and even wrote an ebook detailing some of those – API Fortress eBook. One of the stories involves a book publisher with a partner API that had tens of thousands of ISBNs. That API was used by resellers to know what items were in stock and still for sale. Before using API Fortress the publisher recognized a 99%+ uptime.

When API Fortress was introduced, the publisher created data-driven tests that did simply this:

  1. Call the partner API with all ISBNs
  2. Choose 500+ of those ISBNs at random and dive into the product information

What they found was that hundreds of these allegedly “valid ISBNs” were actually not valid. How could that be possible when the ISBNs were actually pulled from an API of good ISBNs?

Simply put, they were updating their product databases once every two weeks, but weren’t resetting the cache of their API gateway. This isn’t even an API failure, but a database and gateway failure that only proper API testing could have captured. There are more examples in the ebook, but we wanted to convey a really unique problem that is exclusively captured by proper data-driven testing.

Problems such as this are exacerbated by the complex constellation of APIs that each feed off independent databases. Most QA and test automation teams don’t have the time or resources to manually add proper data iteration into their testing suites due to the complexity and time involved. Platforms like API Fortress are built for this sort of testing from Day 1. 

No added complexity. Just smart testing.

$2.84 trillion was the estimated total cost of poor quality software in the U.S. alone during 2018

Source: Consortium for IT Software Quality (CISQ)

Data-driven API Testing Checklist

Data-driven API testing focuses on validating variable data from a database or file repository. Therefore, data-driven testing can be conducted early in the software development lifecycle. But to successfully enable continuous data-driven API testing throughout the life cycle, QA and testing automation teams really need two key elements: centralization and simplicity.

With proper attention to centralization and simplicity, modern data-driven testing provides distributed QA teams with a central repository to unify testing across teams. It also makes it simple for QA teams to ensure that the test logic built from requests and assertions accurately captures real world scenarios at the API level. 

Here’s a Best Practices Checklist for modern data-driven API testing:

  • Centralize data-driven tests and queries for easy reuse

  • Simplify test execution in multiple environments

  • Connect databases with JDBC drivers or convert databases or files into APIs for connections

  • Store variable data sets for easy sharing and reuse

  • Design data-driven test logic, and designate data-driven tests at the Global or Project level

  • Enable any team members to run tests

Data-driven Testing with API Fortress

Sign up for a free trial of API Fortress and put your data-driven testing to the test. Schedule a demo today.

Automate a Jenkins CI/CD Pipeline with API Fortress

Automate a Jenkins CI/CD Pipeline with API Fortress

With a CI/CD pipeline, the work of distributed teams come together in an automated flow to build, test, and deploy new code. That means rewriting the rules of how releases are built and tested. One of the first things that the Jenkins wiki (Jenkins Best Practices) tells newcomers to CI/CD is that “unit testing is often not enough to provide confidence [of desired quality].” The wiki then talks about the necessity to automate API testing throughout the lifecycle to ensure that all distributed teams are continually working with good services and data.

Let’s take a closer look at those two stipulations of a Jenkins (or any) CI/CD pipeline: 

  • Run API Testing Continuously: CI/CD pipelines produce iterative releases so that services and mobile apps can evolve quickly without increasing the number of bugs or vulnerabilities released. Thousands of enterprises are trying to move from monolith to microservices/modern APIs, but face challenges properly incorporating their Jenkins CI/CD pipelines.

API Fortress was built from the ground up to solve these issues for the digital enterprises of today. Our new breed of API testing automation includes unique capabilities to standardize and collaborate across teams thanks to a platform architecture that can be deployed on-premises or cloud. This is good news for any enterprise that needs to strike the right mix of speed and quality concerning their new IT investments and initiatives.

Download The Datasheet


How to Integrate API Fortress with Jenkins

1. APIF-Auto Command-Line Tool: API Fortress is an API-first platform with a robust set of APIs. To make life easier for our customers we created a command-line tool named APIF-Auto that makes it very easy to add a pipeline script and get results in JUnit format. To learn more about that setup you can read here.

2. Connect by API: API Fortress allows anyone with manager access to create a webhook, that can be easily called from within Jenkins. You can read the details on that here.

Advantages of Using the API Fortress with Jenkins

    • Total Automation: Fire your entire test suite, or just some tests with specific tags, as part of your CI pipeline. Test in seconds what used to take days or weeks.
    • Minimal Setup Required: Use the command-line tool or webhook to automate test execution in minutes.
    • Flexibility in Workflows: API Fortress can execute tests stored in the platform, or from your chosen VCS. See those docs here, or speak to your API Fortress rep to learn more.
    • Works with Any CI/CD: Thanks to APIs the platform is completely CI platform agnostic. If your organization changes to another platform such as Azure DevOps, TravisCI, or Bamboo, for example, the integrations are nearly identical and just as simple.

SECURE YOUR API TESTING RESULTS AND DATA

Deploy on-premises or hybrid cloud using our Kubernetes or Docker deployments. Or use our SaaS platform at apifortress.com. Get complete flexibility to allow your company to experience complete control over data and tests, with minimal setup headaches. Use Kubernetes or Docker

We invite you to sign up for a free trial and demo of API Fortress and put your testing to the test.

New Feature! Test Data Generation

API Fortress is constantly looking to help you make your tests and API mocks stronger and smarter. The next big feature for us with this goal is test data generation!

Now, you can dynamically generate test data from within tests and mocks on the API Fortress platform. Generate fake names, emails, addresses, and much more. Stay away from small, static datasets, and leverage the platform’s ability to produce large quantities of dynamic data. It’s time to look beyond your old CSV files, and use APIs, Databases, or test data generation to power tests.

Learn more on the docs here. Please note this feature is available in API Fortress version 17.1.0

API Fortress Automates 3-Legged OAuth for API Testing and Monitoring

New York City — July 25, 2019 — API Fortress, the leader in continuous API testing, announces 3loa Helper, an open source application that automates 3-legged OAuth 2.0 flows from the world’s largest social and search providers. By simply integrating API Fortress with 3loa Helper, developers and test engineers can test and validate 3-legged OAuth flows.

“Too many tests today don’t truly reproduce the user flows a production API sees. This leaves risky holes in a test plan, and ignores what is often the very first step for users.”

Patrick Poulin, CEO and co-founder at API Fortress

It is difficult to automate 3-legged OAuth 2.0 flows for API testing because 3-legged OAuth 2.0 was specifically designed to require user intervention. 3loa Helper solves this problem by creating a UI interaction to execute the 3-legged OAuth 2.0 flow, and then generates an API for consumption by API Fortress (or any system). In this way, engineers can fully capture real world user behavior, which is critical as more enterprise users embrace 3-legged OAuth.

Now, API Fortress users may validate API authentication and handoff work consistently to collect accurate data about the reliability of login procedures for users. Enterprises should know how third-party authorization solutions are affecting their users.

For more information, watch the 3loa Helper video with API Fortress CTO and co-founder, Simone Pezzano. If you would like to collaborate in the open source project, go to the GitHub page.


3-Legged OAuth
3-Legged OAuth Automation

API Fortress Announces a New Test Creation Application for Engineers

New York City — June 20, 2019 — API Fortress announces the release of Forge, a lightweight downloadable test editor that increases flexibility in how you choose to write API tests. With Forge, a user can write a test with API Fortress on their own computer, outside of the platform.

In addition, API Fortress is releasing apif-local, an application that contains the core of the API Fortress platform. With the combination of Forge and apif-local, users can leverage the API Fortress platform however they choose, and transition from localhost to the platform seamlessly. The ability to work locally without losing functionality empowers any engineer to create API tests without obstructing their usual process.

“Continuous testing is a critical need that should involve both testers and engineers. With Forge and apif-local, engineers can take part in writing tests that validate their APIs alongside the code itself, with minimal workflow impact.”

– Simone Pezzano, CTO at API Fortress

Forge and apif-local are available for a free trial. If you would like to try the platform visit API Fortress or contact info@apifortress.com.

Media Contact: Gabe Kaufman at gabe@apifortress.com

About API Fortress:
At API Fortress, we believe that continuous API quality is the key to significantly accelerating software releases while reducing risk. And we believe that our API testing platform made from APIs is the best way to ensure continuous API quality. Teams can easily integrate our platform with existing version control, case management, CI/CD platforms, and other IT investments. Getting started is simple: either generate or write tests in your own IDE, our Forge IDE, or a GUI. Now, distributed teams of developers and test engineers can collaborate and standardize on a single testing strategy across functional, non-functional and performance testing. API Fortress tests and monitors SOAP, REST, and GraphQL APIs.

API Fortress Announces SAML 2.0-based Single Sign-On (SSO)

NEW YORK, June 3, 2019 — API Fortress now offers SAML 2.0-based Single Sign-On (SSO) for enterprise customers. This release candidate gives administrators a simple way to control access rights as well as quickly add or change teams on the API Fortress platform. With SSO, insights about API quality along with detailed reporting and smart notifications can be made available to the right teams, including product and support.

CEO at API Fortress, Patrick Poulin remarks:
“Too many enterprises run into roadblocks caused by testing bottlenecks, which can cause significant delays in go to market. That’s why we built API Fortress: to eliminate testing bottlenecks by platformizing API testing and making it easy to embrace continuous testing. SSO is another feature of our platform that further helps teams complete testing cycles more efficiently and on schedule.”

As an API-first API testing platform, API Fortress has made it easy to integrate with existing IT investments across version control systems, test case managers, API managers, CI/CD platforms, reporting, and notification apps.

CTO at API Fortress, Simone Pezzano comments:
“In addition to improving user productivity, SAML 2.0-based SSO on API Fortress extends our interoperability with business systems and platforms, as well as simplifies compliance with companies’ security policies. If an organization is on a journey to agile or hybrid agile development, microservices or CI/CD, it means increased regression testing on iterative products. SSO makes it easier to extend a standardized testing strategy across distributed teams on those journeys.”

SAML 2.0-based SSO is available as a release candidate to existing and new API Fortress customers. For more information, please visit API Fortress.

About API Fortress
At API Fortress, we believe that continuous API quality is the key to significantly accelerating software releases while reducing risk. And we believe that our API testing platform made 100% from APIs is the best way to ensure continuous API quality. Teams can easily integrate our platform with existing version control, case management, CI/CD platforms and other IT investments. Getting started is simple: either automate or write tests in your own IDE, our Forge IDE, or a GUI. Now, distributed teams of developers and test engineers can collaborate and standardize on a single testing strategy across functional, non-functional and performance testing.