Best Practices – Optional

Integrate with your CI/CD

Integrating your API testing into your existing CI/CD flow is key if you are trying to keep all of your deployments quick and frequent. Kicking off a test or suite of tests during the build process allows you to build your new deployment and test it all at once.

Integrate Notification/Data Analysis Systems

Set up notifications to alert you via email, slack, and more when tests fail.

Do you already use a platform to analyze all your data, well you can send all your API Fortress test data out to those platforms as well. Now all of your data can live in one place, making it easy to pull and analyze all project data.

Integrate with Test Case Managers

Integrating with your Test Case Manager and communicating the results of tests is important. Having all of your information in your test case manager including test result data makes it easy to understand the entire project. Connectors can be setup to automatically send test results to your test case manager upon completion of scheduled tests.

Best Practice — Advanced

Variablize for Test Reuse

Domain, endpoints, and data sets are some of the parts of a test that should be easy to change quickly. You should never have to duplicate a test to execute it against a new staging environment. If you are doing that than your system needs to be reconsidered.

As much as possible aim to have only a single test of a certain type. If you built a partner API, then try to have a single detailed test that can be a regression test during a CI pipeline, and then also used to monitor the live environment. [Learn More]

Business Logic Validating

As described in Course 1, it is important to go beyond the technical facts of a payload. You should also have knowledge as to the business objects. This means that if the Uber payload includes a reference to UberSidecar, but Uber hasn’t announced a Sidecar product yet, this is a major issue for them from a technical and business secrets perspective.

Perhaps they have a major advertising initiative to announce the ability to order a ride in the sidecar of a motorcycle, and this would undermine that. It’s important that your tests are knowledgeable about the goal of the APIs technically, as well as from a business perspective.

Using Lots of Data

One of the most common errors we see with testing is using the same small set of data for every test. A CSV with 50 product IDs is a good start, but now that we can automate this work there is no reason to not write a much more comprehensive test that can validate hundreds of product IDs. You must break free from using small CSV files for datasets.

If you are already using databases or APIs that’s great! If not, we suggest you start this process as well. A test is only as good as the amount of information it is testing against.
[Learn More Using Files | Using JDBC]

Integration / End-to-End Test

Most API programs are a collection of APIs that are meant to interact with each other. Therefore, it is important to create tests that do exactly that. A singular test that goes through a series of calls and validates the responses at each stage. This is what is called an integration test, although it is also called an end-to-end test.

This means writing detailed tests that reproduce normal user flows. For example, if we are talking about an ecommerce platform, a normal user flow would be:

Search > View Product Details Page > Add to Cart > Checkout

A functional test for just the Search API is a great start, but the entire flow also needs to be validated. You should have as many integration tests as there are expected user flows. [Learn More]

Confirm Response Matches Request

This is in the same thinking as the above, but with that advent of PSD2 and Open Banking it’s worth mentioning on its own. When you are requesting information, such as your personal banking history, it is important to make sure only your information is returned. It is also important to validate you can do wildcard searches and get the results of other.

That is exactly the sort of functional error that causes a huge vulnerability for large organizations like the United States Post Office.

Integrations Galore

Every company has an existing workflow, and it’s important to choose a platform that helps keep those integrations seamless. In the basics we talked about integrating with your CI pipeline, but it’s also useful to have notifications and data in one place. If your organization has a lot invested in Slack, Kibana, and TestRail (for example) than having API test notifications and results in one place is imperative. [Learn More]

Best Practice – Basic

Test Every Endpoint

Every endpoint is an integral part of an API program, and that means validating each one works as expected. This even means testing endpoints that add items into a database like Create User. A smart platform allows you to create tests that can add users, validate they were properly entered, and if it’s a success then the user is deleted. [Learn More]

Test Everything

API payloads are filled with information. You should not only be testing every object, but the header also contains vital information. Response-Type, Statuscode, Content-Type, every piece of information in an API call is important to validate.

Test Objects Thoroughly

When testing an API you should make sure all the objects exist, but that is only the first step. You should also be validating the objects have data associated that is within an expected range.

For example, in the below Uber payload you’ll see that

  • image should always be a properly formatted URL
  • shared should be boolean (true or false)
  • display_name should be UberPool, UberX, UberSUV, UberEats, and whatever other products they have available in that city.

Test & Project Organization

Before you begin creating tests for your API program, you should first consider how you want to organize them now and in the future. You may choose to create unique projects for different use cases, or tag the tests thoroughly using a system you create to separate tests that work for version 1 but not version 2. Come up with an organization plan that takes into consideration…

  • Expected Audience
  • Integration Test vs Simple Endpoint Test
  • Versioning
  • Legacy vs New APIs

Monitor Your APIs

Again, monitoring is the most useful and yet underused form of testing. You need to know more about an API than if it’s up or down, you need to validate it’s functioning correctly.

Schedule your existing functional and integration tests against the live environment to truly understand how your APIs are working in the real world. Every API program should have a status page. [Learn More]

Automate!

The key to delivering new and updated APIs quickly, and with confidence of their quality, is with test execution automation. This is particularly important for organization using CI/CD platforms like Jenkins, TravisCI, Bamboo, Azure DevOps, and more. Add your test suite to your pipelines as soon as you can. [Learn More]