Category: Course 2: Best Practices
2. Best Practice – Advanced
Variablize for Test Reuse
Domain, endpoints, and data sets are some of the parts of a test that should be easy to change quickly. You should never have to duplicate a test to execute it against a new staging environment. If you are doing that than your system needs to be reconsidered.
As much as possible aim to have only a single test of a certain type. If you built a partner API, then try to have a single detailed test that can be a regression test during a CI pipeline, and then also used to monitor the live environment. [Learn More]
Business Logic Validating
As described in Course 1, it is important to go beyond the technical facts of a payload. You should also have knowledge as to the business objects. This means that if the Uber payload includes a reference to UberSidecar, but Uber hasn’t announced a Sidecar product yet, this is a major issue for them from a technical and business secrets perspective.
Perhaps they have a major advertising initiative to announce the ability to order a ride in the sidecar of a motorcycle, and this would undermine that. It’s important that your tests are knowledgeable about the goal of the APIs technically, as well as from a business perspective.
Using Lots of Data
One of the most common errors we see with testing is using the same small set of data for every test. A CSV with 50 product IDs is a good start, but now that we can automate this work there is no reason to not write a much more comprehensive test that can validate hundreds of product IDs. You must break free from using small CSV files for datasets.
If you are already using databases or APIs that’s great! If not, we suggest you start this process as well. A test is only as good as the amount of information it is testing against.
[Learn More Using Files | Using JDBC]
Integration / End-to-End Test
Most API programs are a collection of APIs that are meant to interact with each other. Therefore, it is important to create tests that do exactly that. A singular test that goes through a series of calls and validates the responses at each stage. This is what is called an integration test, although it is also called an end-to-end test.
This means writing detailed tests that reproduce normal user flows. For example, if we are talking about an ecommerce platform, a normal user flow would be:
Search > View Product Details Page > Add to Cart > Checkout
A functional test for just the Search API is a great start, but the entire flow also needs to be validated. You should have as many integration tests as there are expected user flows. [Learn More]
Confirm Response Matches Request
This is in the same thinking as the above, but with that advent of PSD2 and Open Banking it’s worth mentioning on its own. When you are requesting information, such as your personal banking history, it is important to make sure only your information is returned. It is also important to validate you can do wildcard searches and get the results of other.
That is exactly the sort of functional error that causes a huge vulnerability for large organizations like the United States Post Office.
Integrations Galore
Every company has an existing workflow, and it’s important to choose a platform that helps keep those integrations seamless. In the basics we talked about integrating with your CI pipeline, but it’s also useful to have notifications and data in one place. If your organization has a lot invested in Slack, Kibana, and TestRail (for example) than having API test notifications and results in one place is imperative. [Learn More]
1. Best Practice – Basic
Test Every Endpoint
Every endpoint is an integral part of an API program, and that means validating each one works as expected. This even means testing endpoints that add items into a database like Create User. A smart platform allows you to create tests that can add users, validate they were properly entered, and if it’s a success then the user is deleted. [Learn More]
Test Everything
API payloads are filled with information. You should not only be testing every object, but the header also contains vital information. Response-Type, Statuscode, Content-Type, every piece of information in an API call is important to validate.
Test Objects Thoroughly
When testing an API you should make sure all the objects exist, but that is only the first step. You should also be validating the objects have data associated that is within an expected range.
For example, in the below Uber payload you’ll see that
- image should always be a properly formatted URL
- shared should be boolean (true or false)
- display_name should be UberPool, UberX, UberSUV, UberEats, and whatever other products they have available in that city.
Test & Project Organization
Before you begin creating tests for your API program, you should first consider how you want to organize them now and in the future. You may choose to create unique projects for different use cases, or tag the tests thoroughly using a system you create to separate tests that work for version 1 but not version 2. Come up with an organization plan that takes into consideration…
- Expected Audience
- Integration Test vs Simple Endpoint Test
- Versioning
- Legacy vs New APIs
Monitor Your APIs
Again, monitoring is the most useful and yet underused form of testing. You need to know more about an API than if it’s up or down, you need to validate it’s functioning correctly.
Schedule your existing functional and integration tests against the live environment to truly understand how your APIs are working in the real world. Every API program should have a status page. [Learn More]
Automate!
The key to delivering new and updated APIs quickly, and with confidence of their quality, is with test execution automation. This is particularly important for organization using CI/CD platforms like Jenkins, TravisCI, Bamboo, Azure DevOps, and more. Add your test suite to your pipelines as soon as you can. [Learn More]