UI Testing is Not Enough

“Our UI Tests Hit the APIs”

This is a true statement. It’s always true when you are doing things such as testing native mobile applications, for example. The problem is that it implies something that isn’t true. It implies that APIs are then being tested, which is not the case. The APIs are being triggered and running, but that is not a test. There is a big difference between being in class during roll call, versus passing a test. APIs are websites & apps without UIs. They are direct links to the data and information, and therefore can have many paths to functional failure that are too often missed when only UI tests are done. We find this topic to be super interesting and important, and so let’s look into a real world example we recently encountered involving a food delivery application:

Now here is an example of where problems can (and do) occur…

Did you notice anything wrong with the payload?

There are three errors in that JSON response, and it’s unlikely
any of them would be caught by an Appium test.

First, Great Pizza is 75 miles away and shouldn’t be in the list.

Why was it listed? That can be any number of errors such as the app sent the wrong GPS location, the range wasn’t set properly, or the platform doesn’t properly confirm delivery range.

Second, Great Pizza’s delivery fee of $5.5 has a comma in the middle.

Commas are actually used instead of decimal points in many countries, and this data could have been uploaded by someone from Canada (for example).

Finally, Great Pizza’s hero image is the same as Good Pizza.

If an API call is made asking for Great Pizza’s hero image, then returning Good Pizza’s hero is incorrect. A proper API test could validate the store ID of the request matches directory ID.

Forrester analyst, Diego Lo Giudice, looks at the rise in the popularity of
API testing in the Forrester Wave in which he writes,
“Modern applications require a shift of the current 80% UI automation… shifting about 80% of that test automation [to] API test automation.”

Giudice also writes, “Modern applications are composed with APIs and services… [UI testing] cannot easily orchestrate tests that need to verify functional paths and back-end APIs and services.”

APIs are data. Data requires a different concept when it comes to testing, but not a complicated one. All information should be formatted correctly, accurate, within expected ranges, and as per what the original API designs were. Testing UI is imperative to delivering quality features and products, but ignoring the APIs that power those UIs with data and information is like only looking at the cover of a book before publishing.

Open the book and do some detailed proofreading with API Fortress.