Send API Test Results to Elastic and Visualize in Kibana

API Fortress’ API-first architecture means that we can seamlessly integrate with any tool in your toolchain. One of the most popular tooling companies in the world is Elastic. Not only do they provide logging and search, but they have a great visualization tool for all the data they collect called Kibana. With the API Fortress data API, all of your test results can be exported in real-time to the data analysis platform of your choice.

Due to its popularity with our customers, we’ve decided to spend some time discussing the Elastic integration. The official doc is here. The connector is freely available to all customers, and it’s already preloaded in our cloud instance.

Kibana Dashboard

API Fortress test results are incredibly detailed, and that level of detail allows for two major advantages:

  1. Rapid Diagnosis
  2. Pattern Recognition

Our reporting when an assertion fails does much more than simply tell the user there is a single failure. We accelerate diagnosis by telling users which assertion failed, how it failed, what we expected, and even include the header information and the entire payload as well. This is particularly important when creating data-driven tests with hundreds of payloads and results. Our reports can help find the needle in the haystack.

The second advantage is an even more interesting differentiation that we make possible by unifying functional, integration, and load tests into “functional uptime” monitors that can run in any environment. This allows API Fortress to aggregate far more usable real-time data for deeper, more accurate insights. To really explain the utility of data aggregation and analysis, let me describe what happened recently with one of our e-commerce/e-retail customers: 

A large book publisher created two partner APIs, one for listing active ISBNs, another to get product details for ISBNs. The publisher’s internal API monitoring platform was suspiciously reporting 100% uptime when the publisher approached API Fortress about getting a second opinion. The following Monday morning, API Fortress detected hundreds of errors and 404 soft errors from 6-8 a.m. Some limited manual testing was conducted by the publisher, but did nothing to help diagnose the root problem.

The issue happened again the next two weeks. The QA team reported to the chief architect that they couldn’t understand what caused it. Fortunately, the architect had set up API Fortress’ data API to send the data to Elastic, and he had set up a dashboard in Kibana.

When the architect reviewed the data in Kibana, a lightbulb went off. He noticed that the failures always happened on Monday mornings from 6-8 a.m. That’s all he needed to see to know the exact cause. They were using an API gateway, and for performance purposes, they had set up the gateway to cache their listing API. The problem was that they were updating their ISBNs database (which feeds the listing API) every Monday morning at 6 a.m. The gateway only refreshed the cache every two hours. That meant every Monday, for months, the book publisher had exposed hundreds of bad ISBNs for two hours with partners to start their week, and they had no idea. A full analysis revealed that this single uncaught API flaw had caused thousands in lost sales, frustrating partners and causing damage to reputation.

The book publisher had no idea their partner API was failing their partners every Monday morning until they created proper integration tests and monitors. Even in taking those steps, they had trouble diagnosing a remediation to the issue until they had the holistic data to see that the issue happened at the same time every week. While this example was simple, it was indicative of the sort of benefit that can come from integrating with a platform like Kibana for your organization.

This sort of scenario happens far too often. We recommend that you don’t just test on release, but use those functional tests as monitors as well. Then use that rich test data in proper data analysis tools like Elastic + Kibana. Unified testing and monitoring allows you to transform your QA efficiency, effectiveness, and ROI overnight with both legacy and new services.

About Kibana:

Kibana lets you visualize your Elasticsearch data and navigate the Elastic Stack so you can do anything from tracking query load to understanding the way requests flow through your apps.

About API Fortress:

API Fortress is a continuous testing and monitoring platform for APIs that was built from the ground up for shift-left automation and simplified collaboration across teams. By unifying data-driven functional tests with monitoring that can run in any environment, API Fortress detects a much wider range of API issues early in the lifecycle, while significantly accelerating diagnosis with detailed reporting. Now, achieve unlimited quality-at-speed as you integrate API Fortress into any CI/CD platform or DevOps toolchain. Use API Fortress on our hosted cloud at APIFortress.com, or your cloud with a self-managed (on-premises) container.