Category: Learn More

When an API Fortress test is executed, a variable stack is built. Every variable in the scope is accessible through API Fortress expressions and components.

The contribution to the variable stack happens in this order:

• Global Vault
• Project Vault
• Test’s global variables
• Test’s input set variables
• Selected environment variables (also referred to as “Overrides”)
• SET components within the test
  
  

Global Vault

Project Vault

Test Global

Test Input Set

Environment / Overrides

In the test creation process, the definition of global and local variables allows you to parametrize the test to allow more flexibility.

The global variables are meant to be the ones that are common for the whole test (i.e. api key, domain etc). For adding a global variable click on the +Add Global Param in the Data Set panel and fill the name and value.

The variable will be present in the test scope, and will be constant during the test execution.

The input set, instead, is a group of input variables representing a scenario (i.e product id). To add in input set, click the + button in the Input Set section and add a name for the set. Then click + Add Param and add the variable name and value.

The unit will be executed once for each input set you have defined. At each iteration, one input set will enter the scope.

If you define a variable both in the global section and the input set, the value used in the test will be the one defined in the input set.

The variables can be defined also in the Vault (for more info see here) and in the Schedule (for more info see here).

The priority order is:

• if the same variable is defined in the Vault and in the composer (no matter if it is a global variable or an input set), the one defined in the composer will be used.
• if the same variable is defined in the composer both as global and input set, the value of the input set will be used
• if the same variable is defined in the Vault (or in the composer) and in the scheduler, the variable defined in the scheduler will be used for the tests.

API tests should rarely need to know facts about themselves and take decisions off those. In fact a test shouldn’t take decisions based on its state, but on what the tested material looks like. Unfortunately, the real world is not always this perfect and forces you to build introspective tests. API Fortress provides you two mini hacks to extend the test logic.

• The ws_failures_count variable will allow you to know how many assertions failed while running the unit against the current input set. You might want to use it to stop the test if a certain number of failures has been reached, or decide whether or not to perform a certain branch.
• The local_ variables. As you know input sets are isolated. This means that any variable I set in the unit while executing input set A will be lost while switching to input set B. This is extremely important because “variable noise” could lead to major uncertainty. However, if I wanted to have a variable to survive input set changes, all I would have to do is use the local_ prefix. As in local_items, local_count.

Most of the times there’s a more conventional way to do things, however when stuff gets nasty, you now know you can use these two tricks to make your life easier.

We present you some tricks that you can use to solve common problems while writing tests.

Unique Emails

If you want to generate a unique random email you can incorporate the current epoch in it. Epochs progress every millisecond so it’s pretty improbable to have a duplicate! The D language extension (see here for more details about this extension) provides date manipulation utilities, so:

<string>${D.nowMillis()}@gmail.com

if ‘string’ is ‘apifortress’ the result will be something like: apifortress1489051395378@gmail.com

String Manipulation

If a data item is a composite string of values and you need to evaluate portions of it, there are numerous ways you can do it. Let’s say we have this string “55,5,84,65,26,7,82” and you want to work with each number. The simpler way is transforming the string into an array. You can do using the ‘split’ function:

<variable_name>.split('<separator>')

In the example above, if the string is included in a variable named ‘randomNumbers’ writing randomNumbers.split(‘,’) you produce an array like [55,5,84,65,26,7,82]. At this point you can work with it as you generally do when you have an array. Another trick you can use to improve your tests is extracting something from a string using the ‘substring’ functionality. If you need to extract the first n characters from a string there is a function that helps you: take(n). Let’s consider the following string: “whatever” saved in a variable named “substring_variable”, if you:

substring_variable.take(4)

“what” will be returned. Another function for a string is drop(n) that removes the first n characters and return the resulting String. Considering the same string and variable used as example for take(n), if you:

 substring_variable.drop(4)

“ever” will return. The last example is extracting a substring using index ranges. If we have the string “wherever” by doing:

wherever[1..4]

we will obtain “here”

Dates Manipulation

Maybe your endpoint requires a date in a specific format as parameter.

${D.format(D.nowMillis(),'MM/dd/YY')}

This expression will generate today date (e.g. 03/09/17)  

An HTTP response is made of a payload (what you are mainly interested in), but also contains contextual information. Using API Fortress you can also deal with the whole response envelope. When you’re making an HTTP request in the composer, you’re providing a variable name. That variable will host the entire response payload. So let’s say “payload” is the name of that variable. When the operation completes, another variable called <variable_name>_response is also created. Therefore various pieces of information such as HTTP header and metrics are contained in the variable “payload_response.” So by referencing the “payload_response.statusCode” expression you can access the status code. For example, if you want to run a branch of code when the status code is 400, here’s how you do it: You can have multiple ‘IF’ conditions for checking all the possible status codes you need to check. Very useful for creating positive and negative tests. Headers are also a big part of the response. Let’s say you want to check that a resource shouldn’t be cached: Furthermore, you can have the performance of the call verified as well. Here is what it looks like in the CODE view:

<assert-less expression="payload_response.metrics.latency" value="350" type="integer"/>

<assert-less expression="payload_response.metrics.fetch" value="350" type="integer"/>

<assert-less expression="payload_response.metrics.overall" value="550" type="integer"/>

Visual Composer view (use the Assert-Less component): Please note that latency is defined as time to first byte. Fetch is the total download time of the payload.

We’ve been asked this question a few times so we wrote a post to make it clear. When you work on a test you are not editing the actual test, but a working copy (a clone) of the test. The working copy can be edited, reset, and saved to continue the work later, but none of these actions alter the state of the live test. When you’re done editing you will need to PUBLISH the test so that the working copy replaces the current live test to become official.

Why

Well, API Fortress is meant to alert people when something goes wrong. If you are editing a test and an incomplete test runs based on the schedule, you will receive a failure notification. A false positive. By using the working copy to make your edits, the scheduler will not execute a test unless it is finalized and published.

To Summarize

• You don’t edit the live test. When you edit one, you’re actually editing the working copy.
• If you want to start over from the live test you can ‘Clear Working Copy’ to create a fresh version of the live test that you can edit.
• When you are done editing click ‘Publish’ so your working copy goes live and the scheduler will use this new test for its executions.
• If you create a new test but never publish it, the test will be marked as ‘incomplete.’

The performance of the API can be mission critical in some cases, and cataloging metrics can be as important as collecting them. The classic approach of creating big tables of HTTP hits with the actual URL being called and its performance is certainly accurate but far from being easy to review because URLs contain IDs and hardly represent what the action was about. API Fortress, as a default, works in this “classic” (dull?) way, but gives you to possibility to change the “footprint” of requests, based on your organization needs. To activate this feature you need to add a “config” component to your I/O operation to reconfigure the footprint. In this example, most of the request remains the same, but the ID value is actually replaced with the static string “[id]”, therefore making the calls with this pattern equivalent when browsed in the project dashboard. And this is just one of the approaches. You’re virtually free to write the footprints that best fit your needs.

Before you start writing your test, you should plan what you want to do. Contrary to most testing frameworks, API Fortress is not endpoint centric, so there’s a number of different paths you can take to make the best of your test.

Single Action Test (active)

You’re willing to write a test that is going to verify how an endpoint action works. We’re also considering this scenario to be active, which means you expect API Fortress to perform a call and test the outcome. If your action takes multiple parameters, we suggest you to create a number of input sets containing combinations of valid input. The unit will be run against each input set. Having multiple inputs is important because it’s the only way you can tell your API is responding uniformly, no matter what you feed it. Then you need to make a request and use those variables. As you can see we’re composing the URL using global variables and we reference the variables in the query parameters. Eventually, you will write your test. This is the simplest API Fortress test, covering the need to verify if an endpoint is up and running and working just fine.

Single Action Test (passive)

A test is passive when API Fortress won’t retrieve the payload on its own but will have it forwarded via the API. In this case, no input sets or I/O operations are necessary. The forwarded payload can be referenced using the “payload” variable. The ideal scenario for this mode is testing live request and responses.

Single Action Test (active, just-in-time parameters)

Also an input set can be overridden using the API. If your process requires an agent to trigger an active test with fresh inputs, create only one input set with valid inputs and have your API call override them. If you run the test by schedule, it test will run against the input you manually inserted. If you run it using the API and override the inputs, your test can run against fresh data all the time. The ideal scenario for such mode is an application where the API is public but the inputs vary a lot over time and cannot be easily determined.

Single Action Test (mixed mode)

There’s a way to have a test to work both in an active way (when triggered by a schedule or manually launched, for example) and in an active way (invoked via API). Here’s how you do it: The GET will be performed only if payload does not have a value (yet). If the test has multiple input sets that you don’t want to run when the payload is being forwarded, invoking the API with the “nosets=true” attribute will prevent it. This is generally a preferred over a plain passive test.

Integration test

In integration tests you chain together multiple calls, from authentication and actions to complete flows. Possible scenarios vary from catalog browsing… … to full booking process for multiple flights