Environments, vault and overrides combinations

Overview

With the introduction of the “Environments” features, your ability to mix and match your tests settings increases drastically.

Let’s review what our variable containers are, moving from the lowest priority up.

Vault: each project has a vault where variables and snippets are contained. Each vault is global to the project and its variables are injected during test execution.

Globals / Input set: they belong to each test and contain the variables that are needed by the test to perform its duties. Since these variables can be overridden to hit different environments, we generally suggest considering these variables as containers of the default scenario. If the variable has been previously declared in the vault, globals/input sets will win over the Vault.

Overrides: when an overridden variable is declared (using the API, the Scheduler or the Jenkins plugin) its value will be injected in the test when it’s executed. If the variable has already been declared in the vault or the globals/input set, it will be rewritten with the new value.

Environments and presents: they are collections of overrides. You can save an environment with a name and reuse it when running a test.

SET commands: the last item of the chain, the SET commands within a test will introduce a runtime variable. If a variable with that name has already been declared, it will be overwritten.

Suggested Usage

Tests should be as self-contained as possible and should host as much information as possible to perform its operations, with the help of the Vault. In other words, Vault + Globals / Input set should always generate a complete variable scope. Therefore, running a test without any environment selection should at least lead to no syntax or runtime errors.

Environments and overrides should be used to change some of the values, or introduce contextual values, for:

  • hitting a staging server instead of the production one
  • run the test against a special product ID

Double evaluation

All variable containers have a “double evaluation” capability. Meaning that a variable declaration can actually reference another variable.
By doing so you can decide to concentrate the actual data in the variable container that best suits your approach, and then reference it.

In the following example, we are storing the actual domains in the Vault, deciding a default in the Globals, and overriding in the environment:

VAULT:
production_domain: example.com
staging_domain: staging_example.com

GLOBALS:
domain: ${production_domain}

ENVIRONMENTS Name: staging
domain: ${staging_domain}

If run without environment selection or overrides, the test will hit the production domain. If run with the staging environment, the test will hit the staging domain.
The Environments will not know the actual domain, therefore the actual data management will happen within the Vault.

Known Issue: the double evaluation will not work on query params. Workaround: in the test, before performing the call, introduce a SET component to resolve the variable, as in <set var=”apikey” value=”${apikey}”/>

Environments loading

If you’re using environments heavily, you will soon realize that the integration using the APIF API (and any CI/CD plugin) may become unfriendly, as lots of data will need to be copied around.

For this reason, we introduced a special override variable that will allow you to ask API Fortress to load a specific environment when invoking a test using the API.

the apif_env variable, passed as an override, will cause the system to load a specific environment.

Example:

apif_env: staging

Will load the “staging” environment and all its override variables.

Notes

As you get acquinted to the platform, you may be tempted to use all these features at once. While this makes us proud, we want to warn you from the overall complexity you may achieve, expecially when you’re double-referencing variables.
An extensive use of all these techniques altogether may increase the difficulty to debug a problem, when it arises, so use them with caution!