Load Testing

Note: Load testing is still in beta. Procedures, functionality, and display are subject to change.


API Fortress Load Testing provides it’s users with functional load testing. Rather than simply overburdening a server with requests, API Fortress Load Testing directs properly formatted requests to the appropriate endpoints and records failures at an API endpoint level.

Step 1: Access Load Testing Control Panel

To access Load Testing, from the main page, click on the “Tools” button, and then “Load Testing”.



Step 2: Create a Task

This is the main Load Testing screen. From here, you can create and run a new task. You can also run, modify or delete a saved task. To create a new task, click “+New Task


The “Create New Task” screen allows you to set the parameters for the new test.

  • Name – The name that the test will be referred to by.
  • Project – A drop-down menu of all of the projects available to your current company.
  • Test – Allows you to select a test from the selected project.
  • Tags – Allow you to tag the test for later use.
  • Duration – How long, in seconds or minutes, the test will last.
  • Ramp Up – A warm-up period, during which the load test will make requests of the server, but at a much lower rate.
  • Users/Agent – Defines how many users will be simulated per load testing agent.
  • Server – Allows you to select servers from the column on the right to behave as agents for the load test. Click the “Select” button to select a server.

Once you have successfully created a test, it will appear in the column on the left side of the screen.


Step 3: Run the Task

You can run the task by hovering over it and clicking the “Play” button. The test will now run at the top of the queue in the middle of the screen. Once it is complete, it will display a summary of the test performance.


On-Premises: Enabling API Fortress to Read Local Files

Using the read-file command, you can have your test read local files.

Currently there is no GUI functionality to upload the files, however, you can set up your container to connect to a local folder on your host machine.

To do so, you have to update your docker-compose.yml file in the core/ directory.

In the “apifortress” service definition, modify the “volumes” block by adding one entry looking like this:

    - /var/local/data:/data

Where /var/local/data is the path in your host machine where you want to store the files.

On-Premises: Backing Up Your Data

When running an on-premises installation, you will certainly want to run periodic backups of all your data.

In this article, we will provide you the scripts to perform a data dump of API Fortress. You will then need to wire them up in your scheduled operations system, such as cron.

We will assume you have a running API Fortress installation, ability to sudo to root privileges and a general idea of how Docker works.


1. In the host server, create a directory that will host your backup. In this example, it’s /var/local/backups but it can be anything. Make sure the directory has read/write permissions docker can use,

2. Run (change the directory according to your needs):

sudo docker run --rm --net apifortress --link core_apifortress-mongo_1:mongo.apifortress -v /var/local/backups:/backup mongo:3.0.14 bash -c 'mongodump --out /backup --host mongo.apifortress'
3. Run (change the directory according to your needs):
sudo docker run --rm --net apifortress --link core_apifortress-postgres_1:postgres.apifortress -v /var/local/backups:/backup postgres:9.5.5 bash -c 'pg_dump --dbname postgresql://apipulse:jk5112@postgres.apifortress:5432/apipulse > /backup/postgres.sql'
4. Access the /var/local/backups directory. You will find both an “apipulse” directory and a “postgres.sql” file. This is all your backup. You can now zip it and copy it wherever your backup procedures require. At this point we suggest you to clear the directory used for backup to have it empty for the next backup iteration.

Backup restore

1. in the core/ directory, stop all services by issuing:

sudo docker-compose stop

2. Remove all data files from your persistent volume on the host machine. EXTREME CAUTION: this will erase all your current data. Make sure that the backup you are going to restore is available. If unsure, just MOVE the current data to another location,

3. Activate MongoDB and PostgreSQL by issuing:

sudo docker-compose up -d apifortress-postgres
sudo docker-compose up -d apifortress-mongo

4. We will assume your backup is located in /var/local/backups. Run the following commands:

sudo docker run --rm --net apifortress --link core_apifortress-mongo_1:mongo.apifortress -v /var/local/backups:/backup mongo:3.0.14 bash -c 'mongorestore /backup --host mongo.apifortress'
sudo docker run --rm --net apifortress --link core_apifortress-postgres_1:postgres.apifortress -v /var/local/backups:/backup postgres:9.5.5 bash -c 'psql -h postgres.apifortress --dbname postgresql://apipulse:jk5112@postgres.apifortress:5432/apipulse  < /backup/postgres.sql'

5. Verify that files are now present in the persistent volume location of your host machine,

6. You can now start the platform by running the ./start_all.sh script.

Known Issues / Scheduled Updates


  • [ETA: TBD] Nested variable referencing (Ie. v1=${v2}, v2=value) for variables declared in input sets, vault and overrides, does not work in:
    Query parameters
    Expression field in assertion
    Workaround:overwrite the variable in the test before using it in one of these fields, to resolve it ahead of time. Example:

    <set var="v1" value="${v2}"/>
  • [ETA: TBD] Assert-Matches for US Zip codes does not include district codes


  • [ETA: End of August] In the test list, provide the user a way to override variables when running a test manually

Composer/Embedded Console

  • [ETA: Next release] Import of requests to console does not resolve nested variables (Ie. v1=${v2} v2=value)
  • [ETA: Next release] Import of requests to console does not resolve vault variables
  • [ETA: End of August] provide the user a way to override variables when running a test in development


  • [ETA: End of August] Self signed certificates won’t get accepted unless a certificate is installed in the downloader (no bypass option)


  • [ETA: Next release] Syntax or runtime errors are rarely reported with details
  • [ETA: Next release] HTTP requests should contain all the details of the request itself
  • [ETA: End of Quarter] On complex tests, reports can become humongous and very hard to load


  • [ETA: End of Quarter] heavy API failures cause email flooding. Emails should be stateful
  • [ETA: End of Quarter] Small memory leak causes connectors to slowly grow in memory demand
  • [ETA: Next release] On-premises deployments should be able to connect to message queue and create custom notification systems


  • [ETA: End of Quarter] On-premises: There’s no way to tell when a license is supposed to expire

On-Premises: Deployment Using Docker


This manual will describe a normal deployment procedure for API Fortress on-premises, using a Docker container. It is important to remember that the goal of this guide is to be as thorough as possible. It may seem long but the process is fairly straightforward.

Also, don’t fret as we can provide as much help and guidance as you need. We are just a video conference away!

You have been provided with apifortress_starter.zip, which contains the following files:

1. Copy the Provided Script Files

Copy the provided core and downloader directories to the server and then type cd core/.

2. Configure the Core Services

Before anything else, let’s configure each service and prepare the environment.
Most configuration keys are stored within the core/docker-compose.yml file.

The only special configuration will be the storage on the host machine.
Create a directory that will host PostgreSQL data in the host machine, and edit configuration file with that location. Replace the “/data/postgres” with your details.

   - /data/postgres:/var/lib/postgresql/data

As with PostgreSQL, you are required to provide a storage location and edit the volumes key accordingly. Replace the “/data/mongodb” with your location.

  - /data/mongodb:/data/db

API Fortress
There are a lot of configuration keys here. None of them should be left empty (a fake value is fine if you’re not using a certain feature). See the API Fortress Configuration Guide below for an explanation of each key.

The essential keys for bootstrap (with dummy values) are:

Admin User Creation
adminEmail: patrick@company.com
adminFullName: Patrick Poulin

Company Creation
defaultCompanyName: Your Company

Base URL that will respond to HTTP requests
grailsServerURL: http://yourcompany.com/app

API Fortress Mailer
Refer below.

API Fortress Downloader
To be configured after the dashboard bootstrap. Refer below.

3. Install Docker

Install Docker on a supported Linux distribution following the official instructions:
The API Fortress stack runs successfully on Docker 1.12.

4. Install Docker Compose

Docker Compose is a utility that simplifies the deployment and management of complete stacks. Follow the official instructions for installation:

5. Provide API Fortress your DockerHub username

For API Fortress to grant you access to the API Fortress registries, your DockerHub username is required.  If you don’t have a DockerHub account, create one at https://hub.docker.com/

6. Login

Type sudo docker login and input your DockerHub credentials.

7. Create the API Fortress network

The default API Fortress subnet is Make sure the default subnet is not in use. If it is then edit it in the create_network.sh script. Issue sudo ./create_network.sh  to create a virtual subnet for API Fortress.

8. Launch the Services

Before you launch any service, we strongly recommend you to run a: docker-compose pull  from the “core” and “downloader” directories to download all packages and preemptively verify any possible connection issue.

To launch all core services, just run the start_all.sh script. It will take some time, but it will ensure every dependency is up and running before launching API Fortress.

9. Verify the Deployment

At the end of the process, the API Fortress dashboard should be up and running in the host server on port 80. You can also check for errors in the logs by issuing the: sudo docker-compose logs command.

The admin user login details are as follows:

  • username: the email address provided in the docker-compose configuration, in the adminEmail field;
  • password: ‘foobar’, change it as soon as you log in.

10. Configure and Run the Downloader

The API Fortress downloader is the agent that retrieves the resources to be tested. Downloaders can be installed in various locations, so factors such as latency and download time can be measured by remote consumers.

In this configuration path, we are deploying a downloader in the same server as API Fortress, and it will serve as the default downloader.

1. Edit the downloader/docker-compose.yml file and take note of the value of the ipv4_address configuration key.

2. Login to API Fortress with the admin user, access the API Fortress admin panel by clicking the “user” icon in the top right, then click Admin Panel.


3. Choose “Downloaders” from the list of actions and click on the “Add Downloader” button.

4. Fill the fields:
Name: Write a recognizable name.
Location: A representation of where the downloader is. ie. Chicago
Latitude / Longitude: The geographical position of the downloader.
Last Resort: Check this to make it the default downloader used.
URL: The address of the downloader, followed by port (default 8819) and path /api. In our example, the ipv4_address and our downloader address would result in
API Key, API Secret: Write these two values down for use later.

5. Edit the  downloader/docker-compose.yml file and enter the API Key and API Secret.

6. Go to the downloader/ directory and issue the sudo docker-compose up -d command.

API Fortress Configuration Guide

A description of each configuration field you may need to alter.

API Fortress Dashboard


 – adminEmail: The admin user email address, also used as login.
 – adminFullName: The admin’s full name.
 – defaultCompanyName: The company name.


– grailsServerURL: the url the server will respond to
 – dbHost: MongoDB host
 – psqlhost: PostgreSQL host
 – rabbitHost: RabbitMQ host

Note: in case you’re considering using an external PostgreSQL provider, the psqlUsername and psqlPassword parameters are also available. The database name is fixed and it’s apipulse.


– apifortressMailUseSES: set to ‘true’ if you will use Amazon SES to send emails. When set to ‘false’, SMTP is used instead.
 – apifortressMailFrom: the email address that will be used to dispatch administrative emails.
 – apifortressMailSmtpHost: SMTP host to dispatch administrative emails.
 – apifortressMailSmtpUsername: SMTP username.
 – apifortressMailSmtpPassword: SMTP password.
 – apifortressMailSmtpPort: SMTP port.
 – amazonkey: Amazon key, if you’re using Amazon SES to send emails.
 – amazonsecret: Amazon secret, if you’re using Amazon SES to send emails.
 – apiaryClientId: client ID if you’re using Apiary services.
 – apiarySecret: secret, if you’re using Apiary services.
 – license: the license string.

API Fortress Mailer

 – twilioSid: SID, if you’re sending SMSes via Twilio.
 – twilioToken: token, if you’re sending SMSes via Twilio.
 – smsFrom: the phone number of the SMS sender, if you’re sending SMSes via Twilio.
 – mailFrom: the email address that will be sending notification emails.
 – mailUseSES: ‘true’ if you’re sending emails via Amazon SES. False if you’re using SMTP.
 – amazonKey: the Amazon key, if you’re sending emails via Amazon SES.
 – amazonSecret: the Amazon secret, if you’re sending emails via Amazon SES.
 – mailSmtpHost: the SMTP host.
 – mailSmtpPort: the SMTP port.
 – mailSmtpUsername: the SMTP username.
 – mailSmtpPassword: the SMTP password.
 – apifortressServerURL: the url the server will respond to.

API Fortress Downloader

– apikey: the API key, as shown in the admin panel.
 – secret: the API secret, as shown in the admin panel.
 – port: the HTTP port the server will be listening to, in HTTP mode.
 – rabbitHost: the RabbitMQ host, when running in active mode.
 – rabbitPort: the RabbitMQ port, when running in active mode.
 – rabbitSsl: ‘true’ if RabbitMQ will need to communicate over SSL when running in active mode.
 – rabbitUsername: the RabbitMQ username when running in active mode.
 – rabbitPassword: the RabbitMQ password when running in active mode.
 – use_rabbit: ‘true’ to run in active mode.
 – use_http: ‘true’ to use the internal HTTP server (passive mode).
 – use_ssl: ‘true’ if the internal HTTP server has to run over SSL.

The network configuration is also important as the IP address may be used for internal communication.

networks.apifortress.ipv4_address: the reserved IP address in the API Fortress subnet.

Appendix: Importing help tools and connectors

The API Fortress database comes free from data, but the provided package gives you the option to import the help tools and the connectors. These operations are meant to be run once the API Fortress stack is fully functional.

Import Help From the /data directory, run the import_help.sh script.
Import Connectors From the /data directory, run the import_connectors.sh script.

Appendix: Tweaking Tomcat Configuration

If you need to tweak the Tomcat configuration, you will need to mount the Tomcat conf/ directory in your system.
1. Change the configuration files you need to edit in the core/tomcat_conf/conf directory
2. Mount the directory by uncommenting the following lines in the core/docker-compose.yml file:

# volumes:
# - ./tomcat_conf/conf:/usr/local/tomcat/conf

Dashboard over SSL

To have Tomcat running over SSL:
1. Copy your JKS keystore containing your certificate in the core/tomcat_conf/conf directory
Edit the core/tomcat_conf/conf/server.xml file and uncomment the block:

<Connector port="8443" protocol="org.apache.coyote.http11.Http11Protocol" maxThreads="150" SSLEnabled="true" scheme="https" secure="true" clientAuth="false" sslProtocol="TLS" />

3. Edit the block by adding the following attributes:


4. Mount the directory by uncommenting the following lines in the core/docker-compose.yml file:

# volumes:
# - ./tomcat_conf/conf:/usr/local/tomcat/conf

5. In the core/docker-compose.yml file, change the port declaration to:

- 443:8443/tcp

On-Premises: System Requirements

The one server setup for API Fortress on-premise is a quick way to get things started in a protected environment. While not ideal for availability or performance, works exactly as expected and provides all the features of the cloud version.

Minimum Hardware Requirements
CPU: Intel based high frequency quad core processor
Memory: 16 GB RAM
HDD: 250 GB
Memory: the memory impacts significantly on the speed of queries on big data sets. 32 GB is a recommended setup
HDD: All API Fortress reports and metrics are stored. 10 million reports + 30 million metrics can require up to 250GB of disk space

Software Requirements
OS: a recent Linux distribution

Classic Deployment
Java: Oracle JDK 1.8 series
Tomcat: 7 series
PostgreSQL: 9.5 series
MongoDB: 3.2 series
RabbitMQ: 3.5 series

Docker Deployment
Docker: 1.12

PostgreSQL: relational database for structured data
MongoDB: document database for reports and metrics
RabbitMQ: message queue
Tomcat: dashboard and engine application
AFScheduler: the API Fortress scheduler
AFMailer: the API Fortress mailer
AFConnector: dynamic data dispatcher for notifications
AFDownloadAgent: the downloader agent (actually performing HTTP calls)

We assume this deployment will be able to access the services to be tested.

Further Connections
HTTP(80) and/or HTTPS(443) inbound traffic enabled for every location that will need access to the dashboards. Ports and services may vary based on system requirements.

For the Docker deployment to succeed and to ease further updates, the server has to be able to communicate with https://hub.docker.com

Local Downloader Setup

1. What it Does

The API Fortress Remote Download Agent sits inside of your infrastructure to allow the platform to test systems that are not exposed externally.

It will listen to an HTTPS port for jobs requested by an API Fortress engine. The agent will perform an HTTP(S) request to an endpoint as described in the job, and once completed will serialize the data back to the engine, adding contextual information such as the metrics.

No data is retained in the agent memory after job completion. The agent will use the DNS settings provided by the machine it’s installed on.

2. General Requirements

    • Software: Java JRE 1.8 or greater
    • Network: the agent should be reachable by the designated API Fortress engine(s)
    • Network: the default inbound port is 8819

3. Cloud Requirements

If the agent is meant to be contacted by the API Fortress cloud, the connection will happen from the following sources:

  • dashboard-aws01-1.apifortress.com –
  • dashboard-aws01-2.apifortress.com –

4. Installation

Unzip the provided package in a location of your choice.

The package should contain the following files:

  • RemoteDownloadAgent-complete.jar : the agent
  • config.properties : the configuration file
  • LICENSE.md :  the license file
  • VERSION.md : the version file
  • bin/start.sh :  the start script for *NIX systems
  • bin/shutdown.sh : the stop script for *NIX systems
  • bin/java_opts.sh : java related settings for *NIX systems
  • bin/start.bat : the start script for Windows
  • bin/java_opts.bat : java related settings for Windows

5. Configuration

We will preconfigure the config.properties based on your needs.

Here the main items:

serviceName: a human readable identifier of the downloader.
apikey: unique to every downloader. Used by the API Fortress engine to authenticate.
secret: used by the API Fortress engine to authenticate.
use_http (true): enables the HTTP server.
use_ssl (true): enables the SSL encryption for the HTTP server.

6. Running the Agent – *NIX

Run the bin/start.sh script to start the agent which will run in the background.

Run the bin/shutdown.sh script to stop the service.

7. Running the Agent – Windows

Run the bin/start.bat script to start the agent which will run in foreground.

8. Tuning

Even though the agent does not require relevant system resources, it is a good practice to allow the agent to use as much memory as available. By editing the java_opts script (.bat or .sh, depending on the platform) you can tweak the heap memory settings.

Note: Remember that heap memory is not the only type of memory Java uses. Raising the heap to the system limits is not advised, and it will not work. If you are not familiar with these settings, the default will likely meet your requirements.

API Tests vs. Schema Validation

One of the most common questions we receive about API testing is:

What’s the deal? Schema validation was not enough?

Some of these inquiries are moved by real curiosity, others are just polemic, but either cases, the question makes a good point if you’ve never actually tried our platform.

Schema validation has two major flaws:

  1. You need to write a schema
  2. It statically validates your API syntax and grammar, not your data and certainly not your darkest secrets.

For what concerns ( 1 ) deal with it.
( 2 ) is a little more intriguing. Let’s work by example.

  "group": "food",
  "items": [ 10,15,17,19 ]
  "install": false,
  "installDay": null
  "delivery": true
  "request_date": 1456249628

Of course a schema can determine whether these items are the right types, but no schema can tell that:

  1. Food items should have ids lower than 100
  2. Food items should not be installed (weird, right?)
  3. Food items have no installation day, but they’re deliverable
  4. The response shouldn’t be older than 1 day (caching?)

Moreover, API testing allows you to compare things with the request data. You don’t want to get bathrobes listed while you were looking for your birthday cake. Or, more technically, you don’t want the data within a credit card transaction to be cached.


The On-Premises Engine

What Is It

API Fortress can also come in an on-premises version. On-premises means that an API Fortress engine will live inside your infrastructure and will interact with your APIs from the inside, as opposed to the cloud solution where everything resides on the API Fortress infrastructure at apifortress.com.


There are multiple reasons for having an on-prem engine, and these are some of the most common:

  • Security Restrictions
  • Access to Private / Sensitive Information
  • Large Companies That Want an Unlimited Amount of Deployments, Tests, and Users

But there’s another reason that makes it suitable for a number of users: customization


API Fortress is extremely modular and most functionalities can be replaced with different code, behaving in a different way. Some use cases are:

  • Storing the results of the tests in a dedicate archive, such as DynamoDB, a private MongoDB instance, or an object storage.
  • Customizing the chain of alerts with internal tools.
  • Storing the code of the tests in a location that is not the API Fortress cloud.
  • Adding the ability to ingest and analyze exotic data types.

All this is done with a few lines of Java. The engine itself can work as an SDK to build what you need. Or you can ask our team, we are glad to help.


A simple Docker deployment. We ask a handful of questions, setup a configuration file for you, and then you deploy with Docker. The system requirements here.


The engine operates exactly the same way API Fortress does in the cloud.