APF – Local Engine (APIF-Local)

apif-local-Local is the API Fortress local engine.

The objective of this program is to allow developers to run API Fortress tests on their computers, and to validate the tests being developed locally.

Requirements

  • A modern Linux/Mac/Windows computer
  • Java 8

Installation

  1. Make sure Java 8 is installed and is present in the command line path
  2. Unzip the file provided by our team in a location that is convenient to you

Configuration

The only essential key in the configuration is the license. Everything else can be provided on the command line. We do, however, suggest you configure the tool properly once you get accustomed to it and start using it regularly.

Example:

license: 'abcabcabc'
hooks:
- key: examples
  url: https://mastiff.apifortress.com/app/api/rest/v3/abcabc-123-123-123
  credentials:
    username: info@example.com
    password: password1

- key: local
  url: http://mydeployment.com:8080/app/api/rest/v3/abcabcabc-123-123-123
 credentials:
    username: john@doe.com
    password: password1

In this example, we configured two different hooks, related to two different API Fortress instances. The value of the “key” can be used to reference a profile without the need to explicitly introduce the hook URL and credentials every time in the command line.

The configuration file can be placed in multiple locations, based on your preferences.

The priority check is the following:

  1. A path provided in the command line
  2. The current working directory
  3. The directory where apif-local.jar resides
  4. A .apifortress sub-directory in the user’s home directory

apif-local – Download

java -jar apif-local.jar download [hook] <options>

This mode allows you to download full projects, or, selectively, tests, vault and environments, from an API Fortress platform instance. See apif-local.jar help for details about the possible switches.

Examples:

java -jar apif-local.jar download examples -l /tmp/stuff

Will download the full project represented by they key “examples” in the directory /tmp/stuff.

java -jar apif-local.jar download examples -l /tmp/stuff -v

Will download just the project’s vault.

apif-local – Run

java -jar apif-local.jar run [path] <options>

Runs an API Fortress test, located in the local file system. See apif-local.jar help for details about the possible switches.

The expected file system should look like the following:

\any_directory
  \api_fortress_tests
     \simple_test_1
         | unit.xml
         | input.xml
     \advanced_test_2
         | unit.xml
         | input.xml

Every test is represented by a directory, containing an input.xml and a unit.xml file. This exact format is also followed by apif-local – Download.

Examples:

java -jar apif-local.jar run any_directory/api_fortress_tests/simple_test_1

Will run the test simple_test_1.

java -jar apif-local.jar run any_directory/api_fortress_tests -r

Will run all the tests contained in the api_fortress_tests directory.

Load Testing

Note: Load testing is still in beta. Procedures, functionality, and display are subject to change.

Introduction

API Fortress Load Testing provides it’s users with functional load testing. Rather than simply overburdening a server with requests, API Fortress Load Testing directs properly formatted requests to the appropriate endpoints and records failures at an API endpoint level.

Step 1: Access Load Testing Control Panel

To access Load Testing, from the main page, click on the “Tools” button, and then “Load Testing”.

toolstolt

 

Step 2: Create a Task

This is the main Load Testing screen. From here, you can create and run a new task. You can also run, modify or delete a saved task. To create a new task, click “+New Task

createTest

The “Create New Task” screen allows you to set the parameters for the new test.

  • Name – The name that the test will be referred to by.
  • Project – A drop-down menu of all of the projects available to your current company.
  • Test – Allows you to select a test from the selected project.
  • Tags – Allow you to tag the test for later use.
  • Duration – How long, in seconds or minutes, the test will last.
  • Ramp Up – A warm-up period, during which the load test will make requests of the server, but at a much lower rate.
  • Users/Agent – Defines how many users will be simulated per load testing agent.
  • Server – Allows you to select servers from the column on the right to behave as agents for the load test. Click the “Select” button to select a server.

Once you have successfully created a test, it will appear in the column on the left side of the screen.

taskList

Step 3: Run the Task

You can run the task by hovering over it and clicking the “Play” button. The test will now run at the top of the queue in the middle of the screen. Once it is complete, it will display a summary of the test performance.

runTest

On-Premises: Enabling API Fortress to Read Local Files

Using the read-file command, you can have your test read local files.

Currently there is no GUI functionality to upload the files, however, you can set up your container to connect to a local folder on your host machine.

To do so, you have to update your docker-compose.yml file in the core/ directory.

In the “apifortress” service definition, modify the “volumes” block by adding one entry looking like this:

volumes: 
    - /var/local/data:/data

Where /var/local/data is the path in your host machine where you want to store the files.

On-Premises: Backing Up Your Data

When running an on-premises installation, you will certainly want to run periodic backups of all your data.

In this article, we will provide you the scripts to perform a data dump of API Fortress. You will then need to wire them up in your scheduled operations system, such as cron.

We will assume you have a running API Fortress installation, ability to sudo to root privileges and a general idea of how Docker works.

Backup

1. In the host server, create a directory that will host your backup. In this example, it’s /var/local/backups but it can be anything. Make sure the directory has read/write permissions docker can use,

2. Run (change the directory according to your needs):

sudo docker run --rm --net apifortress --link core_apifortress-mongo_1:mongo.apifortress -v /var/local/backups:/backup mongo:3.0.14 bash -c 'mongodump --out /backup --host mongo.apifortress'
3. Run (change the directory according to your needs):
sudo docker run --rm --net apifortress --link core_apifortress-postgres_1:postgres.apifortress -v /var/local/backups:/backup postgres:9.5.5 bash -c 'pg_dump --dbname postgresql://apipulse:jk5112@postgres.apifortress:5432/apipulse > /backup/postgres.sql'
4. Access the /var/local/backups directory. You will find both an “apipulse” directory and a “postgres.sql” file. This is all your backup. You can now zip it and copy it wherever your backup procedures require. At this point we suggest you to clear the directory used for backup to have it empty for the next backup iteration.

Backup restore

1. in the core/ directory, stop all services by issuing:

sudo docker-compose stop

2. Remove all data files from your persistent volume on the host machine. EXTREME CAUTION: this will erase all your current data. Make sure that the backup you are going to restore is available. If unsure, just MOVE the current data to another location,

3. Activate MongoDB and PostgreSQL by issuing:

sudo docker-compose up -d apifortress-postgres
sudo docker-compose up -d apifortress-mongo

4. We will assume your backup is located in /var/local/backups. Run the following commands:

sudo docker run --rm --net apifortress --link core_apifortress-mongo_1:mongo.apifortress -v /var/local/backups:/backup mongo:3.0.14 bash -c 'mongorestore /backup --host mongo.apifortress'
sudo docker run --rm --net apifortress --link core_apifortress-postgres_1:postgres.apifortress -v /var/local/backups:/backup postgres:9.5.5 bash -c 'psql -h postgres.apifortress --dbname postgresql://apipulse:jk5112@postgres.apifortress:5432/apipulse  < /backup/postgres.sql'

5. Verify that files are now present in the persistent volume location of your host machine,

6. You can now start the platform by running the ./start_all.sh script.

Known Issues / Scheduled Updates

Engine

  • [ETA: TBD] Nested variable referencing (Ie. v1=${v2}, v2=value) for variables declared in input sets, vault and overrides, does not work in:
    Query parameters
    Expression field in assertion
    POST/PUT/PATCH/DELETE bodies
    Workaround:overwrite the variable in the test before using it in one of these fields, to resolve it ahead of time. Example:

    <set var="v1" value="${v2}"/>
  • [ETA: TBD] Assert-Matches for US Zip codes does not include district codes

Dashboard

  • [ETA: End of August] In the test list, provide the user a way to override variables when running a test manually

Composer/Embedded Console

  • [ETA: Next release] Import of requests to console does not resolve nested variables (Ie. v1=${v2} v2=value)
  • [ETA: Next release] Import of requests to console does not resolve vault variables
  • [ETA: End of August] provide the user a way to override variables when running a test in development

Downloader

  • [ETA: End of August] Self signed certificates won’t get accepted unless a certificate is installed in the downloader (no bypass option)

Reports

  • [ETA: Next release] Syntax or runtime errors are rarely reported with details
  • [ETA: Next release] HTTP requests should contain all the details of the request itself
  • [ETA: End of Quarter] On complex tests, reports can become humongous and very hard to load

Notifications

  • [ETA: End of Quarter] heavy API failures cause email flooding. Emails should be stateful
  • [ETA: End of Quarter] Small memory leak causes connectors to slowly grow in memory demand
  • [ETA: Next release] On-premises deployments should be able to connect to message queue and create custom notification systems

Misc

  • [ETA: End of Quarter] On-premises: There’s no way to tell when a license is supposed to expire

On-Premises: Deployment Using Docker

Introduction

This manual will describe a normal deployment procedure for API Fortress on-premises, using a Docker container. It is important to remember that the goal of this guide is to be as thorough as possible. It may seem long but the process is fairly straightforward.

Also, don’t fret as we can provide as much help and guidance as you need. We are just a video conference away!

You have been provided with apifortress_starter.zip, which contains the following files:
/create_network.sh
/
core/docker-compose.yml
/
core/tomcat_conf/conf/
/core/start_all.sh
/
downloader/docker-compose.yml
/data/connectors.tgz
/data/help.tgz
/data/import_help.sh
/data/import_connectors.sh

1. Copy the Provided Script Files

Copy the provided core and downloader directories to the server and then type cd core/.

2. Configure the Core Services

Before anything else, let’s configure each service and prepare the environment.
Most configuration keys are stored within the core/docker-compose.yml file.

PostgreSQL
The only special configuration will be the storage on the host machine.
Create a directory that will host PostgreSQL data in the host machine, and edit configuration file with that location. Replace the “/data/postgres” with your details.

    volumes:
   - /data/postgres:/var/lib/postgresql/data

MongoDB
As with PostgreSQL, you are required to provide a storage location and edit the volumes key accordingly. Replace the “/data/mongodb” with your location.

  volumes:
  - /data/mongodb:/data/db

API Fortress
There are a lot of configuration keys here. None of them should be left empty (a fake value is fine if you’re not using a certain feature). See the API Fortress Configuration Guide below for an explanation of each key.

The essential keys for bootstrap (with dummy values) are:

Admin User Creation
adminEmail: patrick@company.com
adminFullName: Patrick Poulin

Company Creation
defaultCompanyName: Your Company

Base URL that will respond to HTTP requests
grailsServerURL: http://yourcompany.com/app

API Fortress Mailer
Refer below.

API Fortress Downloader
To be configured after the dashboard bootstrap. Refer below.

3. Install Docker

Install Docker on a supported Linux distribution following the official instructions:
https://docs.docker.com/engine/installation/
The API Fortress stack runs successfully on Docker 1.12.

4. Install Docker Compose

Docker Compose is a utility that simplifies the deployment and management of complete stacks. Follow the official instructions for installation:
https://docs.docker.com/compose/install/

5. Provide API Fortress your DockerHub username

For API Fortress to grant you access to the API Fortress registries, your DockerHub username is required.  If you don’t have a DockerHub account, create one at https://hub.docker.com/

6. Login

Type sudo docker login and input your DockerHub credentials.

7. Create the API Fortress network

The default API Fortress subnet is 172.18.0.0/16. Make sure the default subnet is not in use. If it is then edit it in the create_network.sh script. Issue sudo ./create_network.sh  to create a virtual subnet for API Fortress.

8. Launch the Services

Before you launch any service, we strongly recommend you to run a: docker-compose pull  from the “core” and “downloader” directories to download all packages and preemptively verify any possible connection issue.

To launch all core services, just run the start_all.sh script. It will take some time, but it will ensure every dependency is up and running before launching API Fortress.

9. Verify the Deployment

At the end of the process, the API Fortress dashboard should be up and running in the host server on port 80. You can also check for errors in the logs by issuing the: sudo docker-compose logs command.

The admin user login details are as follows:

  • username: the email address provided in the docker-compose configuration, in the adminEmail field;
  • password: ‘foobar’, change it as soon as you log in.

10. Configure and Run the Downloader

The API Fortress downloader is the agent that retrieves the resources to be tested. Downloaders can be installed in various locations, so factors such as latency and download time can be measured by remote consumers.

In this configuration path, we are deploying a downloader in the same server as API Fortress, and it will serve as the default downloader.

1. Edit the downloader/docker-compose.yml file and take note of the value of the ipv4_address configuration key.

2. Login to API Fortress with the admin user, access the API Fortress admin panel by clicking the “user” icon in the top right, then click Admin Panel.

login

3. Choose “Downloaders” from the list of actions and click on the “Add Downloader” button.

4. Fill the fields:
Name: Write a recognizable name.
Location: A representation of where the downloader is. ie. Chicago
Latitude / Longitude: The geographical position of the downloader.
Last Resort: Check this to make it the default downloader used.
URL: The address of the downloader, followed by port (default 8819) and path /api. In our example, the ipv4_address and our downloader address would result in https://172.18.1.1:8819/api
API Key, API Secret: Write these two values down for use later.

5. Edit the  downloader/docker-compose.yml file and enter the API Key and API Secret.

6. Go to the downloader/ directory and issue the sudo docker-compose up -d command.

API Fortress Configuration Guide

A description of each configuration field you may need to alter.

API Fortress Dashboard

Bootstrap

 – adminEmail: The admin user email address, also used as login.
 – adminFullName: The admin’s full name.
 – defaultCompanyName: The company name.

System

– grailsServerURL: the url the server will respond to
 – dbHost: MongoDB host
 – psqlhost: PostgreSQL host
 – rabbitHost: RabbitMQ host

Note: in case you’re considering using an external PostgreSQL provider, the psqlUsername and psqlPassword parameters are also available. The database name is fixed and it’s apipulse.

Email

– apifortressMailUseSES: set to ‘true’ if you will use Amazon SES to send emails. When set to ‘false’, SMTP is used instead.
 – apifortressMailFrom: the email address that will be used to dispatch administrative emails.
 – apifortressMailSmtpHost: SMTP host to dispatch administrative emails.
 – apifortressMailSmtpUsername: SMTP username.
 – apifortressMailSmtpPassword: SMTP password.
 – apifortressMailSmtpPort: SMTP port.
 – amazonkey: Amazon key, if you’re using Amazon SES to send emails.
 – amazonsecret: Amazon secret, if you’re using Amazon SES to send emails.
 – apiaryClientId: client ID if you’re using Apiary services.
 – apiarySecret: secret, if you’re using Apiary services.
 – license: the license string.

API Fortress Mailer

 – twilioSid: SID, if you’re sending SMSes via Twilio.
 – twilioToken: token, if you’re sending SMSes via Twilio.
 – smsFrom: the phone number of the SMS sender, if you’re sending SMSes via Twilio.
 – mailFrom: the email address that will be sending notification emails.
 – mailUseSES: ‘true’ if you’re sending emails via Amazon SES. False if you’re using SMTP.
 – amazonKey: the Amazon key, if you’re sending emails via Amazon SES.
 – amazonSecret: the Amazon secret, if you’re sending emails via Amazon SES.
 – mailSmtpHost: the SMTP host.
 – mailSmtpPort: the SMTP port.
 – mailSmtpUsername: the SMTP username.
 – mailSmtpPassword: the SMTP password.
 – apifortressServerURL: the url the server will respond to.

API Fortress Downloader

– apikey: the API key, as shown in the admin panel.
 – secret: the API secret, as shown in the admin panel.
 – port: the HTTP port the server will be listening to, in HTTP mode.
 – rabbitHost: the RabbitMQ host, when running in active mode.
 – rabbitPort: the RabbitMQ port, when running in active mode.
 – rabbitSsl: ‘true’ if RabbitMQ will need to communicate over SSL when running in active mode.
 – rabbitUsername: the RabbitMQ username when running in active mode.
 – rabbitPassword: the RabbitMQ password when running in active mode.
 – use_rabbit: ‘true’ to run in active mode.
 – use_http: ‘true’ to use the internal HTTP server (passive mode).
 – use_ssl: ‘true’ if the internal HTTP server has to run over SSL.

The network configuration is also important as the IP address may be used for internal communication.

networks.apifortress.ipv4_address: the reserved IP address in the API Fortress subnet.

Appendix: Importing help tools and connectors

The API Fortress database comes free from data, but the provided package gives you the option to import the help tools and the connectors. These operations are meant to be run once the API Fortress stack is fully functional.

Import Help From the /data directory, run the import_help.sh script.
Import Connectors From the /data directory, run the import_connectors.sh script.

Appendix: Tweaking Tomcat Configuration

If you need to tweak the Tomcat configuration, you will need to mount the Tomcat conf/ directory in your system.
1. Change the configuration files you need to edit in the core/tomcat_conf/conf directory
2. Mount the directory by uncommenting the following lines in the core/docker-compose.yml file:

# volumes:
# - ./tomcat_conf/conf:/usr/local/tomcat/conf

Dashboard over SSL

To have Tomcat running over SSL:
1. Copy your JKS keystore containing your certificate in the core/tomcat_conf/conf directory
2. 
Edit the core/tomcat_conf/conf/server.xml file and uncomment the block:

<Connector port="8443" protocol="org.apache.coyote.http11.Http11Protocol" maxThreads="150" SSLEnabled="true" scheme="https" secure="true" clientAuth="false" sslProtocol="TLS" />

3. Edit the block by adding the following attributes:

keystoreFile="/usr/local/tomcat/conf/keystore.jks"
keystorePass="thePasswordHere"

4. Mount the directory by uncommenting the following lines in the core/docker-compose.yml file:

# volumes:
# - ./tomcat_conf/conf:/usr/local/tomcat/conf

5. In the core/docker-compose.yml file, change the port declaration to:
ports:

- 443:8443/tcp

On-Premises: System Requirements

The one server setup for API Fortress on-premise is a quick way to get things started in a protected environment. While not ideal for availability or performance, works exactly as expected and provides all the features of the cloud version.

Minimum Hardware Requirements
CPU: Intel based high frequency quad core processor
Memory: 16 GB RAM
HDD: 250 GB
Memory: the memory impacts significantly on the speed of queries on big data sets. 32 GB is a recommended setup
HDD: All API Fortress reports and metrics are stored. 10 million reports + 30 million metrics can require up to 250GB of disk space

Software Requirements
OS: a recent Linux distribution

Classic Deployment
Java: Oracle JDK 1.8 series
Tomcat: 7 series
PostgreSQL: 9.5 series
MongoDB: 3.2 series
RabbitMQ: 3.5 series

Docker Deployment
Docker: 1.12

Processes
PostgreSQL: relational database for structured data
MongoDB: document database for reports and metrics
RabbitMQ: message queue
Tomcat: dashboard and engine application
AFScheduler: the API Fortress scheduler
AFMailer: the API Fortress mailer
AFConnector: dynamic data dispatcher for notifications
AFDownloadAgent: the downloader agent (actually performing HTTP calls)

Networking
We assume this deployment will be able to access the services to be tested.

Further Connections
HTTP(80) and/or HTTPS(443) inbound traffic enabled for every location that will need access to the dashboards. Ports and services may vary based on system requirements.

Docker
For the Docker deployment to succeed and to ease further updates, the server has to be able to communicate with https://hub.docker.com

Local Downloader Setup

1. What it Does

The API Fortress Remote Download Agent sits inside of your infrastructure to allow the platform to test systems that are not exposed externally.

It will listen to an HTTPS port for jobs requested by an API Fortress engine. The agent will perform an HTTP(S) request to an endpoint as described in the job, and once completed will serialize the data back to the engine, adding contextual information such as the metrics.

No data is retained in the agent memory after job completion. The agent will use the DNS settings provided by the machine it’s installed on.

2. General Requirements

    • Software: Java JRE 1.8 or greater
    • Network: the agent should be reachable by the designated API Fortress engine(s)
    • Network: the default inbound port is 8819

3. Cloud Requirements

If the agent is meant to be contacted by the API Fortress cloud, the connection will happen from the following sources:

  • dashboard-aws01-1.apifortress.com – 52.0.5.117
  • dashboard-aws01-2.apifortress.com – 52.2.216.188

4. Installation

Unzip the provided package in a location of your choice.

The package should contain the following files:

  • RemoteDownloadAgent-complete.jar : the agent
  • config.properties : the configuration file
  • LICENSE.md :  the license file
  • VERSION.md : the version file
  • bin/start.sh :  the start script for *NIX systems
  • bin/shutdown.sh : the stop script for *NIX systems
  • bin/java_opts.sh : java related settings for *NIX systems
  • bin/start.bat : the start script for Windows
  • bin/java_opts.bat : java related settings for Windows

5. Configuration

We will preconfigure the config.properties based on your needs.

Here the main items:

serviceName: a human readable identifier of the downloader.
apikey: unique to every downloader. Used by the API Fortress engine to authenticate.
secret: used by the API Fortress engine to authenticate.
use_http (true): enables the HTTP server.
use_ssl (true): enables the SSL encryption for the HTTP server.

6. Running the Agent – *NIX

Run the bin/start.sh script to start the agent which will run in the background.

Run the bin/shutdown.sh script to stop the service.

7. Running the Agent – Windows

Run the bin/start.bat script to start the agent which will run in foreground.

8. Tuning

Even though the agent does not require relevant system resources, it is a good practice to allow the agent to use as much memory as available. By editing the java_opts script (.bat or .sh, depending on the platform) you can tweak the heap memory settings.

Note: Remember that heap memory is not the only type of memory Java uses. Raising the heap to the system limits is not advised, and it will not work. If you are not familiar with these settings, the default will likely meet your requirements.

API Tests vs. Schema Validation

One of the most common questions we receive about API testing is:

What’s the deal? Schema validation was not enough?

Some of these inquiries are moved by real curiosity, others are just polemic, but either cases, the question makes a good point if you’ve never actually tried our platform.

Schema validation has two major flaws:

  1. You need to write a schema
  2. It statically validates your API syntax and grammar, not your data and certainly not your darkest secrets.

For what concerns ( 1 ) deal with it.
( 2 ) is a little more intriguing. Let’s work by example.

{
  "group": "food",
  "items": [ 10,15,17,19 ]
  "install": false,
  "installDay": null
  "delivery": true
  "request_date": 1456249628
}

Of course a schema can determine whether these items are the right types, but no schema can tell that:

  1. Food items should have ids lower than 100
  2. Food items should not be installed (weird, right?)
  3. Food items have no installation day, but they’re deliverable
  4. The response shouldn’t be older than 1 day (caching?)

Moreover, API testing allows you to compare things with the request data. You don’t want to get bathrobes listed while you were looking for your birthday cake. Or, more technically, you don’t want the data within a credit card transaction to be cached.

Right?

The On-Premises Platform

What Is It?

API Fortress can be deploy in nearly any manner that works for you – whether cloud, on-premises, or hybrid.

Our most popular is on-premises, which means the entire platform lives in your environment behind your firewall.

Why?

There are multiple reasons for having an on-prem engine, and these are some of the most common:

  • Satisfies Any Security Requirements
  • Complete Data Ownership
  • Access to Private Staging Environments
  • Mocking
  • Load Testing
  • Unlimited Use
  • Customization

Customization

API Fortress is extremely modular, and most functionalities can be replaced with different code and behave in a different way. Some use cases are:

  • Storing the results of the tests in a dedicate archive, such as DynamoDB, a private MongoDB instance, or an object storage.
  • Customizing the chain of alerts with internal tools.
  • Storing the code of the tests in a location that is not the API Fortress cloud.
  • Adding the ability to ingest and analyze exotic data types.

All this is done with a few lines of Java that we can help you with. The engine itself can work as an SDK to build what you need. For some companies we can be your entire solutions, while others might have certain requirements that will always require customization. For those we can be the ones that cover your 90% use case.

Deployment

A simple Docker/Kubernetes/Open Shift deployment. We ask a handful of questions, setup a configuration file for you, and then you deploy with your preferred container. The system requirements here.

If you would like to trial API Fortress on-premises, simply fill out this survey and we will work on a configuration file for you.

Operations

The engine operates exactly the same way API Fortress does in the cloud.