Known issues / scheduled updates

Engine

  • [ETA: TBD] Nested variable referencing (Ie. v1=${v2} v2=value) does not work in:
    Query parameters
    Expression field in assertion
  • [ETA: TBD] Assert-Matches for US Zip codes does not include district codes

Dashboard

  • [ETA: End of August] In the test list, provide the user a way to override variables when running a test manually

Composer/Embedded Console

  • [ETA: Next release] Import of requests to console does not resolve nested variables (Ie. v1=${v2} v2=value)
  • [ETA: Next release] Import of requests to console does not resolve vault variables
  • [ETA: End of August] provide the user a way to override variables when running a test in development

Downloader

  • [ETA: End of August] Self signed certificates won’t get accepted unless a certificate is installed in the downloader (no bypass option)

Reports

  • [ETA: Next release] Syntax or runtime errors are rarely reported with details
  • [ETA: Next release] HTTP requests should contain all the details of the request itself
  • [ETA: End of Quarter] On complex tests, reports can become humongous and very hard to load

Notifications

  • [ETA: End of Quarter] heavy API failures cause email flooding. Emails should be stateful
  • [ETA: End of Quarter] Small memory leak causes connectors to slowly grow in memory demand
  • [ETA: Next release] On-premises deployments should be able to connect to message queue and create custom notification systems

Misc

  • [ETA: End of August] On-premises: There’s no way to tell when a license is supposed to expire

On-Premises Requirements

The one server setup for API Fortress on-premise is a quick way to get things started in a protected environment. While not ideal for availability or performance, works exactly as expected and provides all the features of the cloud version.

Minimum hardware requirements
CPU: Intel based high frequency quad core processor
Memory: 16 GB RAM
HDD: 250 GB
Memory: the memory impacts significantly on the speed of queries on big data sets. 32 GB is a recommended setup
HDD: All API Fortress reports and metrics are stored. 10 million reports + 30 million metrics can require up to 20GB of disk space

Software requirements
OS: a recent Linux distribution

Classic deployment
Java: Oracle JDK 1.8 series
Tomcat: 7 series
PostgreSQL: 9.5 series
MongoDB: 3.2 series
RabbitMQ: 3.5 series

Docker Deployment
Docker: 1.12

Processes
PostgreSQL: relational database for structured data
MongoDB: document database for reports and metrics
RabbitMQ: message queue
Tomcat: dashboard and engine application
AFScheduler: the API Fortress scheduler
AFMailer: the API Fortress mailer
AFConnector: dynamic data dispatcher for notifications
AFDownloadAgent: the downloader agent (actually performing HTTP calls)

Networking
We assume this deployment will be able to access the services to be tested.

Further connections
HTTP(80) and/or HTTPS(443) inbound traffic enabled for every location that will need access to the dashboards. Ports and services may vary based on system requirements.

Docker
For the Docker deployment to succeed and to ease further updates, the server has to be able to communicate with utils.apifortress.com

API Tests vs. Schema Validation

One of the most common questions we receive about API testing is:

What’s the deal? Schema validation was not enough?

Some of these inquiries are moved by real curiosity, others are just polemic, but either cases, the question makes a good point if you’ve never actually tried our platform.

Schema validation has two major flaws:

  1. You need to write a schema
  2. It statically validates your API syntax and grammar, not your data and certainly not your darkest secrets.

For what concerns ( 1 ) deal with it.
( 2 ) is a little more intriguing. Let’s work by example.

{
  "group": "food",
  "items": [ 10,15,17,19 ]
  "install": false,
  "installDay": null
  "delivery": true
  "request_date": 1456249628
}

Of course a schema can determine whether these items are the right types, but no schema can tell that:

  1. Food items should have ids lower than 100
  2. Food items should not be installed (weird, right?)
  3. Food items have no installation day, but they’re deliverable
  4. The response shouldn’t be older than 1 day (caching?)

Moreover, API testing allows you to compare things with the request data. You don’t want to get bathrobes listed while you were looking for your birthday cake. Or, more technically, you don’t want the data within a credit card transaction to be cached.

Right?

The “on-premises” Engine

What it is

API Fortress can also come in an “on-premises” flavor.
“On-prem” means that an API Fortress engine will live inside your infrastructure and will interact with your APIs from the inside, in opposition to the cloud solution where everything resides in the API Fortress Inc. infrastructure.

Why

There are multiple reasons for having an on-prem engine, and these are some of the most common:

  • Super big projects that need to scale over API Fortress cloud capacity
  • Extreme security restrictions
  • Access to private / sensitive infrastructures

But there’s another reason that makes it suitable for a number of users: customization

Customization

API Fortress is extremely modular and most functionalities can be replaced with different code, behaving in a different way.
Some common use cases are:

  • Storing the results of the tests in a dedicate archive, such as DynamoDB, a private MongoDB instance or an object storage
  • Customizing the chain of alert with internal tools
  • Storing the code of the tests in a location that is not the API Fortress cloud
  • Adding the ability to ingest and analyze exotic data types

All this is done with a few lines of Java, really. The engine itself will work as an SDK to build what you want to build. Or you can ask our team, we will be glad to help.

Deployment

Docker, Amazon AMI? We decided to keep it classic, plain and simple.
A JAR file will be delivered to you will all the necessary configuration files. The system requirements are:

  • Java 8
  • 1GB RAM
  • 80MB HDD
  • Linux, Windows, OSX

As a default, the engine will communicate with the cloud service to operate, but as we previously said, customization will provide infinite variations.
Some common use plug-ins will also be provided with the package.

Operations

The engine operates exactly as an API Fortress cloud engine. This means it will listen for scheduled events from the cloud and will make itself available to triggered events via its API.
The latter way of interacting with the engine probably is the most interesting as it allows you to better integrate with your company automations.