Assertions for Metrics / Performance

We have a more detailed guide on working with the http response, but wanted to provide specifics on creating assertions against performance metrics.

Here is an example using the CODE VIEW of the code. Please note that overall is fetch and latency combined.

<assert-less expression="payload_response.metrics.latency" value="200" type="integer"/>

<assert-less expression="payload_response.metrics.fetch" value="350" type="integer"/>

<assert-less expression="payload_response.metrics.overall" value="450" type="integer"/>

Setup Connectors (Slack)

Here is a quick guide to setting up Slack.

  1. FIRST! Get your Slack webhook. This is the link to generate a new one.
  2. In API Fortress go to company settings (top right gear icon)
  3. Click on Alert Groups
  4. Create a new Alert Group (if necessary)
  5. Add recipients to the Alert Group (if necessary)
  6. Click on the Connectors icon
  7. Choose Slack from the dropdown
  8. Add your Slack webhook
    Important! If the URL is https://hooks.slack.com/services/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
    then only copy in
    T08QUN1SR/B08QUGM8T/qVvPDLTa3OtvskrpyKBTfhvI

notifications - alert group and slack

Setup Notifications (Performance Alerts)

Performance is the first thing that people think of when they are concerned about their APIs. By scheduling accuracy tests to run automatically, the data from those executions can then be used for proper insight into how the API is performing globally.

Step 1. Go to the Dashboard

Step 2. Click on Metrics

Step 3. Click on a Footprint

Step 4. Click on the Bell to Add a Monitor

Step 5. Fill in the Information

Enter the name of the alert, select the type of monitor (latency or fetch), and the max value for the trigger. Latency is the time it takes to ping the endpoint and it responds, and fetch is the time it takes to download the response. Large responses will lead to large fetch times, so be aware of this when setting a number. Values are in milliseconds.

Alert Thresholds (Using JSON Connector)

The JSON Alert Connector sends information about a failure in JSON format and posts it to a given WebHook URL.

The JSON connector with the threshold variation does the same thing with a twist – rather than posting every event, it does so when the error rate exceeds a certain threshold. The connector has been designed to be used with services like Zapier or IFTTT. It can also work with other similar services, but those are the two popular ones.

Configure the Connector

  1. Access the company dashboard by clicking the gear icon top right
  2. Select “Alert groups”
  3. Create a new alert group if you don’t have already one for this purpose
  4. Select the socket icon
  5. Select “+ Connector to this group”
  6. Select “JSON Alert /w threshold” from the list
  7. Enter the required settings*
  8. Save
  9. From the main dashboard, edit the “Settings” the project(s) you want to assign the alert group to if the group has not been assigned yet

Settings

URL: The URL of the WebHook the alert should be sent to.
Counter: The number of errors per specific test in a time frame before the alert is actually sent out.
TTL: The time frame, expressed in seconds.

For example, 3 errors over a TTL of 600 (10mins).

Behavior

The connector will collect errors for each specific test. When the first error occurs, the time slot expressed by the TTL starts.  When the error reaches the counter number, the alert is sent and the time slot resets.

The JSON being sent is going to look like the following:

{
    "date": "2017-12-29T14:36:31+0000",
    "eventId": "5a4aa0a1-d071-4a05-981d-ff57e4ff3897",
    "test": {
        "name": "book",
        "id": "123a"
    },
    "value2": "fake project",
    "value1": "book",
    "companyName": "fake company",
    "eventType": "failure",
    "criticalFailures": [{
        "action": "get",
        "expression": "get http://www.example.com",
        "status": "Generic Failure",
        "failureType": "MissingPropertyException",
        "extra": "Error parsing URL. Missing variable yay"
    }],
    "companyId": 1,
    "failuresCount": 1,
    "location": "Ashburn,Virginia",
    "projectName": "fake project",
    "projectId": 1
}

Typical use cases

Zapier – webhook to email

Configure a Zap triggered by a WebHook:

zapier_1That URL is the one that needs to be used in the connector configuration screen.

Add a “Send Email” step, and configure it to use the retrieved data:

zapier_2For this example, we used a template like this:

The test "{{30468424__test__name}}" for the project "{{30468424__projectName}}" Hit the maxium error threshold.
See a sample event by accessing:
https://mastiff.apifortress.com/app/web/events/details/{{30468424__eventId}}?cid={{30468424__companyId}}

Where 30468424 is the Zap ID.

IFTTT – webhook to email

ifttt_1Create an Applet that gets triggered by a WebHook and configure it to send an email. This is pretty simple actually…

 

Copy a Test

Here we will show you how to copy and paste a test. Before that, it should be noted that this is often not the right thing to do. Take a step back and ask yourself if this is actually the perfect solution. Often users get into habits of doing things the way they were used to, but there are many things we have specifically built to improve work efficiency. Some questions to ask yourself:

  • Are you looking to do negative testing? You can potentially do that in the same test using data sets and the IF component + status codes.
  • Are you looking to run from different environments? No problem with Presets.
  • Do you want to run most, but not all, test in a specific project? Execute tests using tags. (example, using a unique webhook and the tag of ‘live.’)

First, enter the project and test. Then click over to the Published version of the test, then the Copy to Clipboard option becomes available.

copy 1

Click Copy to Clipboard and then browse to the Tests view of the same, or another project. There Paste from Clipboard becomes available.

copy 2

So here is that entire adventure in video form.

Write Tests in Code

API Fortress has three unique advantages in the market – magic, the visual composer, and the built in assertions/operations. With that said, you are not bound to them exclusively. If you are more comfortable using code, that option is available as well.

Code View

First, the whole test can be seen and edited “naked,” without our glamorous UI. Behind the curtains, the test is described using the XML markup language.

To use it, you simply need to look at the top right of the composer. The default is VISUAL COMPOSER, but right next to it is CODE VIEW. Click that.

code view - 1

Now you will see the markup language that is the basis of API Fortress.

code view - 2More experienced tests may find this to be the most efficient manner to use the platform.
Tip: The best way to learn the markup? Build your tests using the visual composer/magic, then switch to code view and have a look!

A Groovier Approach

Whether you are using the code view, or the visual composer, one important aspect to note is that all “evaluated” fields are actually able to execute a subset of Groovy commands.

For example, let’s take this assertion that verifies whether the “items” element is an array.
assert-is
Or in code view:
<assert-is expression="payload.whatever.items" type="array"/>

Now let’s say you know something more about this array, such as it should always contain more than 3 elements:

assert_greater

Or in code view
<assert-greater expression="payload.whatever.items.size()"  value="3" type="integer"/>

Notice how in the expression field we deliberately used the size() command to retrieve the size of the object at its left.

Even More Serious Grooviness

Moreover, Groovy can be put within SET components.

The first scenario is when you want to set a variable that is not a String. The best way to do it is using the Variable Mode “Object.” The value, in this case, will be evaluated as Groovy code.

set_obj
<set var="number" object="otherNumber+50"/>

Here we are assuming that otherObject is a predefined numeric variable. When the SET is executed, the number variable will be an integer.

The second scenario is when you want to write extensive logic.

Choose the SET component, then choose the item “Language” in the type drop-down (when using Visual Composer), or enter lang=”java” when writing it in Code View.

setOr in code view:

<set var="myVar" lang="java">
<![CDATA[
def item = payload.whatever.items.find { it -> it.id==11 }
return item.amount
]]>
</set>

Templating

What about all the fields that are not explicitly evaluated? Like URL, value, or POST Body? Or the content of a comment? It is often extremely useful to evaluate content on those as well. This is possible using the template syntax.
eq

<assert-equals expression="payload.id" value="${req_id}"/>

This assertion, for example, is evaluating the req_id variable right within the value.

A Little Bit of Everything

Let’s join everything we’ve learned into one snippet:

<set var="data" lang="java">
<![CDATA[
  def items = payoad.whatever.items.find{ it-> it.id>100}
  return items.collect{ it -> it.providerId}
]]>
</set>
<each expression="data">
  <post url="http://www.example.com/${counter+1}" params="[:]" var="payload2" mode="json">
   <postBody contentType="application/json">
     <![CDATA[{"providerId":${_1}}]]>
   </postBody>
  </post>
  <set var="counter" object="counter+1"/>
</each>

Want to learn more about Groovy?

Follow this link to the official documentation:
http://groovy-lang.org/documentation.html

Important Note
For security concerns, the cloud version of API Fortress is sandboxed. Meaning many programming language features are disabled. On-premises eliminates that restraint. While on cloud, if you think a specific feature should be safe to be enabled but is disabled, please contact us and we’ll do our best to help you!

Change Environments and Presets

One of the key aspects of creating meaningful tests is being able to run them against different environments, to make sure that the upcoming release will pass the tests designed for the production release.

To do so, API Fortress introduced the “Environments” feature.

An “environment” is a collection of override variables. An override variable is a variable that will replace the value of a variable that has been hardcoded within the Vault or the test itself, during a predefined session.

You can access the feature in the “tests list” by clicking the “hamburger menu” icon:

hamOr in the composer, clicking on “environments”:

ham2

Simply Overriding Variables

In the override panel, you can type in which variables you want to override and activate the preset:

overBy doing so, you will be overriding the variables for the current session, as long as the preset is activated. This override will be performed to your session only.

Saving the Environment

overOnce your preset is ready and you are willing to save it for later use, you can click on the “save” button and give it a name. The Preset will be associated with the current project and will be available to you and the other users.

Activating a Saved Environment

Once your environment is saved, you can activate it by choosing it from the “environments” dropdown that will show up by clicking on the arrow-down, to the right of the environments icon.

selIn this example, we named the environment “staging”.

To disable the environment and go back to defaults, just select “None”.

Using a Saved Environment via API

When using the API by invoking a test execution, you can have API Fortress load an environment based on its name.

To do so, just provide the special variable “apif_env” in the params section, and provide the name of the environment you want to load, as in:

{
  "Content-Type":"application/json",
  "payload":"{}",
  "params":{"apif_env":"staging"}
}

Use the API (For CI/CD & More)

Here is a guide showing you various ways to call the API. This is particularly useful for those that want to use the platform to run tests during continuous deployments, and who are not using our Jenkins plugin.

First, you need to create the API Hook by going to the Company Settings page and clicking API Hooks section. To do this you need to be a Manager of the Company.

companySetting

Here, click +API Hook you will be prompted to choose a project. Once done, you will have the Hook URL that will give you the ability to query resources from that project. An example is
https://mastiff.apifortress.com/app/api/rest/v3/34d8mm70-c03e-267a-9fa1-90b9flsbcea2607
Note: That unique project hook is 34d8mm70-c03e-267a-9fa1-90b9flsbcea2607. This is useful for later.

A Quick Table of Contents:

Unauthenticated Endpoints

Test Run

Invoking the execution of a test via the API is a key feature of API Fortress. It allows you to trigger one, or multiple, tests to run. It can also receive input variables to override those in the tests themselves. Finally, you can also send a payload to the platform to be tested against.

Optionally, for all the endpoints you can add the following query parameters:

  • sync (boolean): the test will run synchronously and return the result as the response payload.
  • dryrun (boolean): the test will run but no events or metrics will be stored.
  • silent (boolean): no alerts will be triggered by the execution of the test.

Basic: Run a Test By Test ID

How to find a test ID.

GET https://mastiff.apifortress.com/app/api/rest/v3/project_hook/tests/test_id/run

Mock Example:

curl -v https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/34d8mm70-c03e-267a-9fa1-90b9flsbcea2607/tests/129d32j9dksdoo23e393/run

Advanced: Run a Test With Additional Information (Variable Override)

If you want to override variables, or include a payload, you have to use:

POST https://mastiff.apifortress.com/app/api/rest/v3/project_hook/tests/test_id/run

The request body will contain the variable you want to override.

Example body

{
  "payload": "{\"id\":\"123\"}",
  "Content-Type": "application/json",
  "params": {
    "server": "staging"
  }
}
  • Params will be accessible as variables inside the test.
  • Payload will be parsed based on the “Content-Type” field and will be accessible via the “payload” variable.

payload AND params are not required at the same time.

Mock Example:

curl -X POST \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/tests/35082607e4b0mm613lm82e8f/run \
-H 'content-type: application/json' \
-d '{
"payload": "{\"id\":\"123\"}",
"Content-Type": "application/json",
"params": {"server":"staging"}
}'

Automatch: Run Multiple Tests Based On a URL Pattern

Automatch is a way to simultaneously launch tests inside a project. The tests to run are selected by comparing the URL provided in the payload to the “automatch” configuration pattern in the tests. Just like the advanced run endpoint, you can override both variables and the payload:

POST https://mastiff.apifortress.com/app/api/rest/v3/project_hook/tests/automatch

Example body

{
  "payload": "{\"id\":\"123\"}",
  "Content-Type": "application/json",
  "params": {
    "server": "staging"
  },
  "url": "http://www.testme.com"
}

Mock Example:

curl -X POST \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/tests/automatch \
 -H 'content-type: application/json' \
 -d '{
 "payload": "{\"id\":\"123\"}",
 "Content-Type": "application/json",
 "params": {"server":"staging"}
}'

Basic Tags: Run Tests With Certain Tags

GET https://mastiff.apifortress.com/app/api/rest/v3/project_hook/tests/tag/tag/run

Mock Example:

curl -X GET \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/tests/tag/staging/run \
 -d '{
 "payload": "{\"id\":\"123\"}",
 "Content-Type": "application/json",
 "params": {"server":"staging"}
}'

Advanced Tags: Run Tests Tagged With a Certain Word

POST https://mastiff.apifortress.comapp/api/rest/v3/project_hook/tests/tag/tag/run

Example body

{
  "payload": "{\"id\":\"123\"}",
  "Content-Type": "application/json",
  "params": {
    "server": "staging"
  }
}

Mock Example:

curl -X POST \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/tests/tag/staging/run \
 -H 'content-type: application/json' \
 -d '{
 "payload": "{\"id\":\"123\"}",
 "Content-Type": "application/json",
 "params": {"server":"staging"}
}'

Basic By Project: Run all Published Tests of a Specific Project

GET https://mastiff.apifortress.com/app/api/rest/v3/project_hook/tests/run-all

Mock Example:

curl -X GET \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/tests/run-all \
 -d '{
 "payload": "{\"id\":\"123\"}",
 "Content-Type": "application/json",
 "params": {"server":"staging"}
}'

Advanced By Project: Run All Published Tests of a Project With Additional Information

POST https://mastiff.apifortress.com/app/api/rest/v3/project_hook/tests/run-all

Example body

{
  "payload": "{\"id\":\"123\"}",
  "Content-Type": "application/json",
  "params": {
    "server": "staging"
  }
}

Mock Example:

curl -X POST \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/tests/run-all \
 -H 'content-type: application/json' \
 -d '{
 "payload": "{\"id\":\"123\"}",
 "Content-Type": "application/json",
 "params": {"server":"staging"}
}'

Insights (Data & Information)

The API Fortress API also allows you to retrieve metrics and data of your tests.

Events: Shows Information About Test Failures and Successes

GET https://mastiff.apifortress.com/app/api/rest/v3/project_hook/insights/events

Mock Example:

curl -X GET \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/insights/events

Events Stream: Show Test Success and Failure Information As An Event Stream

GET https://mastiff.apifortress.com/app/api/rest/v3/project_hook/insights/events/stream

Mock Example:

curl -X GET \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/insights/events/stream

Metrics: Provide Details on Performance

GET https://mastiff.apifortress.com/app/api/rest/v3/project_hook/insights/metrics

Mock Example:

curl -X GET \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/insights/metrics

Metrics Stream: Provide Details on Performance in an Event Stream

GET https://mastiff.apifortress.com/app/api/rest/v3/project_hook/insights/metrics/stream

Mock Example:

curl -X GET \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/insights/metrics/stream

List of Tests in a Project

GET https://mastiff.apifortress.com/app/api/rest/v3/project_hook/tests

Mock Example:

curl -X GET \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/tests

Details of a Project

GET https://mastiff.apifortress.com/app/api/rest/v3/project_hook/

Mock Example:

curl -X GET \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/

Authenticated endpoints

The following endpoints might contain sensitive information, therefore authentication is required.

Authentication

GET https://mastiff.apifortress.com/app/api/rest/v3/project_hook/login

The user credentials, provided by basic HTTP authentication, need to match the user profile selected during the project hook creation. This endpoint will generate a JWT token which is valid only for the provided project hook. The token is necessary for all the endpoints containing potentially sensitive information, and updating data. The authentication is achieved by sending a valid access token in the request header, in the form Authorization: Bearer access_token

Mock Example:

curl -X GET \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/login \
 -H 'authorization: Basic ABCD' \
 -H 'content-type: application/json'

Insights (Data & Information)

Provide Details About a Specific Event

GET https://mastiff.apifortress.com/app/api/rest/v3/project_hook/insights/events/event_id
The event_id can be retrieved by performing the events endpoint first.

Mock Example:

curl -X GET \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/insights/events/5963451d12b87d3379d2f1co \
 -H 'autho: Bearer ABCD' \
 -H 'content-type: application/json'

Tests

Provide the Details About a Specific Test

GET https://mastiff.apifortress.com/api/rest/v3/project_hook/tests/test_id

The test_id can be retrieved by calling the test endpoint or, the easiest way, by copying it from the interstitial page:

testidIntersitialtestID

Mock Example:

curl -X GET \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/tests/35082607e4b0mm613lm82e8f \
 -H 'authorization: Bearer ABCD' \
 -H 'content-type: application/json'

Show the Content of the Unit for the Given Test

GET https://mastiff.apifortress.com/app/api/rest/v3/project_hook/tests/test_id/unit

Mock Example:

curl -X GET \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/tests/35082607e4b0mm613lm82e8f/unit \
 -H 'authorization: Bearer ABCD' \
 -H 'content-type: application/json'

Show the Content of the Unit for the Given Test as a Downloadable File

GET https://mastiff.apifortress.com/app/api/rest/v3/project_hook/tests/test_id/unit/file

Mock Example:

curl -X GET \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/tests/35082607e4b0mm613lm82e8f/unit/file \
 -H 'authorization: Bearer ABCD' \
 -H 'content-type: application/json'

Show the Content of the Input-Set for the Given Test

GET https://mastiff.apifortress.com/app/api/rest/v3/project_hook/tests/test_id/input

Mock Example:

curl -X GET \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/tests/35082607e4b0mm613lm82e8f/input \
 -H 'authorization: Bearer ABCD' \
 -H 'content-type: application/json'

Show the Content of the Input-Set for the Given Test as a Downloadable File

GET https://mastiff.apifortress.com/app/api/rest/v3/project_hook/tests/test_id/input/file

Mock Example:

curl -X GET \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/tests/35082607e4b0mm613lm82e8f/input/file \
 -H 'authorization: Bearer ABCD' \
 -H 'content-type: application/json'

Update Tests

Update the Unit of a Given Test

POST https://mastiff.apifortress.com/app/api/rest/v3/project_hook/tests/test_id/unit/update
The unit will be passed as body.

Example body:

{
  "text": "<unit xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" name=\"main\" xsi:noNamespaceSchemaLocation=\"http://apifortress.com/app/unit.xsd\"><requirements></requirements><configs></configs><sequence name=\"main\"><assert-exists expression=\"payload\"/></sequence></unit>"
}

Mock Example:

curl -X POST \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/tests/35082607e4b0mm613lm82e8f/unit/update \
 -H 'authorization: Bearer ABCD' \
 -H 'content-type: application/json' \
 -d '{
 "text": "<unit xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" name=\"main\" xsi:noNamespaceSchemaLocation=\"http://apifortress.com/app/unit.xsd\"><requirements></requirements><configs></configs><sequence name=\"main\"><assert-exists expression=\"payload\"/></sequence></unit>"
}'

Update the Unit of a Given Test by Accepting It As a Raw POST Body

POST https://mastiff.apifortress.com/app/api/rest/v3/project_hook/tests/test_id/unit/update/file
The unit will be passed as body.

Example body

<unit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" name="main" xsi:noNamespaceSchemaLocation="http://apifortress.com/app/unit.xsd">
    <requirements></requirements>
    <configs></configs>
    <sequence name="main">
        <get url="${protocol}${domain}${endpoint}" params="['id':id]" var="payload" mode="json"></get>
        <assert-equals expression="payload.status" value="200" comment="status is equal to 200, if not stop the test" stoponfail="true"/>
    </sequence>
</unit>

Mock Example:

curl -X POST \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/tests/35082607e4b0mm613lm82e8f/unit/update/file \
 -H 'authorization: Bearer ABCD' \
 -H 'content-type: text/xml' \
 -d '<unit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" name="main" xsi:noNamespaceSchemaLocation="http://apifortress.com/app/unit.xsd">
 <requirements></requirements>
 <configs></configs>
 <sequence name="main">
 <get url="${protocol}${domain}${endpoint}" params="['\''id'\'':id]" var="payload" mode="json"></get>
 <assert-equals expression="payload.status" value="200" comment="status is equal to 200, if not stop the test" stoponfail="true"/>
 </sequence>
</unit>'

Update the Input-Set of the Given Test

POST https://mastiff.apifortress.com/app/api/rest/v3/project_hook/tests/test_id/input/update
The input will be passed as body.

Example body

{
  "text": "<sets xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" name=\"main\" xsi:noNamespaceSchemaLocation=\"http://mastiff.apifortress.com/app/input.xsd\"><global><param name=\"country\">US</param></global><set name=\"search 1\"><param name=\"q\">music</param></set></sets>"
}

Mock Example:

curl -X POST \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/tests/35082607e4b0mm613lm82e8f/input/update \
 -H 'authorization: Bearer ABCD' \
 -H 'content-type: application/json' \
 -d '{
 "text": "<sets xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" name=\"main\" xsi:noNamespaceSchemaLocation=\"http://mastiff.apifortress.com/app/input.xsd\"><global><param name=\"country\">US</param></global><set name=\"search 1\"><param name=\"q\">music</param></set></sets>"
}'

Update the Input of a Given Test By Accepting It As a Raw Post Body

POST https://mastiff.apifortress.com/app/api/rest/v3/project_hook/tests/test_id/input/update/file
The input will be passed as body.

Example body

<sets xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" name="main" xsi:noNamespaceSchemaLocation="http://mastiff.apifortress.com/app/input.xsd">
    <global>
        <param name="protocol">https://</param>
        <param name="domain">mastiff.apifortress.com</param>
        <param name="endpoint">/app/api/examples/retail/product</param>
    </global>
    <set name="ID 1">
        <param name="id">511</param>
    </set>
</sets>

Mock Example:

curl -X POST \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/6de267cd-df01-4782-8046-3seee6f4c093782/tests/35082607e4b0mm613lm82e8f/input/update/file \
 -H 'authorization: Bearer ABCD' \
 -H 'content-type: text/xml' \
 -d '<sets xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" name="main" xsi:noNamespaceSchemaLocation="http://mastiff.apifortress.com/app/input.xsd">
 <global>
 <param name="protocol">https://</param>
 <param name="domain">mastiff.apifortress.com</param>
 <param name="endpoint">/app/api/examples/retail/product</param>
 </global>
 <set name="ID 1">
 <param name="id">511</param>
 </set>
</sets>'

Creating Tests Outside of API Fortress

Validators

When developing outside the API Fortress Composer, you will want to be sure the syntax of your units and input-sets are correct. For this work there are two different endpoints. One for the input-set, and one for the unit.

Validate the Unit

POST https://mastiff.apifortress.com/app/api/rest/v3/validators/unit
The unit will be passed as body.

Example Body

{
  "text": "<unit xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" name=\"main\" xsi:noNamespaceSchemaLocation=\"http://apifortress.com/app/unit.xsd\"><requirements></requirements><configs></configs><sequence name=\"main\"><assert-exists expression=\"payload\"/></sequence></unit>"
}

Mock Example:

curl -X POST \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/validators/unit \
 -H 'content-type: application/json' \
 -d '{
 "text": "<unit xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" name=\"main\" xsi:noNamespaceSchemaLocation=\"http://apifortress.com/app/unit.xsd\"><requirements></requirements><configs></configs><sequence name=\"main\"><assert-exists expression=\"payload\"/></sequence></unit>"
}'

Validate the Input-Set

POST https://mastiff.apifortress.com/app/api/rest/v3/validators/input
The input-set will be passed as body

Example Body:

{
  "text": "<sets xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" name=\"main\" xsi:noNamespaceSchemaLocation=\"http://mastiff.apifortress.com/app/input.xsd\"><global><param name=\"country\">US</param></global><set name=\"search 1\"><param name=\"q\">music</param></set></sets>"
}

Mock Example:

curl -X POST \
https://private-e9aac-apifortressv3.apiary-mock.com/app/api/rest/v3/validators/input \
 -H 'content-type: application/json' \
 -d '{
 "text": "<sets xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" name=\"main\" xsi:noNamespaceSchemaLocation=\"http://mastiff.apifortress.com/app/input.xsd\"><global><param name=\"country\">US</param></global><set name=\"search 1\"><param name=\"q\">music</param></set></sets>"
}'

 

 

Use the Jenkins Plugin

The API Fortress Jenkins Plugin allows you to run tests during automated builds. The tests that fail may, or may not, cause a build failure based on your settings.

Watch the walkthrough video.

Install the Jenkins plugin

From the Jenkins Update Center, search and install the API Fortress plugin.

jenkins1A Jenkins restart may be required. You will be prompted accordingly.

Configure a build step

Add the API Fortress build step where it best suits your needs.

jenkins2

Modes

There are 4 testing modes available. “Run single test” will run a single test, while the others will run suites of tests.

Run Single Test

You can run a single test by providing a test ID. Test IDs can be found in test interstitial pages.

See where to retrieve the test IDs later in this document.

Run Automatch

You can run our “automatch” mode by providing a URL representing a certain endpoint. You can configure automatch patterns in the “Automatch” section of each test. Read more about automatch here .

Run by Tag

Choosing “by tag” will run multiple tests marked with a certain tag. Tags can be added and edited in the test details.

Run Project

By running a full project, you will be running all tests contained in the project.

Hook URL

The hook URL is an API endpoint that references a specific project within an API Fortress environment.

See the Hook URLs section to create one.

Options

The following options can apply to any mode.

Blocking

The plugin can silently run (blocking = false), which lets the build continue with a success and inform you if the test failed using the various methods available. Or it can actively determine the build success (blocking=true), so that the build will wait for the tests result and stop the build if the tests fail.

Dry-run

If checked, no events will be stored within API Fortress. To be used in conjunction with “blocking.”

Silent

If checked, no alerts will be sent if the tests fail.

Parameters Override

You can override up to 3 parameters in the test scope. These variables will appear within the test scope just like any other variable. Useful if you are looking to override the domain of the service being tested (ie. staging vs production).

Hook URLs

Hook URLs are API endpoints representing one of your projects. These endpoints have unique paths, and do not require authentication for most non-disclosing operations.

To create a Hook URL:

  1. Access your API Fortress dashboard.
  2. Enter the company configuration screen (gear icon in the top right).
  3. Choose API Hooks on the side menu.
  4. Create a new Hook by choosing the user (necessary for advanced functions) and the project.
  5. A unique URL will be generated. That is the URL to be used in your build step configuration.

Test IDs and Project ID

To retrieve a specific test ID from the API Fortress dashboard:

  1. Access the tests list for the project you intend to use.
  2. Edit one of tests and access the interstitial page.
  3. Click on “show/hide test information.”
    jenkins4
  4. The test ID and the Project ID are shown here.

Build from Postman Collections

API Fortress can now generate a test from a Postman collection!

The first step is exporting your collection from within Postman.

collection

Next to your Collection name click the ellipsis (three dots) … then click Export and choose ‘Collection v2.’

export exportCollectionV2

Once done, go to API Fortress and create a New Test.

newTest

On the interstitial page, click Build from Spec.

buildFromSpec

In the following page, choose Postman Collection from the dropdown menu, and upload the collection file we exported. Click Save.

specFile

For a new test, choose From Scratch and then click the check. If you are updating a test then you would use the Merge option.

buildoptions

This imports the API call to the composer. Now we can use this to call the payload and build a test automatically!

testcreated

The easiest way to create a test is using our Magic tool. To do so you need to import the call into the console. First, select the call in the composer, which will highlight it, then click the Import button (shown highlighted below).

importFromComposer

This will open the HTTP console at the bottom and load the call.

importToConsole

When the call is in the console, click the Send button in the top right and you will see the response.

response

At this point, the Magic tool can do the test for you. Click the magic wand icon next to the send icon at the top right of the console, that will launch the Magic tool. Click Continue a few times and voila, a complete API test generated without having to write any code.

magic

You can review the test and add some more logic. This is just a simple version of the test!