Artillery Load Testing Introduction

Artillery is a Node.js package designed for testing backend systems, such as APIs. Out of the box, it supports your standard HTTP APIs, along with protocols like WebSockets. It also integrates with the Socket.io library for Node.js applications. Artillery also sports a plugin interface to extend its functionality.

An important distinction to make with Artillery is that it’s not designed for testing web applications through the browser or client-side JavaScript. If you need to do load testing on a web application through the browser, you’ll need a tool focused on UI testing or built to do web performance testing. For example, you can do performance testing using TestCafe or use a more robust load testing tool like Apache JMeter.

Setting up Artillery

Since Artillery is a Node.js package, it’s relatively simple to install as long as your system runs Node.js (version 12.0 or above). You can download Node.js to install it in your system if you want to follow along, or install it using my preferred method or setting up nvm (Node Version Manager).

Once you have Node.js set up, installing Artillery is a simple one-line command:

npm install -g artillery

Testing with a consistent flow of requests per second

First, we’ll start with a simple example, creating a test that hits an API endpoint for one minute, sending 25 virtual users per second. The following Artillery test script would run this load test:

config:
  target: "https://airportgap-stress-test.dev-tester.com/api"
  phases:
    - duration: 60
      arrivalRate: 25

scenarios:
  - name: "Get first page of airports"
    flow:
      - get:
          url: "/airports"

After saving this script in a file – in our example, our file is called airports.yml – we can run through the test with Artillery from the command line with the following:

artillery run airports.yml

This command kicks off the load test, with Artillery making requests to the specified target URL at a rate of 25 requests every second. Every ten seconds, your terminal prints out an update of what Artillery performed and different results such as scenarios launched and completed, response status codes received, and different response time stats.

All virtual users finished
Summary report @ 17:12:54(+0900) 2021-02-27
  Scenarios launched:  1500
  Scenarios completed: 1500
  Requests completed:  1500
  Mean response/sec: 24.81
  Response time (msec):
    min: 110.1
    max: 443.3
    median: 137.6
    p95: 178.1
    p99: 251
  Scenario counts:
    Get first page of airports: 1500 (100%)
  Codes:
    200: 1500

For more detailed use cases, you can check this blog.