Performance testing is a crucial part of ensuring the stability and scalability of web applications. k6 is a modern, open-source load testing tool that allows developers and testers to script and execute performance tests efficiently. In this blog, we’ll explore the basics of k6 and write a simple test script to get started.

What is k6?

k6 is a load testing tool designed for developers. It is written in Go but uses JavaScript for scripting tests. Key features include,

• High performance with minimal resource consumption
• JavaScript-based scripting
• CLI-based execution with detailed reporting
• Integration with monitoring tools like Grafana and Prometheus

Installation

For installation check : https://grafana.com/docs/k6/latest/set-up/install-k6/

Writing a Basic k6 Test

A k6 test is written in JavaScript. Here’s a simple script to test an API endpoint,

import http from 'k6/http';
import { check, sleep } from 'k6';

export let options = {
  vus: 10, // Number of virtual users
  duration: '10s', // Test duration
};

export default function () {
  let res = http.get('https://api.restful-api.dev/objects');
  check(res, {
    'is status 200': (r) => r.status === 200,
  });
  sleep(1); // Simulate user wait time
}

Running the Test

Save the script as script.js and execute the test using the following command,

k6 run script.js

Understanding the Output

After running the test, k6 will provide a summary including

1. HTTP requests: Total number of requests made during the test.

2. Response time metrics:

• min: The shortest response time recorded.

• max: The longest response time recorded.

• avg: The average response time of all requests.

• p(90), p(95), p(99): Percentile values indicating response time distribution.

3. Checks: Number of checks passed or failed, such as status code validation.

4. Virtual users (VUs):

• vus_max: The maximum number of virtual users active at any time.

• vus: The current number of active virtual users.

5. Request Rate (RPS – Requests Per Second): The number of requests handled per second.

6. Failures: Number of errors or failed requests due to timeouts or HTTP status codes other than expected.

Next Steps

Once you’ve successfully run your first k6 test, you can explore,

• Load testing different APIs and endpoints
• Running distributed tests
• Exporting results to Grafana
• Integrating k6 with CI/CD pipelines

k6 is a powerful tool that helps developers and QA engineers ensure their applications perform under load. Stay tuned for more in-depth tutorials on advanced k6 features!

How i came across this Apache Bench aka AB tool ?

When i need to load test to check the rate limiting in my previous blog on Gatekeeper Cloud Pattern, i was searching a new tool to load test. Usually i prefer Locust, but this time i wanted to search new tool. That’s how i came across ab .

Apache Bench

Apache Bench (ab) is a simple and powerful command-line tool for performance testing web servers. It helps developers measure the performance of HTTP services by simulating multiple requests and analyzing the results.

In this blog, we’ll cover everything you need to get started, with examples for various use cases which i tried.

0. Output

Server Software:        Werkzeug/3.1.3
Server Hostname:        localhost
Server Port:            8090

Document Path:          /
Document Length:        16 bytes

Concurrency Level:      10
Time taken for tests:   2.050 seconds
Complete requests:      2000
Failed requests:        0
Total transferred:      378000 bytes
HTML transferred:       32000 bytes
Requests per second:    975.61 [#/sec] (mean)
Time per request:       10.250 [ms] (mean)
Time per request:       1.025 [ms] (mean, across all concurrent requests)
Transfer rate:          180.07 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       1
Processing:     2   10   2.3     10      37
Waiting:        2   10   2.2     10      37
Total:          3   10   2.3     10      37

It gives a detailed output on time taken, data transferred and other stress details.

1. Basic Load Testing

To send 100 requests to a server with 10 concurrent connections

ab -n 100 -c 10 http://example.com/

• -n 100 : Total 100 requests.
• -c 10 : Concurrent 10 requests at a time.
• http://example.com/: The URL to test.

2. Testing with POST Requests

For testing POST requests, we can use the -p option to specify the data file and -T to set the content type.

ab -n 50 -c 5 -p post_data.txt -T "application/x-www-form-urlencoded" http://example.com/api

• -p post_data.txt – File containing POST data.
• -T "application/x-www-form-urlencoded" – Content type of the request.

3. Testing with Custom Headers

To add custom headers to your request

ab -n 100 -c 20 -H "Authorization: Bearer <token>" http://example.com/api

• -H "Authorization: Bearer <token>" – Adds an Authorization header.

4. Benchmarking with Keep-Alive

By default, ab closes the connection after each request. Use -k to enable keep-alive

ab -n 500 -c 50 -k http://example.com/

• -k Enables HTTP Keep-Alive.

5. Outputting Detailed Results

To write the results to a file for analysis

ab -n 200 -c 20 http://example.com/ > results.txt

• > results.txt – Saves the results in a text file.

6. Configuring Timeout Values

To set a custom timeout,

ab -n 100 -c 10 -s 60 http://example.com/

• -s 60: Sets a 60-second timeout for each request.

Apache Bench outputs key metrics such as

• Requests per second: Average number of requests handled.
• Time per request: Average time per request.
• Transfer rate: Data transfer rate.

Locust provides powerful event hooks, such as test_start and test_stop, to execute custom logic before and after a load test begins or ends. These events allow you to implement setup and teardown operations at the test level, which applies to the entire test run rather than individual users.

In this blog, we will

1. Understand what test_start and test_stop are.
2. Explore their use cases.
3. Provide examples of implementing these events.
4. Discuss how to run and validate the setup.

What Are test_start and test_stop?

• test_start: Triggered when the test starts. Use this event to perform actions like initializing global resources, starting external systems, or logging test start information.
• test_stop: Triggered when the test ends. This event is ideal for cleanup operations, aggregating results, or stopping external systems.

These events are global and apply to the entire test environment rather than individual user instances.

Why Use test_start and test_stop?

• Global Setup: Initialize shared resources, like database connections or external services.
• Logging: Record timestamps or test details for audit or reporting purposes.
• External System Management: Start/stop services that the test depends on, such as mock servers or third-party APIs.

Example: Basic Usage of test_start and test_stop

Here’s a basic example demonstrating the usage of these events

from locust import User, task, between, events
from datetime import datetime

# Global setup: Perform actions at test start
@events.test_start.add_listener
def on_test_start(environment, **kwargs):
    print("Test started at:", datetime.now())

# Global teardown: Perform actions at test stop
@events.test_stop.add_listener
def on_test_stop(environment, **kwargs):
    print("Test stopped at:", datetime.now())

# Simulated user behavior
class MyUser(User):
    wait_time = between(1, 5)

    @task
    def print_datetime(self):
        """Task that prints the current datetime."""
        print("Current datetime:", datetime.now())

Running the Example

• Save the code as locustfile.py.
• Start Locust -> `locust -f locustfile.py`
• Configure the test parameters (number of users, spawn rate, etc.) in the web UI at http://localhost:8089.
• Observe the console output:
  • A message when the test starts (on_test_start).
  • Messages during the test as users execute tasks.
  • A message when the test stops (on_test_stop).

Example: Logging Test Details

You can log detailed test information, like the number of users and host under test, using environment and kwargs

from locust import User, task, between, events

@events.test_start.add_listener
def on_test_start(environment, **kwargs):
    print("Test started!")
    print(f"Target host: {environment.host}")
    print(f"Total users: {environment.runner.target_user_count}")

@events.test_stop.add_listener
def on_test_stop(environment, **kwargs):
    print("Test finished!")
    print("Summary:")
    print(f"Requests completed: {environment.stats.total.num_requests}")
    print(f"Failures: {environment.stats.total.num_failures}")

class MyUser(User):
    wait_time = between(1, 5)

    @task
    def dummy_task(self):
        pass

Observing the Results

When you run the above examples

• At Test Start: Look for messages indicating setup actions, like initializing external systems or printing start time.
• During the Test: Observe user tasks being executed.
• At Test Stop: Verify that cleanup actions were executed successfully.

Locust provides two special methods, on_start and on_stop, to handle setup and teardown actions for individual users. These methods allow you to execute specific code when a simulated user starts or stops, making it easier to simulate real-world scenarios like login/logout or initialization tasks.

In this blog, we’ll cover,

1. What on_start and on_stop do.
2. Why they are important.
3. Practical examples of using these methods.
4. Running and testing Locust scripts.

What Are on_start and on_stop?

• on_start: This method is executed once when a new simulated user starts. It’s commonly used for tasks like logging in or setting up the environment.
• on_stop: This method is executed once when a simulated user stops. It’s often used for cleanup tasks like logging out.

These methods are executed only once per user during the lifecycle of a test, as opposed to tasks that are run repeatedly.

Why Use on_start and on_stop?

1. Simulating Real User Behavior: Real users often start a session with an action (e.g., login) and end it with another (e.g., logout).
2. Initial Setup: Some tasks require initializing data or setting up user state before performing other actions.
3. Cleanup: Ensure that actions like logout are performed to leave the system in a clean state.

Examples

Basic Usage of on_start and on_stop

In this example, we just print on start and `on stop` for each user while running a task.

from locust import User, task, between, constant, constant_pacing
from datetime import datetime


class MyUser(User):

    wait_time = between(1, 5)

    def on_start(self):
        print("on start")

    def on_stop(self):
        print("on stop")

    @task
    def print_datetime(self):
        print(datetime.now())

Locust allows you to define multiple user types in your load tests, enabling you to simulate different user behaviors and traffic patterns. This is particularly useful when your application serves diverse client types, such as web and mobile users, each with unique interaction patterns.

In this blog, we will

1. Discuss the concept of multiple user types in Locust.
2. Explore how to implement multiple user classes with weights.
3. Run and analyze the test results.

Why Use Multiple User Types?

In real-world applications, different user groups interact with your system differently. For example,

• Web Users might spend more time browsing through the UI.
• Mobile Users could make faster but more frequent requests.

By simulating distinct user types with varying behaviors, you can identify performance bottlenecks across all client groups.

Understanding User Classes and Weights

Locust provides the ability to define user classes by extending the User or HttpUser base class. Each user class can,

• Have a unique set of tasks.
• Define its own wait times.
• Be assigned a weight, which determines the proportion of that user type in the simulation.

For example, if WebUser has a weight of 1 and MobileUser has a weight of 2, the simulation will spawn 1 web user for every 2 mobile users.

Example: Simulating Web and Mobile Users

Below is an example Locust test with two user types

from locust import User, task, between

# Define a user class for web users
class MyWebUser(User):
    wait_time = between(1, 3)  # Web users wait between 1 and 3 seconds between tasks
    weight = 1  # Web users are less frequent

    @task
    def login_url(self):
        print("I am logging in as a Web User")


# Define a user class for mobile users
class MyMobileUser(User):
    wait_time = between(1, 3)  # Mobile users wait between 1 and 3 seconds
    weight = 2  # Mobile users are more frequent

    @task
    def login_url(self):
        print("I am logging in as a Mobile User")

How Locust Uses Weights

With the above configuration

• For every 3 users spawned, 1 will be a Web User, and 2 will be Mobile Users (based on their weights: 1 and 2).

Locust automatically handles spawning these users in the specified ratio.

Running the Locust Test

1. Save the Code
   Save the above code in a file named locustfile.py.
2. Start Locust
   Open your terminal and run `locust -f locustfile.py`
3. Access the Web UI
   • Open your browser and go to http://localhost:8089.
4. Enter Test Parameters
   • Number of users (e.g., 30).
   • Spawn rate (e.g., 5 users per second).
   • Host: If you are testing an actual API or website, specify its URL (e.g., http://localhost:8000).
5. Analyze Results
   • Observe how Locust spawns the users according to their weights and tracks metrics like request counts and response times.

After running the test:

• Check the distribution of requests to ensure it matches the weight ratio (e.g., for every 1 web user request, there should be ~3 mobile user requests).
• Use the metrics (response time, failure rate) to evaluate performance for each user type.

Locust is an excellent load testing tool, enabling developers to simulate concurrent user traffic on their applications. One of its powerful features is wait times, which simulate the realistic user think time between consecutive tasks. By customizing wait times, you can emulate user behavior more effectively, making your tests reflect actual usage patterns.

In this blog, we’ll cover,

1. What wait times are in Locust.
2. Built-in wait time options.
3. Creating custom wait times.
4. A full example with instructions to run the test.

What Are Wait Times in Locust?

In real-world scenarios, users don’t interact with applications continuously. After performing an action (e.g., submitting a form), they often pause before the next action. This pause is called a wait time in Locust, and it plays a crucial role in mimicking real-life user behavior.

Locust provides several ways to define these wait times within your test scenarios.

FastAPI App Overview

Here’s the FastAPI app that we’ll test,

from fastapi import FastAPI

# Create a FastAPI app instance
app = FastAPI()

# Define a route with a GET method
@app.get("/")
def read_root():
    return {"message": "Welcome to FastAPI!"}

@app.get("/items/{item_id}")
def read_item(item_id: int, q: str = None):
    return {"item_id": item_id, "q": q}

Locust Examples for FastAPI

1. Constant Wait Time Example

Here, we’ll simulate constant pauses between user requests

from locust import HttpUser, task, constant

class FastAPIUser(HttpUser):
    wait_time = constant(2)  # Wait for 2 seconds between requests

    @task
    def get_root(self):
        self.client.get("/")  # Simulates a GET request to the root endpoint

    @task
    def get_item(self):
        self.client.get("/items/42?q=test")  # Simulates a GET request with path and query parameters

2. Between wait time Example

Simulating random pauses between requests.

from locust import HttpUser, task, between

class FastAPIUser(HttpUser):
    wait_time = between(1, 5)  # Random wait time between 1 and 5 seconds

    @task(3)  # Weighted task: this runs 3 times more often
    def get_root(self):
        self.client.get("/")

    @task(1)
    def get_item(self):
        self.client.get("/items/10?q=locust")

3. Custom Wait Time Example

Using a custom wait time function to introduce more complex user behavior

import random
from locust import HttpUser, task

def custom_wait():
    return max(1, random.normalvariate(3, 1))  # Normal distribution (mean: 3s, stddev: 1s)

class FastAPIUser(HttpUser):
    wait_time = custom_wait

    @task
    def get_root(self):
        self.client.get("/")

    @task
    def get_item(self):
        self.client.get("/items/99?q=custom")

Full Test Example

Combining all the above elements, here’s a complete Locust test for your FastAPI app.

from locust import HttpUser, task, between
import random

# Custom wait time function
def custom_wait():
    return max(1, random.uniform(1, 3))  # Random wait time between 1 and 3 seconds

class FastAPIUser(HttpUser):
    wait_time = custom_wait  # Use the custom wait time

    @task(3)
    def browse_homepage(self):
        """Simulates browsing the root endpoint."""
        self.client.get("/")

    @task(1)
    def browse_item(self):
        """Simulates fetching an item with ID and query parameter."""
        item_id = random.randint(1, 100)
        self.client.get(f"/items/{item_id}?q=test")

Running Locust for FastAPI

1. Run Your FastAPI App
   Save the FastAPI app code in a file (e.g., main.py) and start the server

uvicorn main:app --reload

By default, the app will run on http://127.0.0.1:8000.

2. Run Locust
Save the Locust file as locustfile.py and start Locust.

locust -f locustfile.py

3. Configure Locust
Open http://localhost:8089 in your browser and enter:

• Host: http://127.0.0.1:8000
• Number of users and spawn rate based on your testing requirements.

4. Run in Headless Mode (Optional)
Use the following command to run Locust in headless mode

locust -f locustfile.py --headless -u 50 -r 10 --host http://127.0.0.1:8000`

-u 50: Simulate 50 users.

-r 10: Spawn 10 users per second.