Dockerized JMeter - A Distributed Load Testing Workflow

1_Dockerised_JMeter.md

Dockerized JMeter - A Distributed Load Testing Workflow

Supported Cloud Providers:

• Amazon (https://aws.amazon.com/free/)
• DigitalOcean (https://www.digitalocean.com/join/)

Images used:

• JMeter Server (https://hub.docker.com/r/hhcordero/docker-jmeter-server)
• JMeter Client Non-gui (https://hub.docker.com/r/hhcordero/docker-jmeter-client)

Pre-requisites:

1. Docker and Docker-Machine cli installed on your host (https://docs.docker.com/installation/)

2. You can choose either Amazon or DigitalOcean

   Amazon:

   • Access Key
   • Secret Key
   • Region
   • VPC ID
   • Subnet ID
   • Zone
   • AMI

   DigitalOcean:

   • Access Token

3. JMeter test plan on your host (http://jmeter.apache.org/usermanual/build-web-test-plan.html)

Steps:

1. Provision machine for JMeter (Client - Non-gui mode)

   For AWS:

   $ ./launch_jmeter_client_aws
   

   For DigitalOcean:

   $ ./launch_jmeter_client_do
   

2. Provision machine for JMeter (Server mode)

   Optional parameter - indicate number of servers to provision, value between 1 to 20. Default to 1 no parameter is given.

   For AWS:

   $ ./launch_jmeter_server_aws 2
   

   For DigitalOcean:

   $ ./launch_jmeter_server_do 2
   

3. Copy test plan (jmx file) to jmeter-client machine inside /load_tests

   Connect to jmeter-client machine and create /load_tests directory

   $ eval "$(docker-machine env --swarm jmeter-client)"
   $ docker-machine ssh jmeter-client
   

   For AWS:

   $ sudo mkdir /load_tests && sudo chown ubuntu:ubuntu /load_tests
   

   For DigitalOcean:

   $ mkdir /load_tests
   

   Then go back to host

   $ exit
   

   Copy test plan from host to jmeter-client machine

   Syntax:

   docker-machine scp [path to test directory] [machine name]:/load_tests
   

   Example:

   $ docker-machine scp -r /home/user/docker-jmeter-client/load_tests/my_test jmeter-client:/load_tests
   

4. Run JMeter load test - execute the following commands, replace values if necessary before executing

   Get JMeter client machine ip

   $ IP=$(docker-machine ip jmeter-client)
   

   Replace the value with the output from ./launch_jmeter_server - must be comma separated jmeter server ip addresses

   $ REMOTE_HOSTS=""
   

   Parent directory for the test plan

   $ TEST_DIR="my_test"
   

   Test plan without file extension

   $ TEST_PLAN="test-plan"
   

   Run the JMeter client non-gui and perform load test

   $ docker run \
       --detach \
       --publish 1099:1099 \
       --volume /load_tests/$TEST_DIR:/load_tests/$TEST_DIR \
       --env TEST_DIR=$TEST_DIR \
       --env TEST_PLAN=$TEST_PLAN \
       --env IP=$IP \
       --env REMOTE_HOSTS=$REMOTE_HOSTS \
       --env constraint:type==client \
       hhcordero/docker-jmeter-client
   

   To monitor output, follow the logs:

   Syntax:

   docker logs -f [container name]
   

   Example:

   $ docker logs -f jmeter-client/tender_feynman
   

5. Save the result, look for the .jtl file inside jmeter-client /load_tests/$TEST_DIR

   Syntax:

   docker-machine scp [machine name]:/load_tests/[test dir]/[test plan result] [path to test directory]
   

   Example:

   $ docker-machine scp jmeter-client:/load_tests/${TEST_DIR}/${TEST_PLAN}.jtl /home/user/docker-jmeter-client/load_tests/my_test/.
   

2_app_aws.env

#!/bin/bash
export TOKEN=[auto-populate by running launch_jmeter_client_aws]
export AWS_ACCESS_KEY_ID=[access key here]
export AWS_SECRET_ACCESS_KEY=[secret key here]
export AWS_DEFAULT_REGION=[region here ie. ap-southeast-1]
export AWS_VPC_ID=[vpc id here ie. vpc-xxxxxxxx]
export AWS_SUBNET_ID=[subnet id here ie. subnet-xxxxxxxx]
export AWS_ZONE=[zone here ie. a]
export AWS_AMI=[ami id here ie. ami-b899a2ea]

2_launch_jmeter_client_aws

#!/bin/bash

# Generate token - use swarm for service discovery
TOKEN=$(docker run --rm swarm create)

sed -i "/export TOKEN=/c\export TOKEN=$TOKEN" ./app_aws.env

# Initialise env variables
source ./app_aws.env

# Provision new machine
echo -e "
Provision machine for JMeter (Client):
docker-machine --debug create
    --driver amazonec2
    --amazonec2-access-key $AWS_ACCESS_KEY_ID
    --amazonec2-secret-key $AWS_SECRET_ACCESS_KEY
    --amazonec2-region $AWS_DEFAULT_REGION
    --amazonec2-vpc-id $AWS_VPC_ID
    --amazonec2-subnet-id $AWS_SUBNET_ID
    --amazonec2-zone $AWS_ZONE
    --amazonec2-ami $AWS_AMI
    --engine-label type=client
    --swarm
    --swarm-master
    --swarm-discovery token://$TOKEN
    jmeter-client"

docker-machine create \
    --driver amazonec2 \
    --amazonec2-access-key $AWS_ACCESS_KEY_ID \
    --amazonec2-secret-key $AWS_SECRET_ACCESS_KEY \
    --amazonec2-region $AWS_DEFAULT_REGION \
    --amazonec2-vpc-id $AWS_VPC_ID \
    --amazonec2-subnet-id $AWS_SUBNET_ID \
    --amazonec2-zone $AWS_ZONE \
    --amazonec2-ami $AWS_AMI \
    --engine-label type=client \
    --swarm \
    --swarm-master \
    --swarm-discovery token://$TOKEN \
    jmeter-client

echo "To manage swarm, make sure to run: eval \"\$(docker-machine env --swarm jmeter-client)\""
exit

2_launch_jmeter_server_aws

#!/bin/bash
# Default to 1 if no parameter is given
count=${1-1}

if ! [[ $count =~ ^[0-9]+$ ]] || ! (( count > 0)); then
    echo "[ERROR] Parameter is not a positive integer (must be 1 to 20)." >&2; exit 1
fi

# Max of 10 machines only, change accordingly
max=10

if [ "$count" -gt "$max" ]; then
    count=$max
fi

# Initialise env variables
source ./app_aws.env

iterator=1

while [ "$iterator" -le "$count" ]; do

    # Provision new machine
    echo -e "
    Provision machine for JMeter (Server):
    docker-machine create
        --driver amazonec2
        --amazonec2-access-key $AWS_ACCESS_KEY_ID
        --amazonec2-secret-key $AWS_SECRET_ACCESS_KEY
        --amazonec2-region $AWS_DEFAULT_REGION
        --amazonec2-vpc-id $AWS_VPC_ID
        --amazonec2-subnet-id $AWS_SUBNET_ID
        --amazonec2-zone $AWS_ZONE
        --amazonec2-ami $AWS_AMI
        --engine-label type=server
        --swarm
        --swarm-discovery token://$TOKEN
        jmeter-server-$iterator"
    
    docker-machine create \
        --driver amazonec2 \
        --amazonec2-access-key $AWS_ACCESS_KEY_ID \
        --amazonec2-secret-key $AWS_SECRET_ACCESS_KEY \
        --amazonec2-region $AWS_DEFAULT_REGION \
        --amazonec2-vpc-id $AWS_VPC_ID \
        --amazonec2-subnet-id $AWS_SUBNET_ID \
        --amazonec2-zone $AWS_ZONE \
        --amazonec2-ami $AWS_AMI \
        --engine-label type=server \
        --swarm \
        --swarm-discovery token://$TOKEN \
        jmeter-server-$iterator

    # Set env variables to point to newly created machine
    eval "$(docker-machine env jmeter-server-$iterator)"

    # Get public ip
    ip=$(docker-machine ip jmeter-server-$iterator)

    # Run jmeter server on new machine
    echo -e "
    Run JMeter in Server Mode:
    docker run
        --detach
        --publish 1099:1099
        --env IP=$ip
        hhcordero/docker-jmeter-server"

    docker run \
        --detach \
        --publish 1099:1099 \
        --env IP=$ip \
        hhcordero/docker-jmeter-server

    # Concatenate server ip addresses
    server_ips+="$ip,"

    let "iterator += 1"
done

echo "Server IP's, for use in JMeter Client: $(echo $server_ips | sed 's/,*$//')"
exit

3_app_do.env

#!/bin/bash
export TOKEN=[auto-populate by running launch_jmeter_client_do]
export DIGITALOCEAN_ACCESS_TOKEN=[access token here]
export DIGITALOCEAN_REGION=[region here]
export DIGITALOCEAN_SIZE=[memory here]

3_launch_jmeter_client_do

#!/bin/bash

# Generate token - use swarm for service discovery
TOKEN=$(docker run --rm swarm create)

sed -i "/export TOKEN=/c\export TOKEN=$TOKEN" ./app_do.env

# Initialise env variables
source ./app_do.env

# Provision new machine
echo -e "
Provision machine for JMeter (Client):
docker-machine create
    --driver digitalocean
    --digitalocean-access-token $DIGITALOCEAN_ACCESS_TOKEN
    --digitalocean-region $DIGITALOCEAN_REGION
    --digitalocean-size $DIGITALOCEAN_SIZE
    --engine-label type=client
    --swarm
    --swarm-master
    --swarm-discovery token://$TOKEN
    jmeter-client"

docker-machine create \
    --driver digitalocean \
    --digitalocean-access-token $DIGITALOCEAN_ACCESS_TOKEN \
    --digitalocean-region $DIGITALOCEAN_REGION \
    --digitalocean-size $DIGITALOCEAN_SIZE \
    --engine-label type=client \
    --swarm \
    --swarm-master \
    --swarm-discovery token://$TOKEN \
    jmeter-client

echo "To manage swarm, make sure to run: eval \"\$(docker-machine env --swarm jmeter-client)\""
exit

3_launch_jmeter_server_do

#!/bin/bash
# Default to 1 if no parameter is given
count=${1-1}
        
if ! [[ $count =~ ^[0-9]+$ ]] || ! (( count > 0)); then
    echo "[ERROR] Parameter is not a positive integer (must be 1 to 20)." >&2; exit 1
fi      

# Max of 10 machines only, change accordingly
max=10

if [ "$count" -gt "$max" ]; then
    count=$max
fi  
    
# Initialise env variables
source ./app_do.env
    
iterator=1
        
while [ "$iterator" -le "$count" ]; do
        
    # Provision new machine
    echo -e "
    Provision machine for JMeter (Server):
    docker-machine create 
        --driver digitalocean
        --digitalocean-access-token $DIGITALOCEAN_ACCESS_TOKEN
        --digitalocean-region $DIGITALOCEAN_REGION
        --digitalocean-size $DIGITALOCEAN_SIZE
        --engine-label type=server
        --swarm
        --swarm-discovery token://$TOKEN
        jmeter-server-$iterator"
        
    docker-machine create \
        --driver digitalocean \
        --digitalocean-access-token $DIGITALOCEAN_ACCESS_TOKEN \
        --digitalocean-region $DIGITALOCEAN_REGION \
        --digitalocean-size $DIGITALOCEAN_SIZE \
        --engine-label type=server \
        --swarm \
        --swarm-discovery token://$TOKEN \
        jmeter-server-$iterator

    # Set env variables to point to newly created machine
    eval "$(docker-machine env jmeter-server-$iterator)"

    # Get public ip
    ip=$(docker-machine ip jmeter-server-$iterator)

    # Run jmeter server on new machine
    echo -e "
    Run JMeter in Server Mode:
    docker run
        --detach
        --publish 1099:1099
        --env IP=$ip
        hhcordero/docker-jmeter-server"

    docker run \
        --detach \
        --publish 1099:1099 \
        --env IP=$ip \
        hhcordero/docker-jmeter-server

    # Concatenate server ip addresses
    server_ips+="$ip,"

    let "iterator += 1"
done

echo "Server IP's, for use in JMeter Client: $(echo $server_ips | sed 's/,*$//')"
exit