Skip to content

Instantly share code, notes, and snippets.

@dolph dolph/auth.json
Last active Jun 7, 2018

Embed
What would you like to do?
Keystone performance benchmarking
{
"auth": {
"identity": {
"methods": [
"password"
],
"password": {
"user": {
"domain": {
"id": "default"
},
"password": "password",
"name": "admin"
}
}
},
"scope": {
"project": {
"domain": {
"id": "default"
},
"name": "admin"
}
}
}
}
import os
from keystoneclient.v3 import client
HOST = os.environ.get('HOST', '23.253.157.126')
KEYSTONE_ENDPOINT = os.environ.get(
'KEYSTONE_ENDPOINT', 'http://%s:35357/' % HOST)
project_scoped = client.Client(
username='admin',
password='password',
project_name='admin',
auth_url=KEYSTONE_ENDPOINT + 'v3')
print('%s' % project_scoped.auth_token)
set -e
export HOST=23.253.157.242
export DB_HOST=23.253.157.126
echo "Ensuring Keystone is bootstrapped with the expected credentials..."
python bootstrap.py
echo "Truncating the keystone.token table..."
ssh root@$DB_HOST "mysql keystone -e 'truncate token;'"
echo "Creating a token to run benchmarks with..."
ADMIN_TOKEN=`python authenticate.py`
SUBJECT_TOKEN=`python authenticate.py`
echo "Admin token: $ADMIN_TOKEN"
echo "Subject token: $SUBJECT_TOKEN"
echo "Warming up Apache..."
ab -c 100 -n 1000 -T 'application/json' http://$HOST:35357/ > /dev/null 2>&1
echo "Benchmarking token creation..."
ab -r -c 1 -n 200 -p auth.json -T 'application/json' http://$HOST:35357/v3/auth/tokens > create_token
if ! grep -q 'Failed requests: 0' create_token; then
echo 'WARNING: Failed requests!'
fi
git diff --color create_token | grep --fixed-strings ' [ms] (mean)' || true
echo "Benchmarking token validation..."
ab -r -c 1 -n 10000 -T 'application/json' -H "X-Auth-Token: $ADMIN_TOKEN\n" -H "X-Subject-Token: $SUBJECT_TOKEN\n" http://$HOST:35357/v3/auth/tokens > validate_token
if ! grep -q 'Failed requests: 0' validate_token; then
echo 'WARNING: Failed requests!'
fi
git diff --color validate_token | grep --fixed-strings ' [ms] (mean)' || true
echo "Benchmarking token creation concurrently..."
ab -r -c 100 -n 2000 -p auth.json -T 'application/json' http://$HOST:35357/v3/auth/tokens > create_token_concurrent
if ! grep -q 'Failed requests: 0' create_token_concurrent; then
echo 'WARNING: Failed requests!'
fi
git diff --color create_token_concurrent | grep --fixed-strings ' [#/sec] (mean)' || true
echo "Benchmarking token validation concurrency..."
ab -r -c 100 -n 100000 -T 'application/json' -H "X-Auth-Token: $ADMIN_TOKEN" -H "X-Subject-Token: $SUBJECT_TOKEN" http://$HOST:35357/v3/auth/tokens > validate_token_concurrent
if ! grep -q 'Failed requests: 0' validate_token_concurrent; then
echo 'WARNING: Failed requests!'
fi
git diff --color validate_token_concurrent | grep --fixed-strings ' [#/sec] (mean)' || true
import os
from keystoneclient.v3 import client
HOST = os.environ.get('HOST', '192.168.111.222')
KEYSTONE_ENDPOINT = os.environ.get(
'KEYSTONE_ENDPOINT', 'http://%s:35357/' % HOST)
c = client.Client(
token='ADMIN',
endpoint=KEYSTONE_ENDPOINT + 'v3')
domain = c.domains.get('default')
try:
role = c.roles.create(name='admin')
except Exception:
role = c.roles.list(name='admin')[0]
try:
group = c.groups.create(domain=domain, name='admin')
except Exception:
group = c.groups.list(domain=domain, name='admin')[0]
c.roles.grant(group=group, domain=domain, role=role)
try:
project = c.projects.create(domain=domain, name='admin')
except Exception:
project = c.projects.list(domain=domain, name='admin')[0]
c.roles.grant(group=group, project=project, role=role)
password = 'password'
try:
user = c.users.create(
domain=domain, name='admin', password=password)
except Exception:
user = c.users.list(domain=domain, name='admin')[0]
c.users.add_to_group(user=user, group=group)
services = c.services.list(name='Keystone', type='identity')
if services:
service = services[0]
else:
service = c.services.create(
name='Keystone', type='identity')
endpoints = c.endpoints.list(service=service)
public_endpoints = [e for e in endpoints if e.interface == 'public']
admin_endpoints = [e for e in endpoints if e.interface == 'admin']
if not public_endpoints:
public_endpoint = c.endpoints.create(
service=service,
interface='public',
url=KEYSTONE_ENDPOINT + 'v3')
if not admin_endpoints:
admin_endpoint = c.endpoints.create(
service=service,
interface='admin',
url=KEYSTONE_ENDPOINT + 'v3')
unscoped = client.Client(
username='admin',
password='password',
auth_url=KEYSTONE_ENDPOINT + 'v3')
print('Unscoped token: %s' % unscoped.auth_token)
project_scoped = client.Client(
username='admin',
password='password',
project_name='admin',
auth_url=KEYSTONE_ENDPOINT + 'v3')
print('Project-scoped token: %s' % project_scoped.auth_token)
domain_scoped = client.Client(
username='admin',
password='password',
domain_name='Default',
auth_url=KEYSTONE_ENDPOINT + 'v3')
print('Domain-scoped token: %s' % domain_scoped.auth_token)
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 23.253.157.242 (be patient)
Server Software: Apache/2.2.22
Server Hostname: 23.253.157.242
Server Port: 35357
Document Path: /v3/auth/tokens
Document Length: 886 bytes
Concurrency Level: 1
Time taken for tests: 14.596 seconds
Complete requests: 200
Failed requests: 0
Write errors: 0
Total transferred: 553836 bytes
Total POSTed: 123400
HTML transferred: 177200 bytes
Requests per second: 13.70 [#/sec] (mean)
Time per request: 72.981 [ms] (mean)
Time per request: 72.981 [ms] (mean, across all concurrent requests)
Transfer rate: 37.05 [Kbytes/sec] received
8.26 kb/s sent
45.31 kb/s total
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 62 73 19.4 70 284
Waiting: 59 70 19.4 66 280
Total: 62 73 19.4 70 284
Percentage of the requests served within a certain time (ms)
50% 70
66% 71
75% 71
80% 72
90% 80
95% 82
98% 123
99% 214
100% 284 (longest request)
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 23.253.157.242 (be patient)
Server Software: Apache/2.2.22
Server Hostname: 23.253.157.242
Server Port: 35357
Document Path: /v3/auth/tokens
Document Length: 886 bytes
Concurrency Level: 100
Time taken for tests: 11.235 seconds
Complete requests: 2000
Failed requests: 0
Write errors: 0
Total transferred: 5538728 bytes
Total POSTed: 1234000
HTML transferred: 1772000 bytes
Requests per second: 178.02 [#/sec] (mean)
Time per request: 561.726 [ms] (mean)
Time per request: 5.617 [ms] (mean, across all concurrent requests)
Transfer rate: 481.46 [Kbytes/sec] received
107.27 kb/s sent
588.72 kb/s total
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.1 0 1
Processing: 133 555 278.6 477 1862
Waiting: 125 542 274.6 463 1852
Total: 134 555 278.6 477 1862
Percentage of the requests served within a certain time (ms)
50% 477
66% 596
75% 707
80% 771
90% 928
95% 1107
98% 1328
99% 1465
100% 1862 (longest request)
python-keystoneclient
This is ApacheBench, Version 2.3 <$Revision$>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 23.253.157.242 (be patient)
Server Software: Apache/2.2.22
Server Hostname: 23.253.157.242
Server Port: 35357
Document Path: /v3/auth/tokens
Document Length: 886 bytes
Concurrency Level: 1
Time taken for tests: 62.751 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Total transferred: 27620000 bytes
HTML transferred: 8860000 bytes
Requests per second: 159.36 [#/sec] (mean)
Time per request: 6.275 [ms] (mean)
Time per request: 6.275 [ms] (mean, across all concurrent requests)
Transfer rate: 429.84 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 5 6 0.6 6 39
Waiting: 4 5 0.6 5 38
Total: 5 6 0.6 6 39
Percentage of the requests served within a certain time (ms)
50% 6
66% 6
75% 6
80% 6
90% 7
95% 7
98% 7
99% 7
100% 39 (longest request)
This is ApacheBench, Version 2.3 <$Revision$>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 23.253.157.242 (be patient)
Server Software: Apache/2.2.22
Server Hostname: 23.253.157.242
Server Port: 35357
Document Path: /v3/auth/tokens
Document Length: 886 bytes
Concurrency Level: 100
Time taken for tests: 59.193 seconds
Complete requests: 100000
Failed requests: 0
Write errors: 0
Total transferred: 276200000 bytes
HTML transferred: 88600000 bytes
Requests per second: 1689.40 [#/sec] (mean)
Time per request: 59.193 [ms] (mean)
Time per request: 0.592 [ms] (mean, across all concurrent requests)
Transfer rate: 4556.75 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 1
Processing: 6 59 79.7 17 470
Waiting: 4 55 79.4 14 469
Total: 6 59 79.7 17 470
Percentage of the requests served within a certain time (ms)
50% 17
66% 30
75% 56
80% 120
90% 212
95% 230
98% 252
99% 267
100% 470 (longest request)
@sxc731

This comment has been minimized.

Copy link

commented Jan 8, 2018

Hi there, thanks for the v. useful benchmarking code!

Just one q: I've been testing this on an openstack-ansible-deployed Pike environment and Keystone was rejecting all my tokens validation requests ('Invalid user token') until I removed the linefeeds from the request headers (eg from "X-Auth-Token: $ADMIN_TOKEN\n"). Is this an indication of a config issue? I'm trying to understand cause of subpar Keystone validation performance (~500ms single-threaded on lightly loaded metal controllers).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.