Last active
August 29, 2015 14:26
-
-
Save chadladensack/cfb786effa7a9aed663c to your computer and use it in GitHub Desktop.
A tool for benchmarking the Aerospike PHP client for setting multiple records with a single bin and value.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<?php | |
$options = getopt('h:p:n:s:t:r:f:'); | |
$host = isset($options['h']) ? $options['h'] : '127.0.0.1'; | |
$port = isset($options['p']) ? $options['p'] : 3000; | |
$namespace = isset($options['n']) ? $options['n'] : 'test'; | |
$set = isset($options['s']) ? $options['s'] : 'testing'; | |
$ttl = isset($options['t']) ? $options['t'] : 120; | |
$records = isset($options['r']) ? $options['r'] : 10000; | |
$prefix = isset($options['f']) ? $options['f'] : ''; | |
$config = array( | |
'hosts' => array( | |
array( | |
'addr' => $host, | |
'port' => $port, | |
), | |
), | |
); | |
$aerospike = new Aerospike($config); | |
if (!$aerospike->isConnected()) { | |
echo "Failed to connect to the Aerospike server [{$aerospike->errorno()}]: {$aerospike->error()}\n"; | |
exit(1); | |
} | |
for ($i = 0; $i < $records; $i++) { | |
$primary_key = sprintf('test_%s_%s', $prefix, $i); | |
$key = $aerospike->initKey($namespace, $set, $primary_key); | |
$status = $aerospike->put($key, array('test_bin' => 'test_value'), $ttl); | |
if ($status != Aerospike::OK) { | |
echo "Failed to put data [{$aerospike->errorno()}]: {$aerospike->error()}\n"; | |
exit(1); | |
} | |
} | |
$aerospike->close(); |
@rbotzer Thanks! I've reviewed the performance examples and they match the same execution times we're seeing using the aerospike_benchmark.php gist.
Running 3 processes in parallel using GUI Parallel shows the load distributed among the processes. However, we are planning on calling a script like aerospike_benchmark.php over 5,000 times. Each execution would focus on a specific datapoint and insert/update around 50,000 records distinct to a datapoint, resulting in a total of 250,000,000 records being inserted/updated. Having 5,000 cores available isn't realistic so we're exploring different options at this time, LDT's being one of them.
Test Parallel Execution
$ time seq 3 | parallel /usr/bin/php aerospike_benchmark.php -h localhost -p 3000 -n test -s testing -r 50000 -f {}
real 0m42.768s
user 0m25.890s
sys 0m4.724s
Example Desired Parallel Execution
$ seq 5000 | parallel -j80% /usr/bin/php aerospike_benchmark.php -h localhost -p 3000 -n test -s testing -r 50000 -f {}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Nice work. Check out the performance examples in the aerospike/aerospike-client-php repo. There's a limit of what you can push through a single PHP process, so I recommend wrapping your script in something like the
rw-concurrent.sh
script, as I'm using there. Run the tests again in parallel if you have more cores to spare.