PRNG Benchmarks

README.md

In response to some people claiming that using a CSPRNG is "going way overboard" and/or is "overkill", I've written this test to verify the performance impact of using a CSPRNG versus their insecure mt_rand() based hacks.

I think the results are conclusive (at least on my device): A 50% speed increase. In addition to less-predictable randomness.

If anyone would like to suggest a benchmark script (or conditions that lead to different results with mine), let me know and I will link to them here.

functions.php

<?php
function shitty_prng($bytes = 32)
{
   $buf = '';
   for ($i = 0; $i < $bytes; ++$i) {
      $buf .= chr(mt_rand(0, 255));
   }
}

function better_prng($bytes = 32)
{
   if (function_exists('mcrypt_create_iv')) {
      return mcrypt_create_iv(32, MCRYPT_DEV_URANDOM);
   }
   return openssl_random_pseudo_bytes(32);
}

function openssl_prng($bytes = 32)
{
   return openssl_random_pseudo_bytes(32);
}
function mcrypt_prng($bytes = 32)
{
   return mcrypt_create_iv(32, MCRYPT_DEV_URANDOM);
}

test.php

<?php
require "functions.php";

$buf = '';
$tests = [];
$start = microtime(true);
for ($i = 0; $i < 100000; ++$i) {
        $buf = shitty_prng();
}
$tests['mtrand'] = ( microtime(true) - $start ); 

$start = microtime(true);
for ($i = 0; $i < 100000; ++$i) {
        $buf = better_prng();
}
$tests['csprng'] = ( microtime(true) - $start ); 

$start = microtime(true);
for ($i = 0; $i < 100000; ++$i) {
        $buf = openssl_prng();
}
$tests['openssl'] = ( microtime(true) - $start ); 

$start = microtime(true);
for ($i = 0; $i < 100000; ++$i) {
        $buf = mcrypt_prng();
}
$tests['mcrypt'] = ( microtime(true) - $start ); 

var_dump($tests);

test_results

array(4) {
  ["mtrand"]=>
  float(2.3792960643768)
  ["csprng"]=>
  float(1.0584290027618)
  ["openssl"]=>
  float(0.38547611236572)
  ["mcrypt"]=>
  float(0.97102904319763)
}
array(4) {
  ["mtrand"]=>
  float(2.4055750370026)
  ["csprng"]=>
  float(1.0631558895111)
  ["openssl"]=>
  float(0.30554485321045)
  ["mcrypt"]=>
  float(1.106586933136)
}
array(4) {
  ["mtrand"]=>
  float(2.3207230567932)
  ["csprng"]=>
  float(1.0591180324554)
  ["openssl"]=>
  float(0.29997992515564)
  ["mcrypt"]=>
  float(1.0387818813324)
}
array(4) {
  ["mtrand"]=>
  float(2.3104860782623)
  ["csprng"]=>
  float(1.1197648048401)
  ["openssl"]=>
  float(0.2982759475708)
  ["mcrypt"]=>
  float(1.0270299911499)
}

scott@resonantcorenet ~/bench $ php -v
PHP 5.6.6-1~dotdeb.1 (cli) (built: Feb 20 2015 19:57:36) 
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2015 Zend Technologies
    with Zend OPcache v7.0.4-dev, Copyright (c) 1999-2015, by Zend Technologies

That's the machine I ran this on.

I ran this through my simple PHP Benchmark library and got these results on PHP 5.5.20.

openssl_prng = 225800 iterations per second (Fastest)
mcrypt_prng = 107595 iterations per second
better_prng = 99619 iterations per second
shitty_prng = 51612 iterations per second (Slowest)

You can re-run the benchmark and see the comparison on my site.

Hey @sarciszewski, I only just saw this. I am co-author of the CSPRNG-in-core RFC for PHP 7.

https://github.com/lt/php-src/tree/rand-bytes

On a vanilla Linux install the random_bytes() function is comparable in speed to OpenSSL. OpenSSL still has the edge because (by default) it doesn't read directly from the systems sources of random all the time, it maintains it's own random pool and does a bunch of "stuff" for FIPS compliance, so it has an in-memory buffer it can draw on which gives it a bit more speed.

On FreeBSD and OpenBSD it absolutely blows OpenSSL out of the water.

Also worth noting, if you install and build against LibreSSL-portable on Linux, you're also going to get much faster speeds than regular OpenSSL due to it providing it's own arc4random functions.

Edit:

Also, mcrypt_create_iv() does not depend on the mcrypt library. It's just a wrapper around CryptGenRandom on Windows and /dev/*random on Linux. A lot of the performance hit is because it opens/closes the file descriptor for every call.

Edit 2:

I refuse to sign up to reddit. If you have any questions about how (CS)RNGs function in PHP, or why things are slower/faster, ping me here, come have a chat on SO (http://chat.stackoverflow.com/rooms/11/php), or fire an email to leight -> gmail

Hi @lt - just saw your message! Sorry about that. I'll jump into the chatroom.

Just tested on PHP7 minus mcrypt for reasons of it not being in my docker for PHP7 (I think it's deprecated)

array(3) {
    ["mtrand"]=> float(0.54768705368042) 
    ["csprng"]=> float(0.20757412910461) 
    ["openssl"]=> float(0.19972586631775) 
}

I ran multiple times, looks like the numbers are pretty solid, very little difference very old quad core inside docker environment

Try again with random_bytes() instead of mcrypt:

function shitty_prng($bytes = 32)
{
   $buf = '';
   for ($i = 0; $i < $bytes; ++$i) {
      $buf .= chr(mt_rand(0, 255));
   }
}
function better_prng($bytes = 32)
{
   if (function_exists('mcrypt_create_iv')) {
      return mcrypt_create_iv($bytes, MCRYPT_DEV_URANDOM);
   }
   return openssl_random_pseudo_bytes($bytes);
}
function openssl_prng($bytes = 32)
{
   return openssl_random_pseudo_bytes($bytes);
}
function best_prng($bytes = 32)
{
   return random_bytes($bytes);
}

$buf = '';
$tests = [];
$start = microtime(true);
for ($i = 0; $i < 100000; ++$i) {
        $buf = shitty_prng();
}
$tests['mtrand'] = ( microtime(true) - $start ); 
$start = microtime(true);
for ($i = 0; $i < 100000; ++$i) {
        $buf = better_prng();
}
$tests['csprng'] = ( microtime(true) - $start ); 
$start = microtime(true);
for ($i = 0; $i < 100000; ++$i) {
        $buf = openssl_prng();
}
$tests['openssl'] = ( microtime(true) - $start ); 
$start = microtime(true);
for ($i = 0; $i < 100000; ++$i) {
        $buf = best_prng();
}
$tests['best'] = ( microtime(true) - $start ); 
var_dump($tests);

Four runs:

root@debian:~# php bench.php 
array(4) {
  ["mtrand"]=>
  float(0.48157906532288)
  ["csprng"]=>
  float(0.30486798286438)
  ["openssl"]=>
  float(0.15188217163086)
  ["best"]=>
  float(0.28898501396179)
}
array(4) {
  ["mtrand"]=>
  float(0.4787449836731)
  ["csprng"]=>
  float(0.29729914665222)
  ["openssl"]=>
  float(0.13547110557556)
  ["best"]=>
  float(0.28169298171997)
}
array(4) {
  ["mtrand"]=>
  float(0.4635169506073)
  ["csprng"]=>
  float(0.3045289516449)
  ["openssl"]=>
  float(0.13414788246155)
  ["best"]=>
  float(0.28961706161499)
}
array(4) {
  ["mtrand"]=>
  float(0.45156908035278)
  ["csprng"]=>
  float(0.30080914497375)
  ["openssl"]=>
  float(0.13886404037476)
  ["best"]=>
  float(0.29005599021912)
}

https://3v4l.org/LYfu9