bench-queue-stuff.py

#!/usr/bin/env python

import benchmark
import Queue
import threading
import os


class BenchQueueStuff(benchmark.Benchmark):
    def setUp(self):
        self.size = 10000
        self.queue = Queue.Queue()

    def test_big_batch(self):
        for i in xrange(self.size):
            self.queue.put(i)
        for _ in xrange(self.size):
            # should never block, but shit happens
            self.queue.get(block=False)
            
    def test_one_at_a_time(self):
        for i in xrange(self.size):
            self.queue.put(i)
            self.queue.get()
            
    def test_producer_consumer_thread(self):
        def consume():
            for _ in xrange(self.size):
                self.queue.get()

        thr = threading.Thread(target=consume)
        thr.daemon = True
        thr.start()
        for i in xrange(self.size):
            self.queue.put(i)
        thr.join()

    def test_listdir(self):
        cwd = os.getcwd()
        for i in xrange(self.size):
            os.listdir(cwd)


if __name__ == '__main__':
    benchmark.main(each=50, numberFormat="%.4g")

benchmark results

Benchmark Report
================

BenchQueueStuff
---------------

                    name | rank | runs |    mean |        sd | timesBaseline
-------------------------|------|------|---------|-----------|--------------
               big batch |    1 |   50 | 0.04007 |   0.00197 |           1.0
           one at a time |    2 |   50 |  0.0401 | 0.0009213 |  1.0008963986
producer consumer thread |    3 |   50 | 0.04223 |  0.004609 | 1.05394517735
                 listdir |    4 |   50 |  0.7116 |    0.0138 | 17.7617335149

Each of the above 200 runs were run in random, non-consecutive order by
`benchmark` v0.1.5 (http://jspi.es/benchmark) with Python 2.7.3
Linux-3.2.0-53-generic-x86_64 on 2013-09-12 05:52:39.

Benchmark Report
================

BenchQueueStuff
---------------

                    name | rank | runs |    mean |       sd | timesBaseline
-------------------------|------|------|---------|----------|--------------
               big batch |    1 |   50 | 0.04884 | 0.003528 |           1.0
           one at a time |    2 |   50 | 0.05044 | 0.004653 | 1.03279261202
producer consumer thread |    3 |   50 |  0.1442 |  0.06213 | 2.95245249927
                 listdir |    4 |   50 |   0.153 | 0.005393 | 3.13372278754

Each of the above 200 runs were run in random, non-consecutive order by
`benchmark` v0.1.5 (http://jspi.es/benchmark) with Python 2.7.3
Linux-3.10.10-100.fc18.x86_64-x86_64 on 2013-09-19 12:10:30.

The previous above run lead to ... https://gist.github.com/portante/6623371