Skip to content

Instantly share code, notes, and snippets.

@reconbot

reconbot/blperftest.js

Last active Oct 9, 2018
Embed
What would you like to do?
perf test indexof
const Benchmark = require('benchmark')
const Bl = require('./')
const smallBuffers = new Bl(new Array(100).fill(1).map(() => Buffer.alloc(8, 1)))
const largeBuffers = new Bl(new Array(100).fill(1).map(() => Buffer.alloc(1024, 1)))
const mixedBuffers = new Bl(new Array(100).fill(1).map((val, i) => Buffer.alloc(i ** 2 % 1024, 1)))
const veryLargeBuffers = new Bl(new Array(100).fill(1).map(() => Buffer.alloc(1024 * 1024, 1)))
const notFindableSmall = Buffer.alloc(7, 0)
const notFindableMedium = Buffer.alloc(510, 0)
const notFindableLarge = Buffer.alloc(1025, 0)
const suite = new Benchmark.Suite()
suite.add('smallBuffers small search', () => {
smallBuffers.indexOf(notFindableSmall)
})
suite.add('smallBuffers medium search', () => {
smallBuffers.indexOf(notFindableMedium)
})
suite.add('smallBuffers large search', () => {
smallBuffers.indexOf(notFindableLarge)
})
suite.add('largeBuffers small search', () => {
largeBuffers.indexOf(notFindableSmall)
})
suite.add('largeBuffers medium search', () => {
largeBuffers.indexOf(notFindableMedium)
})
suite.add('largeBuffers large search', () => {
largeBuffers.indexOf(notFindableLarge)
})
suite.add('mixedBuffers small search', () => {
mixedBuffers.indexOf(notFindableSmall)
})
suite.add('mixedBuffers medium search', () => {
mixedBuffers.indexOf(notFindableMedium)
})
suite.add('mixedBuffers large search', () => {
mixedBuffers.indexOf(notFindableLarge)
})
suite.add('veryLargeBuffers small search', () => {
veryLargeBuffers.indexOf(notFindableSmall)
})
suite.add('veryLargeBuffers medium search', () => {
veryLargeBuffers.indexOf(notFindableMedium)
})
suite.add('veryLargeBuffers large search', () => {
veryLargeBuffers.indexOf(notFindableLarge)
})
suite.on('cycle', function(event) {
console.log(String(event.target));
})
.on('complete', function() {
console.log('done');
})
.run()
smallBuffers small search x 2,640,068 ops/sec ±0.45% (91 runs sampled)
smallBuffers medium search x 2,519,050 ops/sec ±3.31% (84 runs sampled)
smallBuffers large search x 2,694,251 ops/sec ±0.66% (87 runs sampled)
largeBuffers small search x 2,621,720 ops/sec ±3.07% (88 runs sampled)
largeBuffers medium search x 2,694,856 ops/sec ±0.50% (84 runs sampled)
largeBuffers large search x 2,645,516 ops/sec ±2.36% (89 runs sampled)
mixedBuffers small search x 2,700,883 ops/sec ±0.40% (87 runs sampled)
mixedBuffers medium search x 2,699,650 ops/sec ±0.42% (93 runs sampled)
mixedBuffers large search x 2,657,545 ops/sec ±2.67% (85 runs sampled)
veryLargeBuffers small search x 2,623,236 ops/sec ±0.66% (89 runs sampled)
veryLargeBuffers medium search x 2,605,667 ops/sec ±0.52% (89 runs sampled)
veryLargeBuffers large search x 2,611,727 ops/sec ±2.51% (87 runs sampled)
smallBuffers small search x 2,632,905 ops/sec ±2.51% (87 runs sampled)
smallBuffers medium search x 2,486,993 ops/sec ±3.38% (86 runs sampled)
smallBuffers large search x 2,663,287 ops/sec ±0.66% (91 runs sampled)
largeBuffers small search x 2,639,277 ops/sec ±0.69% (89 runs sampled)
largeBuffers medium search x 2,572,525 ops/sec ±2.76% (89 runs sampled)
largeBuffers large search x 2,632,493 ops/sec ±0.54% (90 runs sampled)
mixedBuffers small search x 2,570,170 ops/sec ±2.55% (88 runs sampled)
mixedBuffers medium search x 2,638,102 ops/sec ±0.59% (89 runs sampled)
mixedBuffers large search x 2,589,559 ops/sec ±0.74% (91 runs sampled)
veryLargeBuffers small search x 2,492,523 ops/sec ±2.30% (91 runs sampled)
veryLargeBuffers medium search x 2,651,338 ops/sec ±0.55% (89 runs sampled)
veryLargeBuffers large search x 2,597,808 ops/sec ±2.44% (89 runs sampled)
@reconbot

This comment has been minimized.

Copy link
Owner Author

@reconbot reconbot commented Oct 6, 2018

Running the new approach today

const smallBuffersLong = new Bl(new Array(1000).map(() => Buffer.alloc(8, 1)))
smallBuffers small search x 2,490,521 ops/sec ±3.04% (84 runs sampled)
smallBuffers medium search x 2,422,262 ops/sec ±2.88% (89 runs sampled)
smallBuffers large search x 2,600,668 ops/sec ±0.56% (94 runs sampled)
smallBuffersLong small search x 2,607,599 ops/sec ±0.59% (90 runs sampled)
smallBuffersLong medium search x 2,551,642 ops/sec ±2.46% (89 runs sampled)
smallBuffersLong large search x 2,588,665 ops/sec ±0.69% (91 runs sampled)
largeBuffers small search x 2,549,903 ops/sec ±2.64% (90 runs sampled)
largeBuffers medium search x 2,557,784 ops/sec ±1.33% (88 runs sampled)
largeBuffers large search x 2,581,822 ops/sec ±0.74% (88 runs sampled)
mixedBuffers small search x 2,470,664 ops/sec ±2.31% (91 runs sampled)
mixedBuffers medium search x 2,565,537 ops/sec ±0.62% (91 runs sampled)
mixedBuffers large search x 2,525,416 ops/sec ±2.39% (92 runs sampled)
veryLargeBuffers small search x 2,568,693 ops/sec ±0.78% (88 runs sampled)
veryLargeBuffers medium search x 2,557,850 ops/sec ±0.61% (89 runs sampled)
veryLargeBuffers large search x 2,487,267 ops/sec ±2.49% (90 runs sampled)

And now with new search init approach with less making of BL objects

smallBuffers small search x 28,823,176 ops/sec ±0.82% (94 runs sampled)
smallBuffers medium search x 28,685,310 ops/sec ±0.93% (95 runs sampled)
smallBuffers large search x 28,619,441 ops/sec ±0.85% (92 runs sampled)
smallBuffersLong small search x 28,845,733 ops/sec ±0.75% (94 runs sampled)
smallBuffersLong medium search x 28,998,154 ops/sec ±0.59% (92 runs sampled)
smallBuffersLong large search x 28,860,999 ops/sec ±0.79% (94 runs sampled)
largeBuffers small search x 28,934,405 ops/sec ±0.76% (90 runs sampled)
largeBuffers medium search x 28,965,175 ops/sec ±0.78% (93 runs sampled)
largeBuffers large search x 25,850,403 ops/sec ±1.59% (85 runs sampled)
mixedBuffers small search x 24,613,339 ops/sec ±0.92% (94 runs sampled)
mixedBuffers medium search x 28,834,310 ops/sec ±0.92% (91 runs sampled)
mixedBuffers large search x 28,900,673 ops/sec ±0.58% (92 runs sampled)
veryLargeBuffers small search x 28,583,304 ops/sec ±0.78% (91 runs sampled)
veryLargeBuffers medium search x 29,038,767 ops/sec ±0.72% (94 runs sampled)
veryLargeBuffers large search x 28,374,545 ops/sec ±1.06% (91 runs sampled)

So that's a lot better still not a great difference in time due to length.

@reconbot

This comment has been minimized.

Copy link
Owner Author

@reconbot reconbot commented Oct 6, 2018

New working perf test, I was passing arrays of undefined before 😅

const Benchmark = require('benchmark')
const Bl = require('./')

const makeArray = num => new Array(num).fill(1)

const smallBuffers = new Bl(makeArray(100).map(() => Buffer.alloc(8, 1)))
const smallBuffersLong = new Bl(makeArray(1000).map(() => Buffer.alloc(8, 1)))
const largeBuffers = new Bl(makeArray(100).map(() => Buffer.alloc(1024, 1)))
const mixedBuffers = new Bl(makeArray(100).map((val, i) => Buffer.alloc(i ** 2 % 1024, 1)))
const veryLargeBuffers = new Bl(makeArray(100).map(() => Buffer.alloc(1024 * 1024, 1)))

const notFindableSmall = Buffer.alloc(7, 0)
const notFindableMedium = Buffer.alloc(510, 0)
const notFindableLarge = Buffer.alloc(1025, 0)

const suite = new Benchmark.Suite()

suite.add('smallBuffers small search', () => {
  smallBuffers.indexOf(notFindableSmall)
})
suite.add('smallBuffers medium search', () => {
  smallBuffers.indexOf(notFindableMedium)
})
suite.add('smallBuffers large search', () => {
  smallBuffers.indexOf(notFindableLarge)
})

suite.add('smallBuffersLong small search', () => {
  smallBuffersLong.indexOf(notFindableSmall)
})
suite.add('smallBuffersLong medium search', () => {
  smallBuffersLong.indexOf(notFindableMedium)
})
suite.add('smallBuffersLong large search', () => {
  smallBuffersLong.indexOf(notFindableLarge)
})

suite.add('largeBuffers small search', () => {
  largeBuffers.indexOf(notFindableSmall)
})
suite.add('largeBuffers medium search', () => {
  largeBuffers.indexOf(notFindableMedium)
})
suite.add('largeBuffers large search', () => {
  largeBuffers.indexOf(notFindableLarge)
})

suite.add('mixedBuffers small search', () => {
  mixedBuffers.indexOf(notFindableSmall)
})
suite.add('mixedBuffers medium search', () => {
  mixedBuffers.indexOf(notFindableMedium)
})
suite.add('mixedBuffers large search', () => {
  mixedBuffers.indexOf(notFindableLarge)
})

suite.add('veryLargeBuffers small search', () => {
  veryLargeBuffers.indexOf(notFindableSmall)
})
suite.add('veryLargeBuffers medium search', () => {
  veryLargeBuffers.indexOf(notFindableMedium)
})
suite.add('veryLargeBuffers large search', () => {
  veryLargeBuffers.indexOf(notFindableLarge)
})

suite.on('cycle', function(event) {
  console.log(String(event.target));
})
.on('complete', function() {
  console.log('done');
})
.run()

// windowing approach

smallBuffers small search x 6,868 ops/sec ±1.12% (90 runs sampled)
smallBuffers medium search x 11,344 ops/sec ±1.09% (90 runs sampled)
smallBuffers large search x 18,718 ops/sec ±1.17% (89 runs sampled)
smallBuffersLong small search x 115 ops/sec ±1.32% (74 runs sampled)
smallBuffersLong medium search x 101 ops/sec ±0.80% (74 runs sampled)
smallBuffersLong large search x 108 ops/sec ±0.92% (79 runs sampled)
largeBuffers small search x 6,679 ops/sec ±1.11% (92 runs sampled)
largeBuffers medium search x 93.14 ops/sec ±0.80% (75 runs sampled)
largeBuffers large search x 46.27 ops/sec ±0.74% (61 runs sampled)
mixedBuffers small search x 6,891 ops/sec ±0.68% (89 runs sampled)
mixedBuffers medium search x 129 ops/sec ±1.10% (73 runs sampled)
mixedBuffers large search x 101 ops/sec ±0.87% (74 runs sampled)
veryLargeBuffers small search x 187 ops/sec ±2.08% (79 runs sampled)
veryLargeBuffers medium search x 59.67 ops/sec ±1.18% (63 runs sampled)
veryLargeBuffers large search x 35.32 ops/sec ±1.23% (62 runs sampled)

// old approach

smallBuffers small search x 5,248 ops/sec ±1.43% (89 runs sampled)
smallBuffers medium search x 5,210 ops/sec ±1.07% (89 runs sampled)
smallBuffers large search x 5,400 ops/sec ±1.08% (91 runs sampled)
smallBuffersLong small search x 159 ops/sec ±1.10% (81 runs sampled)
smallBuffersLong medium search x 158 ops/sec ±1.20% (80 runs sampled)
smallBuffersLong large search x 157 ops/sec ±1.03% (81 runs sampled)
largeBuffers small search x 42.98 ops/sec ±1.42% (57 runs sampled)
largeBuffers medium search x 43.01 ops/sec ±1.79% (57 runs sampled)
largeBuffers large search x 42.33 ops/sec ±1.03% (56 runs sampled)
mixedBuffers small search x 96.89 ops/sec ±1.20% (73 runs sampled)
mixedBuffers medium search x 98.41 ops/sec ±1.19% (72 runs sampled)
mixedBuffers large search x 97.30 ops/sec ±1.06% (72 runs sampled)
veryLargeBuffers small search x 0.04 ops/sec ±2.17% (5 runs sampled)

// the very large buffers took over 4 minutes and I got board and killed it

The windowing approach is objectively better in pretty much every situation and not significantly worse in other situations

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment