Skip to content

Instantly share code, notes, and snippets.

@gillesdemey
Last active May 10, 2016 00:54
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save gillesdemey/71d9d02108e469320deaa1251c5f2ecf to your computer and use it in GitHub Desktop.
Save gillesdemey/71d9d02108e469320deaa1251c5f2ecf to your computer and use it in GitHub Desktop.
Pino RingBuffer
var pino = require('./')
var RingBuffer = require('./ringbuffer')
var rb = new RingBuffer({ limit: 10 })
var log = pino(rb)
setInterval(() => {
log.info(new Date())
}, 5)
setTimeout(() => {
console.log(rb.getRecords())
process.exit(0)
}, 100)
'use strict'
var stream = require('stream')
var util = require('util')
function RingBuffer (options) {
var self = this
self.limit = options && options.limit ? options.limit : 100
self.records = []
stream.Writable.call(this)
}
util.inherits(RingBuffer, stream.Writable)
RingBuffer.prototype.write = function (record) {
if (this.records.length >= this.limit) this.records.pop()
this.records.push(record)
}
RingBuffer.prototype.getRecords = function () {
return this.records.map((record) => record.replace(/\n$/, ''))
}
module.exports = RingBuffer
@davidmarkclements
Copy link

davidmarkclements commented May 9, 2016

  1. Better to require http://npm.im/readable-stream - this gives consistent behaviour accross all node versions, including avoiding backpressure issues (basically, mem and cpu issues), also means module will stay consistent in future
  2. You know how big the array is going to be, preallocating will allow v8 to optimize mem. (and also allows inlining where dynamic arrays don't): this.records = new Array(options.limit).
  3. Pushing and popping is expensive, since you know the size its better to keep an index and write over the top of other old slots
  4. The map and replace in getRecords is expensive, why not just join to a string and leave the newlines since that's the log format anyway? Or if there's a compelling reason, a for loop (particularly for in) and substring would be significantly faster
  5. Nit pick, but no need to assign self to this, there's no scope changes and it's not being used consistently anyway

@davidmarkclements
Copy link

davidmarkclements commented May 9, 2016

Also this won't be compatible with extreme mode, or rather it will but each "record" will be 4kb of logs...

I can see two ways to handle this:

  1. Put a caveat in readme - done.
  2. Check the size of the first line that's written. If it's around 4kb either throw or warn about extreme mode - of course this could trigger for a really large single record, but if the size of a record is that big there should probably be a warning about options.limit * logsize anyway

@gillesdemey
Copy link
Author

Also this won't be compatible with extreme mode

I came to the exact same conclusion, hence I didn't enable extreme mode in the example.

Thanks a bunch for your other comments — it's rare to find good constructive feedback with regards to performance in javascript / node. I'll surely include them in the next iteration and separate this into its own module.

@davidmarkclements
Copy link

Yah cool then we'll add to readme, one other thing - would encourage exporting a function that returns a new instance of RingBuffer rather than exporting the constructor directly. This avoids issues that arise from forgetting new, naming convention stuff, fp paradigm friendly, generally makes for quicker api grocking imo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment