Skip to content

Instantly share code, notes, and snippets.

@rvagg
Created January 10, 2014 01:48
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rvagg/8345644 to your computer and use it in GitHub Desktop.
Save rvagg/8345644 to your computer and use it in GitHub Desktop.
Modular JavaScript storage for databases

Summary: Introduce a totally new approach to highly modular database storage, i.e. taking the vibrant Node.js Level ecosystem below LevelUP to the storage layer in 100% JavaScript.*

Facebook recently released their fork of LevelDB, named "RocksDB". Their fork is a true fork in the sense that it is not backward compatible. Facebook have added a number of new features to RocksDB but more importantly they are attempting to make something more flexible that can be adjusted to suit different workloads.

But this is not about Facebook or RocksDB because they are still riffing on the same tune, albeit with a faster tempo. But their attempt at flexibility hints at the possibilities of modular storage technologies.

The Node.js Level* ecosystem is vibrant and growing rapidly. Most of this growth is in user-land above the storage layer. In 2014 I would like to see more innovation at the storage layer itself. Facebook are demonstrating a very limited kind of modularity in RocksDB but we believe that Node.js is much better suited to properly take on the challenge of a modular database storage engine.

LevelDB.js (https://github.com/rvagg/leveljs) is the beginnings of a rewrite of LevelDB in JavaScript. I would like to make further progress on this effort and encourage others to support it in various ways (either by code contributions or by donations http://feedopensource.com/ style).

LevelDB contains a number of interesting technologies that all have merit as stand-alone components. LevelDB.js will be a highly modular project made up of separate components that will each have their own use-cases apart from the core project. Being able to combine these components in novel ways and mix in new and alternative components ought to give us the ability to create highly customised storage solutions suitable for many different kinds of environments and data.

The main win would be to have a storage layer for JavaScript databases finally written entirely in JavaScript, but the side benefits of having an innovative ecosystem of modular storage systems would be yet another first for Node.js.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment