Skip to content

Instantly share code, notes, and snippets.

@No9 No9/zfs-leveldb.md Secret
Last active Dec 17, 2015

Embed
What would you like to do?
Notes on ZFS and LevelDB

Using leveldb and ZFS for snapshots and backups

There are some interesting database technologies emerging from the node.js arena. One that I have contributed to is leveldb.

leveldb integrates the leveldb key value store implemented by google with node.js bindings provided by leveldown and leveldown is then abstracted into a more javascript friendly and easier to use API in levelup.

You can obtain all the parts separately but level is the packaged distribution of all of these technologies in a nice and handy place.

One of the key questions I have heard is "how do we provide backup and restore functionality?"

When I discussed this on IRC (##leveldb) there was a couple of suggestions.

  1. replicate the data using level-master.
  2. Use Linux Volume Manager (LVM)

While 1. was an option it seemed to be a little heavy for plain backup purposes.
I was intrigued by option 2. the LVM option but I use joyent SmartOS and other illumOS systems for deployment so I decided to investigate what would be involved in using ZFS.

ZFS is a combined file system and logical volume manager designed by Sun Microsystems and is often described as a "modern" file system and given my experience with it I can see why it deserves such an epithet.

While my production deployments are on illumOS based systems I know that Linux is more readily installed so this article will describe how to use level and ZFS technology on Ubuntu and I will append the commands to the bottom of this post to use on illumOS based systems.

Prerequisites

  1. node.js v 0.8 or above
  2. npm v 1.1.x or above 3, Ubuntu Raring 13.04 ZFS may be available for earlier versions but YMMV.

1. On ubuntu install ZFS

sudo apt-get install zfs-fuse

The above command worked for me on 13.04 but recent literature has the following instructions:

apt-add-repository --yes ppa:zfs-native/stable
apt-get update
apt-get install debootstrap ubuntu-zfs

Which ever method you used for the install of ZFS the following command should work.

sudo zpool list
   no pools available

2. Create a zfs file system

In your home folder create a folder called zfslevel the create 4 files

mkdir zfslevel
cd zfslevel
dd if=/dev/zero of=level_test1 bs=134217728 count=1
dd if=/dev/zero of=level_test2 bs=134217728 count=1
dd if=/dev/zero of=level_test3 bs=134217728 count=1
dd if=/dev/zero of=level_test4 bs=134217728 count=1 

3. Create a mirrored database using the two disks

sudo zpool create leveldata mirror ~/zfslevel/level_test1 ~/zfslevel/level_test2

And now the command sudo zpool status leveldata

Should give the output:

  pool: leveldata
 state: ONLINE
 scrub: none requested
config:

	NAME                                  STATE     READ WRITE CKSUM
	leveldata                             ONLINE       0     0     0
	  mirror-0                            ONLINE       0     0     0
	    /home/anton/levelzfs/level_test1  ONLINE       0     0     0
	    /home/anton/levelzfs/level_test2  ONLINE       0     0     0

errors: No known data errors

4. Now we should have a file system available at /leveldata

stat /leveldata

File: ‘/leveldata/’
  Size: 2         	Blocks: 3          IO Block: 512    directory
Device: 19h/25d	Inode: 1           Links: 2
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2013-05-29 17:01:46.000000000 +0100
Modify: 2013-05-29 16:50:23.000000000 +0100
Change: 2013-05-29 16:50:23.000000000 +0100
 Birth: -

5. Build an application that uses leveldb on the zfs store

OK So now we have a mirrored file system lets create an application to use it.

cd to root of home make a directory called levapp and make an index.js and install level

cd ~
mkdir levapp 
cd levapp 
touch index.js
npm install level

Now edit index.js and enter the following code into it

var level = require('level')

// 1) Create our database, supply location and options.
//    This will create or open the underlying LevelDB store.
var db = level('/leveldata/mydb')

// 2) put a key & value
db.put('name', 'Level number 2', function (err) {
  if (err) return console.log('Ooops!', err) // some kind of I/O error

  // 3) fetch by key
  db.get('name', function (err, value) {
    if (err) return console.log('Ooops!', err) // likely the key was not found

    // ta da!
    console.log('name=' + value)
  })
})

now run the script with

sudo env PATH=$PATH node index.js 

This will return with name=Level

I would recommend adding

alias sudo='sudo env PATH=$PATH'

to .bashrc

So we can now see our database has been created on the pool we created by running ls or zpool list

ls /leveldata/mydb/
    
     000005.sst  000006.log  CURRENT  LOCK  LOG  LOG.old  MANIFEST-000004

sudo zpool list

     NAME        SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
     leveldata   123M   350K   123M     0%  1.00x  ONLINE  -

6. Creating and restoring a snapshot

Snapshots create a point of reference that is only takes up space the more changes are applied to the current database.

To create a snapshot

sudo zfs snapshot leveldata@friday

change the line

db.put('name', 'Level', function (err) {

to

db.put('name', 'Level number 2', function (err) {

now run the script with

sudo env PATH=$PATH node index.js 

This will return with name=Level

Lets look at the snapshot

sudo zfs list -r -t snapshot -o name,creation leveldata
     NAME              CREATION
     leveldata@friday  Wed May 29 20:00 2013

OK so lets roll this back

sudo zfs rollback leveldata@friday

Now lets create a script that just gets the data so we can see the rollback touch read.js

var level = require('level')

// 1) Create our database, supply location and options.
//    This will create or open the underlying LevelDB store.
var db = level('/leveldata/mydb')


  // 3) fetch by key
  db.get('name', function (err, value) {
    if (err) return console.log('Ooops!', err) // likely the key was not found

    // ta da!
    console.log('name=' + value)
  })

now

sudo env PATH=$PATH node read.js 
     name=Level

For more info on snapshots see http://docs.huihoo.com/opensolaris/solaris-zfs-administration-guide/html/ch06.html

7. Backups

So we have seen snapshots but now we want to put the data somewhere else so we can restore it if there is an incident.

We will start off by creating a backup device in this case another ZFS pool. IRL this would be a different set of devices or an SSH connection but here we will use the other to file disks we created at the start

sudo zpool create levelbackup mirror ~/zfslevel/level_test3 ~/zfslevel/level_test4

And then we need to move the data from live to backup

# sudo zfs send leveldata@friday | sudo zfs receive levelbackup@friday

So lets over right the data

sudo env PATH=$PATH node index.js 
     name=Level number 2

then lets remove all the files in /leveldata/mydb

sudo rm -fr /leveldata/mydb

and the previous snapshot

sudo zfs destroy leveldata@friday

Now lets restore the backup

sudo zfs send levelbackup/test@friday | sudo zfs recv -F leveldata

You may find that the command returns cannot receive new filesystem stream: I/O error but this does not seem to effect the restore

Now we need to see the data that was restored using the read.js file

sudo env PATH=$PATH node read.js 

for more info on backups see http://breden.org.uk/2008/05/12/home-fileserver-backups-from-zfs-snapshots/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.