Fwissr introduction blog post

fwissr.md

Centralize your configurations

This article introduces Fwissr, our configuration registry tool that we use to centralize all our configurations.

Photo by National Library of Norway

More than five years ago, when we started to build what became fotopedia, we had a main Rails application, a Merb application for user authentication and another Merb application for photo upload. These applications were deployed by Capistrano, and our servers were configured with Puppet with some scripts to glue everything together.

All the pieces of this puzzle needed the same settings, and it became clear for our mental sanity that we needed an unified way to access these settings. So we gave birth to Fwissr, a ruby gem that centralizes configurations.

Since then we have consolidated our backend to a unique Rails application, and we now use Chef to both configure our servers and deploy the Rails app, but we still rely a lot on Fwissr.

[[MORE]]

Quick start

Fwissr is a ruby gem, so as usual, install it with:

$ [sudo] gem install fwissr

Then drop a JSON file to ~/.fwissr/ directory:

$ mkdir ~/.fwissr
$ cat <<EOS > ~/.fwissr/my_app.json
{
  "aws": {
    "access_key_id": "POpoOIHiyugUYF",
    "secret_access_key": "OIJioHUhuiUIH",
    "assets_bucket": "my_assets"
  }
}
EOS

Now you can access these settings in your ruby code:

require 'fwissr'

bucket_name = Fwissr["/my_app/aws/assets_bucket"]

puts "Assets are stored to the #{bucket_name} bucket"
# => "Assets are stored to the my_assets bucket"

Fwissr provides a tool that can be used in shell scripts:

for file in  `find ./ -type f ! -name '.*' ! -mmin +60`; do 
  export AWS_ACCESS_KEY_ID=`fwissr /my_app/aws/access_key_id`
  export AWS_SECRET_ACCESS_KEY=`fwissr /my_app/aws/secret_access_key` 
  s3cmd -v -d put `fwissr /my_app/aws/assets_bucket`:${file#./} ${file#./} "Cache-Control: max-age = 315576000, public" "Expires: Tue, 17 Sep 2019 11:40:58 GMT" "x-amz-acl: public-read"
done

Dump Fwissr registry with:

$ fwissr --dump -jp
{
  "my_app": {
    "aws": {
      "secret_access_key": "OIJioHUhuiUIH",
      "access_key_id": "POpoOIHiyugUYF",
      "assets_bucket": "assets"
    }
  }
}

Tell me more

Fwissr loads all json and yaml files found in the directories /etc/fwissr/ and ~/.fwissr/ then merges them to build a global hash (the registry).

File names are used to prefix keys in registry. For example all settings from conf file my.great.app.json start with /my/great/app/ prefix.

The configuration file fwissr.json is a special one: settings in it are not prefixed, and you can specify additional configuration sources in it thanks to the fwissr_sources setting.

For example, to load all configuration files in /etc/my_app/ directory, edit /etc/fwissr/fwissr.json to:

{
  "fwissr_sources": [
    { "filepath": "/etc/my_app/" },
  ]
}

Database configurations

In addition to configuration files, you can setup configuration databases. The only database kind supported for now is MongoDB.

For example add that source to fwissr.json:

{
  "fwissr_sources": [
    { "mongodb": "mongodb://db1.example.net/my_app", "collection": "production" },
  ]
}

Fwissr will now load all the documents in the production collection of the my_app database. In that collection, the document's _id field is the setting key and the value field is the setting value.

> db["production"].find()
{ "_id" : "foo", "value" : "bar" }
{ "_id" : "database", "value" : { "host": "db.my_app.com", "port": "1337" } }

Refreshing registry

Enable registry auto-update with the refresh source setting.

For example:

{
  "fwissr_sources": [
    { "filepath": "/etc/my_app/my_app.json" },
    { "filepath": "/etc/my_app/stuff.json", "refresh": true },
    { "mongodb": "mongodb://db1.example.net/my_app", "collection": "production" },
    { "mongodb": "mongodb://db1.example.net/my_app", "collection": "config", "refresh": true }
  ]
}

The /etc/my_app/my_app.json configuration file and the production mongodb collection are read once, whereas the settings holded by the /etc/my_app/stuff.json configuration file and the config mongodb collection expire periodically and re-fetched.

The default refresh period is 30 seconds, but you can change it with the fwissr_refresh_period setting:

{
  "fwissr_sources": [
    { "filepath": "/etc/my_app/my_app.json" },
    { "filepath": "/etc/my_app/stuff.json", "refresh": true },
    { "mongodb": "mongodb://db1.example.net/my_app", "collection": "production" },
    { "mongodb": "mongodb://db1.example.net/my_app", "collection": "config", "refresh": true }
   ],
  "fwissr_refresh_period": 60
}

I want it all

• Source code is available on github
• Code documentation on rubydoc

Credits

From Fotonauts:

• Aymerick Jéhanne @aymerick
• Pierre Baillet @octplane

Copyright (c) 2013 Fotonauts released under the MIT license.