Skip to content

Instantly share code, notes, and snippets.

@tkellogg

tkellogg/embedded-db.md

Created December 10, 2014 03:47
Show Gist options
  • Save tkellogg/b7807ea2675ac14da307 to your computer and use it in GitHub Desktop.
Save tkellogg/b7807ea2675ac14da307 to your computer and use it in GitHub Desktop.
Download ZIP
Requirements for an embedded database for Windows commandline apps
Raw
embedded-db.md

A couple open source .NET projects have need for a headache-free storage option. Jump-Location is a command line tool that currently uses flat files to store information about what directories a user has visited. Flat files were great at first, but we need more. Also, PSReadLine is also running into similar problems.

Storage Engine

This will most likely wrap an existing storage engine to take care of the administrative tasks. We'll have to choose a storage engine that meets some basic requirements.

  • Small size -- It will have to be packaged with other apps, especially commandline apps where there is an expectation of small size. The storage engine itself should probably be < 10MB.
  • Easy to use -- Otherwise, what's the point of this project?
  • Dynamic schema -- Running migration scripts to add/change tables is a pain, let's just keep it simple.
  • Multi-process, Multi-threaded, Multi-tenant -- One database might be used by multiple tools (i.e. Jump-Location and PSReadLine), so having namespaces would be nice. Also, might be used by different processes of the same tool.
  • C# APIs. There's some storage engines written in Go, but that'll be hard to consume from .NET

=== Candidates

MongoDB

  • Not small (75-100MB)
  • Easy to setup (single, statically linked executable)
  • Great C# driver API complete with serialization & LINQ
  • Thread-safe, multiple DBs
  • Key-value document store

LMDB

  • Very small (32KB)
  • ACID transactions (good for simplicity)
  • Namespaces (for multi-tenancy)
  • C# driver (not sure about the API)

General Requirements

Generally lean on the underlying storage engine as much as possible and only worry about administrative tasks.

  • Written in C# (it's ubiquitous on Windows, even people who don't know C# don't have much trouble learning it)
  • Runs on Windows and possibly on Linux/Mac later on
  • Determine where database files reside
    • Expose both user-local and global databases
    • Fallback APIs so that local database overrides global database (or vice versa). Think about how app.config works.
  • If out-of-process (MongoDB), take care of starting & stopping the daemon plus all the race conditions that go along with that.
  • Standalone executable (and probably powershell cmdlet) to manually poke at values (the MongoDB CLI would work fine for this)
  • Contribute data access code to Jump-Location and PSReadLine projects
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment