We need a simple, schemaless REST-based storage API for use by Javascript and native mobile apps. This "backend as a service" should provide a simple mechanism for storing state for thick-client apps that don't want to invest, at least initially, in server-side logic.
Parse is the best known example of this kind of service. A service like Parse enables this kind of simple app dev workflow:
- Sign up for an account
- Get an API key
- Call "store" on the API, passing the API key and a JSON object
- Call "get" or "query" on the API to retrieve the JSON object
This kind of service makes it simple to add persistent, sharable state to native mobile applications and one-page Javascript applications.
Noteable anti-patterns:
- No server-side code
- No server or database provisioning
- Native mobile apps (via iOS, Android SDKs)
- One-page JS apps via JS SDK
- Node.js object access (no ORM)
An open source Node.js package which provides a REST interface on top of a Postgres database. The application includes support for:
- Native and federated authentication plus access control
- Automatic conversion of JSON documents to relational storage
- Schema definition and migration on demand
- Native Node.js ORM
Once the open source package is stable, we will launch a hosted version on Heroku.com. The hosted version will allow someone to start using REST DB with just a Heroku account - no git push or provisioning required.
Postgres 9.4 will include new JSONB binary support for JSON data. This will make it easier and more powerful to store nested JSON data in the database. We can leverage this feature to offer what looks much like a NoSQL data store (dynamic schema, document storage) on top of our existing Postgres service. Using Postgres will allow us to leverage a lot of the features of the database (performance, full text search, event system) with very little effort.
Building an open source package means that users can adopt the service without vendor lock-in like with Parse. It also gives us all the benefits of open source development. Building on Node means that a)the package can offer high performance with low cpu/memory utilization, b) it can be 'added' to apps using other languages as long as the node runtime is available.
Offering a hosted version will create a super-easy onramp for mobile app and Javascript app developers. The open source package means that a user can easily migrate to their own dedicated database whenever they wish.
client app ---\ ___________node.js__
--- / REST DB { \
| authentication |
| authorization |
--- | } | |
client app ---/ | V |
| PG-ORM --------|------- [Postgres db]
\____________________/
This node package provides a JSON-document oriented ORM for Node=>Postgres. It includes the core logic for translating document-based CRUD operations into SQL, and exposes a similar syntax to the Mongo DB driver. This package assumes privileged access to the database.
JSON objects will be mapped to relational tables such that top-level scalar-value keys are mapped to strongly-typed database columns, and nested objects are mapped to columns on type JSON.
For exmaple:
var user = {'name':'Mr Pune', 'age':44, 'address': {'street':'45 red dawn', 'city':'Oakland'}};
store(user)
=>
table user:
| name varchar | age integer | address json |
-------------------------------------------------------------------------
| Mr Pune | 44 | {'street':'45 red dawn', 'city':'Oakland'}
This mixed relational/document model allows JSON-PG to be essentially compatible with a typical Rails or Django relational database, while still supporting the ability to store arbitrarily nested data.
One of the features of NoSQL stores is the ability to store any document without defining its schema first. This creates a big developer productivity win. We can get most of this benefit with "schema on demand". When the user attempts to store a document in a new table we can create the table dynamically according to the structure of the document. If the user stores another document in the same table with a new field, then we run alter table to define the new column.
Because the eventual schema will be "semi-static" we should provide simple alteration endpoints, like rename table, rename column, change column type, add column and drop column.
This node package builds on top of JSON-PG to provide user-authenticated access to the database and expose a RESTful interface.
We expect many REST DB clients to execute in an untrusted environment, like a browser or mobile device. So REST DB enforces access control based on user authentication. In general this means that clients must authenticate as a particular user, and that the user has different read/write permissions for different tables.
We also support row-level permissions where rows in a table are stamped with an "owner_id" and then the table level permissions may differentiate between permissions on owned rows and unowned rows. For example, an account_balance table could have r/w perms on owned rows, and no permissions for unowned rows. By contrast a Blog posts table might have r/w perms on owned rows, and read perms on unowned rows.
API Key Super user access is via providing an API key.
Native auth
REST DB supports register(username,password,email) and login(username,password,email) authentication. When using these endpoints a users table is automatically created. Email verification is at the discretion of the caller.
Federated auth REST DB will support various Oauth providers (Facebook, Twitter, Google, ?) for authentication. The server-side of the Oauth handshake will be implemented by the REST DB application.
One very interesting feature is the existing support for Postgres LISTEN/NOTIFY in the Node pg package:
https://github.com/brianc/node-postgres/wiki/Client
This support opens up the possibilty of implementing a pub/sub eventing system within the database that is fully supported through the event system in Node itself. This could allow REST DB clients to subscribe to changes made to tables and receive notifications via Websocket of changes made to those tables. Importantly, the events would originate in the database itself so if you had a Rails app writing data directly to Postgres the events would still flow through REST DB.