I want to create a WebDB instance that initially just reads files from a local path.
To do that, I need to...
- Instantiate WebDB, passing in the DatArchive constructor
- Define my schema
- Open the db
- Add a source
This all makes sense. My first (naive) attempt at creating a webdb that stores the index
at ./index, and stores its files in ./archive is as follows:
const DatArchive = require('node-dat-archive')
const WebDb = require('@beaker/webdb')
const webdb = new WebDb('./index', { DatArchive })
webdb.define('foo', {
schema: {
type: 'object',
properties: {
foo: { type: 'string' },
},
},
filePattern: ['/*.json'],
})
async function main() {
await webdb.open()
await webdb.addSource('./archive')
await webdb.foo.put(`${archive.url}/foo.json`, { foo: 'bar' })
}
main()That fails with:
(node:2610) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: getaddrinfo ENOTFOUND . .:80
(node:2610) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Ok, makes sense - ./archive is not a dat URL.
So I'll just create a new DatArchive instance which stores its files in ./archive, and pass that in to webdb.addSource instead.
The node-dat-archive docs tell me I should use DatArchive.create. Cool:
// ...snip
async function main() {
const archive = await DatArchive.create({ localPath: './archive' })
await webdb.open()
await webdb.addSource(archive)
await webdb.foo.put(`${archive.url}/foo.json`, { foo: 'bar' })
}
main()That doesn't throw an error, great!
I expect this to create a directory called archive which contains a file called foo.json and a .dat directory.
Instead, looking into ./archive, I just see the contents of what would have been in .dat, ie the 'SLEEP files'.
That's not a problem as such, but that brings up a question: How do I use this directory with the dat CLI, eg. to run dat sync on it?
This solution has a different issue - every time I run the above command, it seems like a new dat.json file is written to the archive. (Gathering this from manually looking at the content.data and the metadata.data files)
Ok, so DatArchive.create is maybe not the right API to use every time I want to open the archive. I expected this to work similarly to the node-dat API, ie. if a dat archive exists at the path it just opens the given archive. Otherwise, it creates a new dat archive.
const Dat = require('dat-node')
Dat('./archive', (err, dat) => {
// ...
})Apart from DatArchive.create, there is also a new DatArchive(url, {localPath:}) constructor.
After digging around the DatArchive source for a bit, I try the following:
/// ...snip
async function main() {
const archive = new DatArchive(null, { localPath: './archive' })
await webdb.open()
await webdb.addSource(archive)
await webdb.foo.put(`${archive.url}/foo.json`, { foo: 'bar' })
}While this does seem to create a new dat archive, I get the following error:
(node:3166) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): QueryError: Unable to put(): the given archive is not part of the index
(node:3166) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Calling webdb.listSources() reveals that the sources are [ 'null' ], which is obviously not right.
This can be fixed by adding the following, but that's pretty ugly :(
async function main() {
const archive = new DatArchive(null, { localPath: './archive' })
+ await archive._loadPromise
await webdb.open()
await webdb.addSource(archive)
await webdb.foo.put(`${archive.url}/foo.json`, { foo: 'bar' })
}Undeterred, I try creating & sharing a dat with the dat CLI, and passing in the resulting URL to new DatArchive.
That also doesn't work, because I then can't write to the archive, as I'm not the owner (makes sense).
I still don't know what the 'right' solution is for WebDB - I'm still just playing around.
For the DatArchive API, there's currently no nice way to get the 'create or use existing' semantics that I'm used to from the dat-node API.
Also, it seems a bit of a shame that it doesn't create a "normal" dat, ie. a folder with files that I could just manually edit. This would be amazing for a performant "filesystem as a database" CMS.