Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save mikermcneil/bb09c39e19b50b9d0dd7 to your computer and use it in GitHub Desktop.
Save mikermcneil/bb09c39e19b50b9d0dd7 to your computer and use it in GitHub Desktop.
An example script that imports data from a JSON file into a Sails.js app (feel free to use as a seed/boilerplate script, whatever you like)
#!/usr/bin/env node
/**
* Module dependencies
*/
var Async = require('async');
var Filesystem = require('machinepack-fs');
var Prompts = require('machinepack-prompts');
var Sails = require('sails').Sails;
// Load app to get access to ORM, services, etc (but don't lift an actual server onto a port)
var app = Sails();
app.load({
hooks: { grunt: false },
log: { level: 'warn' }
}, function sailsReady(err){
if (err) {
console.error('Error loading app:',err);
return process.exit(1);
}
// Choose the JSON file to load
// Prompt the command-line user for a value entered interactively as a string.
Prompts.text({
message: 'Please enter the absolute path to the JSON file you\'d like to import.',
exampleValue: '/Users/mikermcneil/my-backup-data/game-of-thrones.io/King.json',
}).exec({
// An unexpected error occurred.
error: function (err){
console.error('Unexpected error interpeting interactive prompt input:', err);
return process.exit(1);
},
// OK- got user input.
success: function (jsonFilePath){
// Read and parse JSON file located at source path on disk into usable data.
Filesystem.readJson({
source: jsonFilePath,
//schema: '*', // <===== you can replace the * with an example of what you get back to guarantee that it's in the right format so you don't accidentally throw confusing errors in the `success` callback below. If you leave `*`, you'll get whatever is in the JSON file.
// schema: [{}] // <===== that would mean you're guaranteed to get an array of any kind of dictionaries (aka js objects)
schema: [{id: 3, friends: ["robert baratheon"]}] // <== and this means you're guaranteed an array of dictionaries that all have precisely two keys- one named "id" that is a number and one named "friends" which is an array of strings. Any extra keys will be sripped. If a key is missing, it will be coerced into existence (missing string becomes "", missing number becomes 0, etc. see github.com/node-machine/rttc for details)
}).exec({
// An unexpected error occurred.
error: function (err){
console.error('could not load json file- error details:',err);
return process.exit(1);
},
// No file exists at the provided `source` path
doesNotExist: function (){
console.error('could not locate json file at '+jsonFilePath);
return process.exit(1);
},
// Could not parse file as JSON.
couldNotParse: function (){
console.error('there\'s something funny about that json file at '+jsonFilePath+'-- could not be parsed... Perhaps you have an extra trailing comma in there somewhere?');
return process.exit(1);
},
// OK.
success: function (data){
// import your stuff into the db here (note you could actually just pass the array straight into `.create()` since it'll trigger the createEach() behavior-- I'm just using async in this example to demonstrate how to use it in case you need to do something more custom (like upsert, etc.)
// e.g.
Async.eachSeries(data, function (item, next){
// item is guaranteed to contain `id` and `friends`, as specified above in our schema when we were importing JSON
app.models.king.create(item).exec(function (err, newKing){
if (err) {
return next(err);
}
return next();
});
}, function afterwards (err) {
if (err) {
console.error('Import failed, error details:\n',err);
return process.exit(1);
}
// At this point, the `data` is guaranteed to match the `schema` you provided above.
console.log('all done! Import finished successfully.');
return process.exit(0);
});
} // </Filesystem.readJson :: success>
}); // </Filesystem.readJson>
} // </Prompts.text :: success>
}); // </Prompts.text>
}); // </SailsApp.prototype.load()>
// To use:
// 1. update to match your schema
// 2. hard-code the name of your JSON file (or expose it via a cli arg, or use something like http://node-machine.org/machinepack-prompts/text)
// 3. save this script to a file like "import-data.js"
// 4. run it like `node import-data.js`
//
// (also note that you can generate scripts like this one without writing any code by hand using treeline-- that's what I did here, for the most part)
//
//
//
//
// 88 88
// 88 ""
// 88
// 88 88 ,adPPYba, ,adPPYba, 8b,dPPYba, ,adPPYba, ,adPPYba,
// 88 88 a8" "" a8P_____88 88P' `"8a I8[ "" a8P_____88
// 88 88 8b 8PP""""""" 88 88 `"Y8ba, 8PP"""""""
// 88 88 "8a, ,aa "8b, ,aa 88 88 aa ]8I "8b, ,aa
// 88 88 `"Ybbd8"' `"Ybbd8"' 88 88 `"YbbdP"' `"Ybbd8"'
//
//
// This script is released under the MIT license.
//
// c. 2015 Mike McNeil
//
// Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@ravi-ture
Copy link

Thanks! Exactly what I needed 👍

@asadpakistani
Copy link

Awesome script, and great project. Best of luck with the treeline initiative.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment