Skip to content

Instantly share code, notes, and snippets.

@artcommacode
Last active April 15, 2018 22:31
Show Gist options
  • Save artcommacode/350051c476bf0746f49c to your computer and use it in GitHub Desktop.
Save artcommacode/350051c476bf0746f49c to your computer and use it in GitHub Desktop.
// 1. Save gist as fixsvpplylinks.js
// 2. Install node.js (http://nodejs.org/)
// 3. 'npm install async request'
// 4. 'node fixsvpplylinks.js /path/to/wants.json'
// 5. Wait.
// 6. Don't worry about those EventEmitter warnings!
// 7. Done.
// 8. If there's any issues then give me a tweet (@artcommacode)
var fs = require('fs')
, url = require('url')
, async = require('async')
, request = require('request');
if (!process.argv[2]) return console.log('You must provide an input file');
fs.readFile(process.argv[2], 'utf8', function readFile (error, body) {
if (error) return console.log(error);
body = body.split('][').join(',');
try {
body = JSON.parse(body);
} catch (e) {
return console.log(e);
}
async.map(body, function parseItem (item, callback) {
request(item.page_url, function retrievedURL (error, response, body) {
if (error) return callback(error);
item.page_url = url.parse(response.request.uri.href, true).query.redirect;
callback(null, item);
});
}, function parsedItems (error, items) {
if (error) return console.log(error);
var newFile = process.argv[2].split('.json').join('.parsed.json');
fs.writeFile(newFile, JSON.stringify(items), function wroteFile (error) {
if (error) return console.log(error);
return console.log('Done: ' + newFile);
});
});
});
@matiassingers
Copy link

My export file contained a null element at the end of the body, so weird.

Just added a quick check that the item is defined at L27:

if(!item) return;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment