Skip to content

Instantly share code, notes, and snippets.

@kissge
Created March 17, 2014 15:14
Show Gist options
  • Save kissge/9601092 to your computer and use it in GitHub Desktop.
Save kissge/9601092 to your computer and use it in GitHub Desktop.
Wikipedia scraper
// usage: node wikipedia.js バールのようなもの 鈍器
var scraper = require('scraper');
var fs = require('fs');
process.argv.forEach(
function (v, i)
{
if (i < 2) return;
scraper('http://ja.wikipedia.org/wiki/' + v,
function (err, $)
{
if (err) throw err;
fs.writeFileSync(v + '.txt', $('#mw-content-text').text());
});
});
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment