Skip to content

Instantly share code, notes, and snippets.

@rusintez
Created April 17, 2017 11:25
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rusintez/46cb0209d7bc6b64b16233042c85d822 to your computer and use it in GitHub Desktop.
Save rusintez/46cb0209d7bc6b64b16233042c85d822 to your computer and use it in GitHub Desktop.
ffffound is shutting down
const fs = require('fs');
const hyperquest = require('hyperquest');
const pump = require('pump');
const xray = require('x-ray');
const parse = require('json-parse-stream');
const through = require('through2').obj;
const x = xray().throttle(1, 300); // be considerate
const schema = [{
title: '.title a',
link: '.title a@title',
description: '.description',
url: 'table a img@src'
}];
x('http://ffffound.com', '#assets .asset', schema)
.paginate('#paging-next@href')
.limit(Infinity)
.stream()
.pipe(parse())
.pipe(through(({ type, value }, enc, cb) => {
if (type !== 'object') return cb();
value.filename = value.url.split('/').pop();
// make sure `img` folder exists
const path = `${__dirname}/img/${value.filename}`;
pump(hyperquest(value.url), fs.createWriteStream(path), (err) => {
if (!err) console.log('Downloaded', value.url);
cb(err, JSON.stringify(value) + '\n');
});
}))
.pipe(fs.createWriteStream(`${__dirname}/meta.json`))
.on('finish', process.exit);
Copy link

ghost commented Jun 15, 2017

Oi! Any idea if there is a torrent dump of ffffound floating around anywhere? I missed the news :(

@Sgouws
Copy link

Sgouws commented Jul 3, 2017

You can still access the old content here. -> ffffound Web Archives

@vslbsl
Copy link

vslbsl commented Sep 6, 2017

@Triune
here is torrent with a dump https://www.dropbox.com/s/lmmi02h2ekruhwr/ffffound.torrent?dl=0

An OSX sparsebundle containing the images
A MySQL dump containing all the image metadata, users, and user/image relationships

@ecrecover
Copy link

@vslbsl Hey man, do you still have the dump file? The torrent is dead after these years.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment