Skip to content

Instantly share code, notes, and snippets.

@rusintez rusintez/ffffound.js

Created Apr 17, 2017
Embed
What would you like to do?
ffffound is shutting down
const fs = require('fs');
const hyperquest = require('hyperquest');
const pump = require('pump');
const xray = require('x-ray');
const parse = require('json-parse-stream');
const through = require('through2').obj;
const x = xray().throttle(1, 300); // be considerate
const schema = [{
title: '.title a',
link: '.title a@title',
description: '.description',
url: 'table a img@src'
}];
x('http://ffffound.com', '#assets .asset', schema)
.paginate('#paging-next@href')
.limit(Infinity)
.stream()
.pipe(parse())
.pipe(through(({ type, value }, enc, cb) => {
if (type !== 'object') return cb();
value.filename = value.url.split('/').pop();
// make sure `img` folder exists
const path = `${__dirname}/img/${value.filename}`;
pump(hyperquest(value.url), fs.createWriteStream(path), (err) => {
if (!err) console.log('Downloaded', value.url);
cb(err, JSON.stringify(value) + '\n');
});
}))
.pipe(fs.createWriteStream(`${__dirname}/meta.json`))
.on('finish', process.exit);
@divisionbyzeroerror

This comment has been minimized.

Copy link

divisionbyzeroerror commented Jun 15, 2017

Oi! Any idea if there is a torrent dump of ffffound floating around anywhere? I missed the news :(

@Sgouws

This comment has been minimized.

Copy link

Sgouws commented Jul 3, 2017

You can still access the old content here. -> ffffound Web Archives

@vslbsl

This comment has been minimized.

Copy link

vslbsl commented Sep 6, 2017

@Triune
here is torrent with a dump https://www.dropbox.com/s/lmmi02h2ekruhwr/ffffound.torrent?dl=0

An OSX sparsebundle containing the images
A MySQL dump containing all the image metadata, users, and user/image relationships

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.