voidfiles

Hello

This is a project that attempts to replicate the results from The Atlantic article How Netflix Reverse Engineered Hollywood by Alexis C. Madrigal. Instead of using "sketchy" software it attempts to use an open source stack.

It takes a 20 hour job and turns it into a 1 hour job.

QuickStart

>>> git clone https://gist.github.com/8330873.git

voidfiles

# The MIT License (MIT)
#
# Copyright (c) 2013 Alex Kessinger
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:

voidfiles

#!/usr/bin/env python
# To Use:
# pip install clint docopt
# place file in your ~/bin/ files
"""
 ____        _
|  _ \      | |
| |_) | __ _| |_ _ __ ___   __ _ _ __
|  _ < / _` | __| '_ ` _ \ / _` | '_ `
| |_) | (_| | |_| | | | | | (_| | | | |

voidfiles

// Use Gists to store code you would like to remember later on
console.log(window); // log the "window" object to the console

voidfiles

# This is a notepad to see if whoosh could do something like percolate for for elastic search
from whoosh.fields import Schema, TEXT
from whoosh.filedb.filestore import RamStorage
from whoosh.index import FileIndex
from whoosh.qparser import QueryParser

schema = Schema(title=TEXT(stored=True), content=TEXT)


def create_in(schema, indexname):

voidfiles

Run this script line by line here, and when you make it work there it will work in pourover.

from google.appengine.api import urlfetch
resp = urlfetch.fetch('http://www.vlachbild.de/feed/', deadline=60)
print resp.content

voidfiles

{
    "meta": {
        "code": 200
    },
    "data": {
        "items": [{
            "generator": {
                "link": "http://wedge.natestedman.com",
                "name": "Wedge",
                "client_id": "PDvZfuW8zhzjwU8PF9KEzFbAcTn6T67U"

voidfiles

/**
 * Isotope v1.5.25
 * An exquisite jQuery plugin for magical layouts
 * http://isotope.metafizzy.co
 *
 * Commercial use requires one-time purchase of a commercial license
 * http://isotope.metafizzy.co/docs/license.html
 *
 * Non-commercial use is licensed under the MIT License
 *

voidfiles

#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
Full script used in blog post at http://h6o6.com/2013/03/using-python-and-the-nltk-to-find-haikus-in-the-public-twitter-stream

4-clause license (original "BSD License")

Copyright (c) 2013, h6o6
All rights reserved.

voidfiles

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import sys, re
import itertools
from functools import wraps
from pyquery import PyQuery as pq

class scrape: