Christian Hochfilzer netconstructor

View message_queue_pipeline.py
1 2 3 4 5 6 7 8 9
from scrapy.xlib.pydispatch import dispatcher
from scrapy import signals
from scrapy.exceptions import DropItem
from scrapy.utils.serialize import ScrapyJSONEncoder
 
from carrot.connection import BrokerConnection
from carrot.messaging import Publisher
 
from twisted.internet.threads import deferToThread
View gist:8860790
1 2 3 4 5 6 7 8 9 10
$f("", {},
{
clip: {
onMetaData: function(c) {
var fd = c.duration;
//create a cue point for 25, 50 and 75% of player progress
var cues = [
{
time: fd * .25 * c.cuepointMultiplier,
name: "25%"
View README.markdown

require('child_process').fork() and beyond

Abstract

The purpose of this sample is to show the power of EventEmitter2 in the context of a specific example centered around [DIRT]0 and [ETL]0 applications in node.js. Given the clear limitations of the V8 heap-size doing any exceptionally large data processing in node will require such patterns, and it is in the interest of the community that we start solidifying them.

Scenario

Lets suppose that you have an ETL that you need to run on a large set of logs which has already been partitioned into files of a size that will by themselves not overload the V8 heap. These kind of size-limited log or data files are common and should need no explaination.

This ETL runs with initial conditions (very common), and thus there may be many sets of worker processes analyzing the same data for different purposes. As an intelligent developer knowning the blocking nature of in-memory data manipulation you decided

View Syns2Syms.java
1 2 3 4 5 6 7 8 9 10
/**
* Based off of the Lucene prolog parser in the wordnet contrib package within the
* main Lucene project. It has been modified to remove the Lucene bits and generate
* a synonyms.txt file suitable for consumption by Solr. The idea was mentioned in
* a sidebar of the book Solr 1.4 Enterprise Search Server by Eric Pugh.
*
* @see <a href="http://lucene.apache.org/java/2_3_2/lucene-sandbox/index.html#WordNet/Synonyms">Lucene Sandbox WordNet page</a>
* @see <a href="http://svn.apache.org/repos/asf/lucene/dev/trunk/lucene/contrib/wordnet/">SVN Repository of the WordNet contrib</a>
* @see <a href="https://www.packtpub.com/solr-1-4-enterprise-search-server/book">Solr 1.4 Enterprise Search Server Book</a>
*/
View inspectElement.js
1 2 3 4 5 6 7 8 9 10
function inspect(obj, maxLevels, level)
{
var str = '', type, msg;
 
// Start Input Validations
// Don't touch, we start iterating at level zero
if(level == null) level = 0;
 
// At least you want to show the first level
if(maxLevels == null) maxLevels = 1;
View gist:5939292
1 2 3 4 5 6 7 8 9 10
# Get a next task from some tasks queue, create and start a crawler
def start_crawler():
settings = CrawlerSettings()
settings.overrides['TELNETCONSOLE_ENABLED'] = 0
settings.overrides['WEBSERVICE_ENABLED'] = False
 
crawler = Crawler(settings)
crawler.configure()
url = get_url_from_some_queue()
View gist:5939290
1 2 3 4 5 6 7 8 9 10
# Get a next task from some tasks queue, create and start a crawler
def start_crawler():
settings = CrawlerSettings()
settings.overrides['TELNETCONSOLE_ENABLED'] = 0
settings.overrides['WEBSERVICE_ENABLED'] = False
 
crawler = Crawler(settings)
crawler.configure()
url = get_url_from_some_queue()
View README.md

Description

Simple Dashing widget (and associated job) to display a Twitter search. Uses Twitter API v1.1.

Dependencies

twitter

Add it to dashing's gemfile:

View README.md
View README.md

Setup

Put the facebooklinkstats.rb to jobs/facebooklinkstats.rb.

Add the code from facebooklinkstats.erb to the desired dashboard.

Preview

Something went wrong with that request. Please try again.