mertonium

Custom Feed Parsers in a Rails App with Feedjira

Note: Just before I started writing this, I noticed that the Feedzirra gem had 1) changed it's name to Feedjira (explanation here) and 2) had pushed a bunch of updates. Everthing I talk about below worked on 0.2.4, and appears to work just fine on 1.2.0. Also, looking at the CHANGELOG, none of the updates should mess with what I talk about here.

Web feeds (RSS, Atom, etc. - but I tend to call them all "RSS feed"), while sometimes viewed as unfashionable/antiquated, are ubiquitous among web publishers. When you are building a system that needs to know when new content is published, RSS feeds are a perfect low-bar method for doing programmatically discovering new content.

In the Ruby world, Feedjira (formerly, Feedzirra) is the [most popular tool](https://www.

mertonium

source 'https://rubygems.org'

gem "nokogiri", "1.6.1"
gem "rspec", "~>2.14.0"

mertonium

Publisher.create! do |pub|
  pub.title = "Street Tree List"
  pub.endpoint = "https://citygram-sf-registries.herokuapp.com/tree-planting"
  pub.active = true
  pub.visible = true
  pub.city = "San Francisco"
  pub.icon = "leaf-collection.png"
  pub.state = "CA"
  pub.description = "List of DPW maintained street trees including: Planting date, species, and location"
  pub.tags = ["san-francisco","san francisco","sf","trees"]

mertonium

<?php
	// Configure control variables
	$couch_url = "http://yourcouchinstance.com";
	$path_to_couchapp = "/local_path_to/couchapp/";
	
	// Counter
	$db_count = 0;
	
	// Time check (we'll use this later)
	$start_ts = time();

mertonium

<?php
	// Config variables
	define("FULL_OVERWRITE", true);
	$source_couch = "http://yourcouchinstance.com";
	$destination_couch = "http://yourothercouchinstance.com";
	
	// Counter
	$db_count = 0;
	
	// Time check (we'll use this later)

mertonium

#!/usr/bin/env ruby

###############################################################################
# Philly Open311 Knowledgebase Scraper
###############################################################################

require 'nokogiri'
require "net/http"
require "uri"
require "CSV"

mertonium

<?php
/*
Plugin Name: Category Excluder
Plugin URI: https://gist.github.com/1143511
Description: Added a way to pass a list of IDs to be excluded from the default Wordpress Categories widget.
Author: @mertonium
Version: 0.1
Author URI: http://mertonium.com
*/
/**

mertonium

function(doc) {
  doc['clean_address'] = doc['ALL_meSSed_uP_addr'].toLowerCase().replace(/\b[a-z]/g, function() { return arguments[0].toUpperCase();});
  return doc;
}

mertonium

[
{
  "rmt":"276/0",
  "latitude":"42.325384",
  "longitude":"-71.075201",
  "markername":"orange",
  "opts":{
    "title":"Roxbury Friendship Fence"
  },
  "text":"<div class=\"gmap-popup\"></div>"

mertonium

var csv     = require('csv'),
    cradle  = require('cradle');

var docs = [];

var successCount = 0,
    failCount = 0,
    curproc = 0,
    throttle = 5;