Skip to content

Instantly share code, notes, and snippets.

John Mertens mertonium

Block or report user

Report or block mertonium

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@mertonium
mertonium / custom_rss_parsers_with_feedjira.md
Last active Aug 29, 2015
Write up of how we make sure are parser work across the ridiculous world of RSS.
View custom_rss_parsers_with_feedjira.md

Custom Feed Parsers in a Rails App with Feedjira

Note: Just before I started writing this, I noticed that the Feedzirra gem had 1) changed it's name to Feedjira (explanation here) and 2) had pushed a bunch of updates. Everthing I talk about below worked on 0.2.4, and appears to work just fine on 1.2.0. Also, looking at the CHANGELOG, none of the updates should mess with what I talk about here.

Web feeds (RSS, Atom, etc. - but I tend to call them all "RSS feed"), while sometimes viewed as unfashionable/antiquated, are ubiquitous among web publishers. When you are building a system that needs to know when new content is published, RSS feeds are a perfect low-bar method for doing programmatically discovering new content.

In the Ruby world, Feedjira (formerly, Feedzirra) is the [most popular tool](https://www.

@mertonium
mertonium / Gemfile
Last active Aug 29, 2015
Nokogiri custom pseudo selector
View Gemfile
source 'https://rubygems.org'
gem "nokogiri", "1.6.1"
gem "rspec", "~>2.14.0"
@mertonium
mertonium / sf_endpoint_creation.rb
Last active Aug 29, 2015
Blocks of ruby code that can add the Citygram-SF endpoints to citygram.org
View sf_endpoint_creation.rb
Publisher.create! do |pub|
pub.title = "Street Tree List"
pub.endpoint = "https://citygram-sf-registries.herokuapp.com/tree-planting"
pub.active = true
pub.visible = true
pub.city = "San Francisco"
pub.icon = "leaf-collection.png"
pub.state = "CA"
pub.description = "List of DPW maintained street trees including: Planting date, species, and location"
pub.tags = ["san-francisco","san francisco","sf","trees"]
@mertonium
mertonium / couchdb_replicator.php
Created Apr 19, 2011
Replicates all the dbs from one couch instance to another
View couchdb_replicator.php
<?php
// Config variables
define("FULL_OVERWRITE", true);
$source_couch = "http://yourcouchinstance.com";
$destination_couch = "http://yourothercouchinstance.com";
// Counter
$db_count = 0;
// Time check (we'll use this later)
@mertonium
mertonium / couchapp_mass_installer.php
Created Apr 19, 2011
Little php script to install the same couch app on a bunch of dbs in the same instance
View couchapp_mass_installer.php
<?php
// Configure control variables
$couch_url = "http://yourcouchinstance.com";
$path_to_couchapp = "/local_path_to/couchapp/";
// Counter
$db_count = 0;
// Time check (we'll use this later)
$start_ts = time();
@mertonium
mertonium / philly311_scraper.rb
Created May 5, 2011
This script cycles through the Philly311 Knowledgebase, devouring FAQs.
View philly311_scraper.rb
#!/usr/bin/env ruby
###############################################################################
# Philly Open311 Knowledgebase Scraper
###############################################################################
require 'nokogiri'
require "net/http"
require "uri"
require "CSV"
@mertonium
mertonium / categoryexcluder.php
Created Aug 13, 2011
Copy of the default WordPress category widget - but you can pass a list of IDs to exclude. (Children categories will be excluded as well)
View categoryexcluder.php
<?php
/*
Plugin Name: Category Excluder
Plugin URI: https://gist.github.com/1143511
Description: Added a way to pass a list of IDs to be excluded from the default Wordpress Categories widget.
Author: @mertonium
Version: 0.1
Author URI: http://mertonium.com
*/
/**
@mertonium
mertonium / tcase_transform.js
Created Aug 16, 2011
Recline transform to Title Case
View tcase_transform.js
function(doc) {
doc['clean_address'] = doc['ALL_meSSed_uP_addr'].toLowerCase().replace(/\b[a-z]/g, function() { return arguments[0].toUpperCase();});
return doc;
}
View boston_public_art.json
[
{
"rmt":"276/0",
"latitude":"42.325384",
"longitude":"-71.075201",
"markername":"orange",
"opts":{
"title":"Roxbury Friendship Fence"
},
"text":"<div class=\"gmap-popup\"></div>"
@mertonium
mertonium / massupload.js
Created Sep 12, 2011
Node.js script that loads a CSV into a Couch. I used it to upload a stop_times.txt file from the Twin Cities GTFS feed.
View massupload.js
var csv = require('csv'),
cradle = require('cradle');
var docs = [];
var successCount = 0,
failCount = 0,
curproc = 0,
throttle = 5;
You can’t perform that action at this time.