Skip to content

Instantly share code, notes, and snippets.

View rrcobb's full-sized avatar

Rob Cobb rrcobb

  • --
  • United States
View GitHub Profile
@rrcobb
rrcobb / testudo_course_scraper.rb
Created October 8, 2014 18:27
Scrapes and prints umd courses from testudo
require 'rubygems'
require 'nokogiri'
require 'open-uri'
require 'pry'
base_url = 'https://ntst.umd.edu/soc/'
pages = []
page = Nokogiri::HTML(open(base_url))
page.css('div.course-prefix a').each do |link|
print '.'
@rrcobb
rrcobb / edrp_soags_scraper.rb
Created January 6, 2015 21:24
Scraper to help with the shout outs and gold stars on the blog - counts the number of submissions from each user, prints in the format needed
# this scraper finds all the submissions after a certain date and outputs the html for the SOaGS post
# need to keep track of the most recent SOaGS post date
# look at the nokogiri docs to understand how we are parsing the page
require 'nokogiri'
require 'open-uri'
require 'date'
n = 20 #number of pages to capture - if there are more posts between dates, change this!
Last_date_string = "Nov 3, 2014 10:05 pm" #here's where you put the last time you checked!
Last_date = DateTime.strptime(Last_date_string, "%b %d, %Y %I:%M %p")
@rrcobb
rrcobb / counter.rb
Created January 9, 2015 15:05
counts number of tags on list of web pages
require 'open-uri'
def just_fetch_page(url)
return open(url).read
end
def just_count_tags(page, tag)
pattern = /<#{tag}\b/
tags = page.scan(pattern)
return tags.length
end
@rrcobb
rrcobb / parser.rb
Created January 9, 2015 16:20
demonstration of the use of rest-client and crack to parse a page of tweets
#tweet parser for web scraping tutorial - originally from the Bastard's Book of Ruby - awesome resource
require 'rest-client'
require 'crack'
URL = "http://ruby.bastardsbook.com/files/tweet-fetcher/tweets-data/USAGov-tweets-page-2.xml"
response = RestClient.get(URL)
if response.code == 200
xml = response.body
parsed = Crack::XML.parse(xml)
parsed["statuses"].each {|tweet|
@rrcobb
rrcobb / gist:8e2e26b9c41a0c904be3
Created July 10, 2015 17:01
stick it in your bash profile, then github-create to make a learn-co-curriculum repo. Need to configure your ssh credentials first
github-create() {
repo_name=$1
dir_name=`basename $(pwd)`
if [ "$repo_name" = "" ]; then
echo "Repo name (hit enter to use '$dir_name')?"
read repo_name
fi
@rrcobb
rrcobb / overreact.rb
Last active July 30, 2016 01:00 — forked from ndevvy/overreact.rb
react scaffold generator
#!/usr/bin/env ruby
# edited from ndevvy's overreact gist
# added flow typing and new import/export syntax
# to use: run from command line in the project root dir with the camelcased name of the component as the argument
# e.g. `ruby ~/code/overreact.rb CalculatorComponent`
# optional arguments:
# - no-test to skip the test file
@rrcobb
rrcobb / details.md
Last active October 16, 2017 16:54
Distributed Systems Book Club @fin

Fin is going to be hosting a book club for engineers looking to up their game.

We're starting out by reading van Steen and Tanenbaum's Distributed Systems 3rd Edition, because they are relevant to the systems we are building at Fin and we want to feel confident in our foundations.

You can get a copy of the book for free online from the authors by putting your email in a box here: https://www.distributed-systems.net/index.php/books/distributed-systems-3rd-edition-2017/

Format:

- 30-40 min quiet time for reading
- 20-30 min for discussion
@rrcobb
rrcobb / export_for_test.js
Created November 17, 2017 17:50
helper for conditionally exporting a function for testing
/*
* Conditionally export a function for testing
*
* Usage:
* (src/file.js)
* $test(privateHelper, module, [nameForExport])
*
* (src/file_test.js)
* import { name } from './file'
*
@rrcobb
rrcobb / reload.js
Created November 29, 2017 06:29
Quickly iterate on a file from the node command line
/*
* Usage:
* ```
* $ node
* > const reload = require('./reload.js');
* > // (make some changes to reload.js)
* > reload()
* > // Voilà! your changes are loaded into the context!
* ```
*/
@rrcobb
rrcobb / negated_change_matcher.rb
Created January 23, 2018 00:08
Rspec negated change matcher
# This should be in rspec core, but add it here so we can chain negative change expectations, like
# expect { do_thing }.to change {a}.and not_change {b}.and change {b}
RSpec::Matchers.define_negated_matcher :not_change, :change