Skip to content

Instantly share code, notes, and snippets.

View donfanning's full-sized avatar
🗃️
Holding up libraries and robbing them blind for the future.

Don Fanning donfanning

🗃️
Holding up libraries and robbing them blind for the future.
View GitHub Profile
<?
/////////////////////
// slack2html
// by @levelsio
/////////////////////
//
/////////////////////
// WHAT DOES THIS DO?
/////////////////////
//
@donfanning
donfanning / checker.sh
Created August 15, 2018 12:16 — forked from sic2/checker.sh
Script to check commonly misused word, typos, or stuff that you want to keep under control in your files
# Requires the silver searcher - https://github.com/ggreer/the_silver_searcher
# Leave empty line at the end
while IFS= read -r line; do
echo '-------------------------'
echo "Checking for word(s): $line"
ag -i "$line" chapters/
done < valid_words.txt
@donfanning
donfanning / GIF-Screencast-OSX.md
Created August 15, 2018 12:17 — forked from sic2/GIF-Screencast-OSX.md
OS X Screencast to animated GIF

OS X Screencast to animated GIF

This gist shows how to create a GIF screencast using only free OS X tools: QuickTime, ffmpeg, and gifsicle.

Screencapture GIF

Instructions

To capture the video (filesize: 19MB), using the free "QuickTime Player" application:

@donfanning
donfanning / red_ex.rb
Created August 15, 2018 12:19 — forked from wflanagan/red_ex.rb
EM-based crawler
require 'simple_worker'
require 'eventmachine'
require 'em-http-request'
require 'nokogiri'
require 'aws'
require 'redis'
class RedEx < SimpleWorker::Base
merge_gem 'em-redis'
@donfanning
donfanning / crawler.js
Created August 15, 2018 12:20 — forked from matthewmorrone/crawler.js
my first attempt at a web crawler in node... obviously I have no idea how to use asynchronicity
var http = require("http");
var fs = require("fs");
var crawler = require("simplecrawler");
var cheerio = require("cheerio");
var util = require('util');
var exit = require('exit');
var log = console.log.bind(console);
function str(a) {return util.inspect(a, false, null);}
@donfanning
donfanning / camcrawler.js
Created August 15, 2018 12:20 — forked from jeffcrouse/camcrawler.js
A quick node-crawler that looks for open surveillance cameras worldwide. Inspired by, and using URL patterns found at http://i.document.m05.de/surveillancesaver/
var Crawler = require("crawler").Crawler; // https://github.com/sylvinus/node-crawler
var S = require('string');
var fs = require('fs');
// A list of some patterns that will show up in webcam URLs
var patterns = ["jpg/image.jpg\?r="
, "mjpg/video.mjpg"
, "record/current.jpg"
, "cgi-bin/faststream.jpg"
, "oneshotimage.jpg"
@donfanning
donfanning / app.js
Created August 15, 2018 12:20 — forked from ToastHelmi/app.js
var Crawler = require("simplecrawler");
var querystring = require('querystring');
var myCrawler = new Crawler("www.geocaching.com", "/login/");
myCrawler.initialProtocol = "https";
myCrawler.acceptCookies = true;
myCrawler.on("addcookie", function (cookie) {
console.log("Cookie");
});
#!/usr/local/bin/ruby
# crawler.rb
# by: Jason Larsen
# a generic web crawler that allows the user to do whatever they want by passing blocks
# @version 0.7
# 14 Dec 2009
# 0.6 things seem to be working well
# 0.7 modified so that URL's being added to the queue truncate fragments,
# this should save a lot of work
@donfanning
donfanning / crawler.rb
Created August 15, 2018 12:22 — forked from zealot128/crawler.rb
Web Crawler Helper class based upon Poltergeist (PhantomJS).Using Capybara as framework for building webcrawlers is surprisingly convenient
class ExampleCrawler < PoltergeistCrawler
def crawl
visit "https://news.ycombinator.com/"
click_on "More"
page.evaluate_script("window.location = '/'")
end
end
ExampleCrawler.new.crawl
@donfanning
donfanning / public_file_crawler.rb
Created August 15, 2018 12:22 — forked from bycoffe/public_file_crawler.rb
A script for scraping the FCC's website and finding political file submissions
require 'open-uri'
require 'nokogiri'
class PublicFileCrawler
def initialize(params={})
@call_sign = params[:call_sign]
@url = "https://stations.fcc.gov/station-profile/#{@call_sign}/political-files/browse-%3e2012"
@checked = {}
@found = {}