Skip to content

Instantly share code, notes, and snippets.

Robin Edwards geotheory

Block or report user

Report or block geotheory

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
View here-test.md
# setup
unlink('~/Downloads/here-test/', recursive = TRUE)
getwd()
#> [1] "/private/var/folders/_3/b8whcf8d1bb8w_lr2rrscb5m0000gp/T/RtmpAnPCwY/reprex5e3e200cebe1"
proj_path = '~/Downloads/here-test'
dir.create(proj_path)
setwd(proj_path)
here::set_here()
#> Created file .here in /Users/robinedwards/Downloads/here-test
View closest_countrycode.R
require(countrycode)
closest_field = function(x){
if(length(x) > length(unique(x))) stop('x must be a vector of unique values')
similarity = sapply(codelist, function(field){ 100 * sum(x %in% field) / length(x) })
similarity = data.frame(field = names(similarity), similarity = as.vector(similarity), stringsAsFactors=FALSE)
similarity = similarity[order(similarity$similarity, decreasing = TRUE),]
if(suppressWarnings(suppressMessages(!require(tibble)))) return(similarity)
return(as_tibble(similarity))
}
View closest_countrycode.R
require(tidyverse)
require(countrycode)
closest_field = function(x){
map_dbl(codelist, ~ 100 * sum(x %in% .x) / length(x)) %>% as.data.frame %>%
tibble::rownames_to_column() %>% `names<-`(c('field','pc_match')) %>% as_tibble() %>%
arrange(-pc_match)
}
closest_field(c('CM','AL','PA','MK','HA','CO','ZI','KU','TH','FI'))
@geotheory
geotheory / html_similarity.R
Created Mar 23, 2019
R utility to measure the similarity of HTML documents
View html_similarity.R
# Measuring the similarity of style and structure between HTML documents
# An R implementation of https://github.com/matiskay/html-similarity
# Dependencies: magrittr, rvest, stringr, xml2, dplyr
require(magrittr)
jaccard_similarity = function(x, y){
x = unique(x)
y = unique(y)
if(length(x) + length(y) == 0) return(1)
View jaccard_similarity.R
jaccard_similarity = function(x, y){
x = unique(x)
y = unique(y)
if(length(x) + length(y) == 0) return(1)
xy = length(intersect(x, y))
xy / (length(x) + length(y) - xy)
}
View Social Media Buzz What people are saying about the Las Vegas shooting (LIVE FEED) | Page 12102.html
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><base href="https://embed.scribblelive.com/Embed/v7.aspx?Id=2678695&amp;Page=12101&amp;overlay=false"><style>body{margin-left:0;margin-right:0;margin-top:0}#bN015htcoyT__google-cache-hdr{background:#f5f5f5;font:13px arial,sans-serif;text-align:left;color:#202020;border:0;margin:0;border-bottom:1px solid #cecece;line-height:16px;padding:16px 28px 24px 28px}#bN015htcoyT__google-cache-hdr *{display:inline;font:inherit;text-align:inherit;color:inherit;line-height:inherit;background:none;border:0;margin:0;padding:0;letter-spacing:0}#bN015htcoyT__google-cache-hdr a{text-decoration:none;color:#1a0dab}#bN015htcoyT__google-cache-hdr a:hover{text-decoration:underline}#bN015htcoyT__google-cache-hdr a:visited{color:#609}#bN015htcoyT__google-cache-hdr div{display:block;margin-top:4px}#bN015htcoyT__google-cache-hdr b{font-weight:bold;display:inline-block;direction:ltr}</style><div id="bN015htcoyT__google-cache-hdr"><div><span>This is Google's cache of <a hr
View readability.js
let Readability = require('readability');
var fs = require('fs')
var JSDOM = require('jsdom').JSDOM;
var url = 'https://www.example.com/the-page-i-got-the-source-from';
var filename = '~/Downloads/test.html';
View alexa-national-top50-sites.csv
We can't make this file beautiful and searchable because it's too large.
id,site,description,daily_time_on_site,daily_pageviews_per_visitor,percent_of_traffic_from_search,total_sites_linking_in,country
1,google.com,"Enables users to search the world's information, including webpages, images, and videos. Offers unique features and search technology. ",7.85,9.85,1.8,2581731,AF
2,youtube.com,"YouTube is a way to get your videos to the people who matter to you. Upload, tag and share your videos worldwide! ",8.83333333333333,5.05,11.9,2014485,AF
3,facebook.com,"A social utility that connects people, to keep up with friends, upload photos, share links and videos. ",9.66666666666667,4.01,7.4,5292468,AF
4,google.com.af,NA,6.05,6.12,2,988,AF
5,acbar.org,"ACBAR.org - Your No. 1 job site in Afghanistan, get the latest job vacancy announcement from more than 1500 employers (UN, Government Agencies, NGOs & Private Sector). ACBAR.org is the most visited website in Afghanistan and is also popular in the Southeast Aisa and Europe. ",8.93333333333333,7.16,12,272,AF
View poly-test.csv
vx vy
1e3 21
1071.519305237606 23
1148.1536214968828 27
1230.268770812381 25
1318.2567385564075 28
1412.537544622754 29
1513.5612484362086 32
1621.8100973589299 35
1737.8008287493763 33
You can’t perform that action at this time.