Skip to content

Instantly share code, notes, and snippets.

@johnburnmurdoch
johnburnmurdoch / defaultData.csv
Last active March 4, 2024 09:13 — forked from phil-pedruco/defaultData.csv
3D Scatter Plot Using three.js
x y z lp_x lp_y lp_z hp_x hp_y hp_z
0.235458 -0.597702 -0.724487 0.232433 -0.593757 -0.717156 0.003025 -0.003945 -0.007332
0.235458 -0.597702 -0.724487 0.232735 -0.594152 -0.717889 0.002723 -0.003550 -0.006598
0.217346 -0.597702 -0.724487 0.231197 -0.594507 -0.718549 -0.013850 -0.003195 -0.005939
0.217346 -0.579590 -0.724487 0.229812 -0.593015 -0.719143 -0.012465 0.013425 -0.005345
0.199234 -0.579590 -0.724487 0.226754 -0.591673 -0.719677 -0.027520 0.012083 -0.004810
0.199234 -0.597702 -0.760712 0.224002 -0.592276 -0.723781 -0.024768 -0.005426 -0.036931
0.163010 -0.579590 -0.706375 0.217903 -0.591007 -0.722040 -0.054893 0.011417 0.015665
0.108673 -0.597702 -0.724487 0.206980 -0.591676 -0.722285 -0.098307 -0.006026 -0.002203
0.090561 -0.615814 -0.724487 0.195338 -0.594090 -0.722505 -0.104777 -0.021724 -0.001982
# Prepare world data
# First up, we need to load the built-up area data that we’re going to be plotting. We download this from the European Commission’s Global Human Settlement Data portal [https://ghsl.jrc.ec.europa.eu/datasets.php] — specifically using the links from this page [http://cidportal.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_BUILT_LDSMT_GLOBE_R2015B/]. We want the 250m-resolution rasters for 1975 and 2015 (GHS_BUILT_LDS1975_GLOBE_R2016A_54009_250 and GHS_BUILT_LDS2014_GLOBE_R2016A_54009_250).
# Once you’ve downloaded these (they’re BIG, so might take a little while...), we can save ourselves a lot of hassle later on by re-projecting them into the same co-ordinate space as the other data we’re going to be using. Specifically we want to change their units from metres to lat/lon. We do this by:
# 1) Unzipping the archive, and then
# 2) Running the following script on the command-line:
# gdalwarp -t_srs EPSG:4326 -tr 0.01 0.01 path/to/your/built-up-area.tif path/to/your/built-up-area_reprojected.
name,group,year,value,subGroup,city_id,lastValue,lat,lon
Agra,India,1575,200,India,Agra - India,200,27.18333,78.01667
Agra,India,1576,212,India,Agra - India,200,27.18333,78.01667
Agra,India,1577,224,India,Agra - India,212,27.18333,78.01667
Agra,India,1578,236,India,Agra - India,224,27.18333,78.01667
Agra,India,1579,248,India,Agra - India,236,27.18333,78.01667
Agra,India,1580,260,India,Agra - India,248,27.18333,78.01667
Agra,India,1581,272,India,Agra - India,260,27.18333,78.01667
Agra,India,1582,284,India,Agra - India,272,27.18333,78.01667
Agra,India,1583,296,India,Agra - India,284,27.18333,78.01667
read_csv("https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_time_series/time_series_19-covid-Confirmed.csv") %>%
gather(date, cases, 5:ncol(.)) %>%
mutate(date = as.Date(date, "%m/%d/%y")) %>%
group_by(country = `Country/Region`, date) %>%
summarise(cases = sum(cases)) %>%
filter(country != "Others" & country != "Mainland China") %>%
bind_rows(
tibble(country = "Republic of Korea", date = as.Date("2020-03-11"), cases = 7755)
) %>%
group_by(country) %>%
@johnburnmurdoch
johnburnmurdoch / .block
Last active March 25, 2023 06:13
Spirograph effect in HTML canvas
license: mit
height: 545
scrolling: no
border: no
@johnburnmurdoch
johnburnmurdoch / .block
Last active December 20, 2022 17:31
Watercolour affect using HTML5 canvas
license: mit
height: 620
scrolling: no
border: no
@johnburnmurdoch
johnburnmurdoch / mourinho_clubelo.R
Created December 18, 2018 17:57
Scripts for downloading and visualising data from clubelo.com showing José Mourinho’s third season problem. A hand-finished version of this chart appears in this Financial Times story: https://www.ft.com/content/56acdd82-02d2-11e9-99df-6183d3002ee1
needs(tidyverse, magrittr, scales)
jose.porto <- read_csv("http://api.clubelo.com/porto")
jose.chelsea <- read_csv("http://api.clubelo.com/chelsea")
jose.inter <- read_csv("http://api.clubelo.com/inter")
jose.real <- read_csv("http://api.clubelo.com/realmadrid")
jose.mufc <- read_csv("http://api.clubelo.com/manunited")
jose.all <- bind_rows(
jose.porto %>% filter(From >= as.Date("2002-01-22") & To <= as.Date("2004-06-30")),
install.packages("needs")
library(needs)
needs(sjlabelled, tidyverse, haven, magrittr, ggrepel, sf)
# Load in data from all waves
USoc_indresp_1 <- read_dta("/path to wave 1/", encoding = "latin1")
USoc_indresp_2 <- read_dta("/path to wave 2/", encoding = "latin1")
etc
# Join all waves together, keeping the most recent driving licence record for each respondent, and their vote in the last election
# Install and load required packages
install.packages("needs")
library(needs)
needs(tidyverse, magrittr, animation, pdftools, png, scales)
# Function that extracts data from Google Mobility PDFs
process_google_mobility <- function(country_code, start_date, end_date){
# Convert first page of PDF into high-res PNG
pdf_convert(paste0("https://www.gstatic.com/covid19/mobility/",end_date,"_",country_code,"_Mobility_Report_en.pdf"), format = "png", pages = 1, dpi = 300, filenames = "IMG1.png")
@johnburnmurdoch
johnburnmurdoch / .block
Last active December 21, 2020 00:18
Tool for creating virtual spray paint / masking tape art
height: 700