Skip to content

Instantly share code, notes, and snippets.

Simon SimonGoring

Block or report user

Report or block SimonGoring

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@SimonGoring
SimonGoring / 01_SGoringPAGES_ECR.md
Last active Jan 29, 2019
Getting ready for the PAGES ECR Webinar
View 01_SGoringPAGES_ECR.md

Getting From There to Here

This webinar is set up as a "storytelling" session, about how I work through my projects that are intended for publication. In practice this is now how I manage most of my projects that involve some element of data exploration and collaboration. I will be live-coding during the presentation and intend to do so in a way that will allow people to follow along to some degree.

The webinar will be recorded and so people should not feel obligated to code along, or to keep up. If you would like to follow along, or revisit the webinar at a later date, these are the programs and R packages that you will need.

Important Links

@SimonGoring
SimonGoring / DataTypesinPython.ipynb
Created Oct 2, 2018
Loading yaml, csv, json and json-ld files in Python.
View DataTypesinPython.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@SimonGoring
SimonGoring / builddb.py
Last active Sep 28, 2018
A script to build the database from the raw files.
View builddb.py
""" Import libraries and create the connection. If the connection fails check the JSON file. """
import json
import csv
import re
import psycopg2
# Set to True if we want to delete the whole database as it runs:
dropall = True
@SimonGoring
SimonGoring / listgit.sh
Last active Dec 11, 2018
A short bash script to check a directory and report whether Git projects have been committed, or are being tracked.
View listgit.sh
#!/bin/bash
curdir=$PWD
RED='\033[1;31m'
YELLOW='\033[1;33m'
GREEN='\033[1;32m'
NC='\033[0m' # No Color
cnt=0
good=0
untr=0
uncom=0
@SimonGoring
SimonGoring / onlinesales.ipynb
Created Sep 20, 2018
Modified ipython notebook with new data and table definitions
View onlinesales.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@SimonGoring
SimonGoring / db_connect.json
Last active Sep 18, 2018
IPython Notebook for BAIT Group Project
View db_connect.json
{
"host": "localhost",
"port": 5432,
"database": "yourdatabase",
"user": "postgres",
"password": "postgres"
}
@SimonGoring
SimonGoring / clean_db.sql
Last active Sep 10, 2018
Basic Database model for real estate data in Postgres.
View clean_db.sql
/* If you've messed up and need to remove all the tables. */
DROP TABLE listings.person CASCADE;
DROP TABLE listings.property CASCADE;
DROP TABLE listings.agency CASCADE;
DROP TABLE listings.agent CASCADE;
DROP TABLE listings.listing CASCADE;
DROP TABLE listings.offer CASCADE;
@SimonGoring
SimonGoring / installLibs.sh
Last active Jan 2, 2019
Recursively searches R and Rmd files in a directory to install all required packages using R.
View installLibs.sh
#!/bin/bash
# This scrips is described in more detail at http://www.goring.org/resources/regex_bash_libraries.html
rinstall=0
while getopts "i" OPTION
do
case $OPTION in
i)
echo Running installLib with the option to install packages.
@SimonGoring
SimonGoring / neotomav2fagus.R
Created May 16, 2018
Obtain data from the v2.0 Neotoma data API and plot it using R.
View neotomav2fagus.R
# Use the newer Neotoma API to pull sites with Fagus into a map from a JSON response.
# Used as part of the Data Demo Derby (hosted by EarthRates)
small_occ <- httr::GET('http://api-dev.neotomadb.org/v2.0/data/occurrence/?taxonname=Fagus&limit=5000')
small_content <- httr::content(small_occ)
library(dplyr)
site_table <- small_content$data %>%
@SimonGoring
SimonGoring / autobuild.sh
Last active Apr 23, 2018
Autobuild RMarkdown presentations
View autobuild.sh
#!/bin/bash
### Set initial time of file
LTIME=`stat -c %Z ./$1.Rmd`
while true
do
ATIME=`stat -c %Z ./$1.Rmd`
if [[ "$ATIME" != "$LTIME" ]]
You can’t perform that action at this time.