Skip to content

Instantly share code, notes, and snippets.

Robin Wilson robintw

Block or report user

Report or block robintw

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
View PyCon2018Proposal.md

What is your session about?

"I wish there was a way to easily manipulate this huge multi-dimensional array in Python...", I thought, as I stared at a huge chunk of satellite data on my laptop. The data was from a satellite measuring air quality - and I wanted to slice and dice the data in some supposedly simple ways. Using pure numpy - the go-to library when the words 'multi-dimensional', 'array' and 'python' are mentioned in the same sentence - was just such a pain. What I wished for was something like pandas - with datetime indexes, fancy ways of selecting subsets, group-by operations and so on - but something that would work with my huge multi-dimensional array.

The solution: XArray - a wonderful library which provides the power of pandas for multi-dimensional data. In this talk I will introduce the XArray library by showing how just a few lines of code can answer questions about my data that would take a lot of complex code to answer with pure numpy - questions like 'What is the average air quality in

@robintw
robintw / create_html_from_opml.py
Created Apr 24, 2018
Creates a HTML list from an OPML file
View create_html_from_opml.py
import dominate
from dominate.tags import h1, ul, li, a
import listparser
parsed = listparser.parse('feedly-11e0b8cd-1b68-422a-bf73-19cb26386252-2018-03-25.opml')
feeds = parsed.feeds
doc = dominate.document(title="Robin's blogroll")
with doc:
View BUA_Analysis.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@robintw
robintw / list_patched_functions.py
Created Sep 18, 2016
List patched functions in recipy
View list_patched_functions.py
import recipy
import pprint
from tinydb import TinyDB
from jinja2 import Template
from recipyCommon.utils import open_or_create_db
db = open_or_create_db()
modules = db.table('patches').all()
@robintw
robintw / reproduce.py
Created Sep 9, 2016
Example `reproduce` command for recipy
View reproduce.py
import sys
from tinydb import where
from recipyCommon.utils import open_or_create_db
id_string = sys.argv[1]
db = open_or_create_db()
results = db.search(where('unique_id').matches('%s.*' % id_string))[0]
View StratifiedRegression.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View gist:5faf577e209d381a55e6a7f3a3374ad0
> python prepare_landsat_image.py D:\HOTBAR\L7Examples\LE72020252001228EDC00.tar.bz
recipy run inserted, with ID 2e1ae247-6dac-489c-9b66-e75ad6e3414b
20:22:47:INFO:Processing image with scene ID = LE72020252001228EDC00 to D:\HOTBAR\L7Examples\LE72020252001228EDC00
20:22:47:INFO:Extracting D:\HOTBAR\L7Examples\LE72020252001228EDC00.tar.bz
20:23:16:INFO:Extraction complete.
20:23:16:INFO:Renaming images
20:23:16:INFO:Not renaming file: README.GTF
20:23:16:INFO:Merging uncorrected images
20:23:16:INFO:Generating radiance scaling factors for Landsat 7
Traceback (most recent call last):
View Notes on changs
Steps I took:
1. Download as .py from the Jupyter Notebook
2. Delete the irrelevant comments with In [3], etc
3. Deleted the gdalInfo bit from the very top - not needed
4. Ran autopep8 on the file to format it nicely (from a terminal: autopep8 -i FILENAME)
View Reprojecting_MAIAC_data_ForAnnie.py
import os
import gdal
import glob
# # Translate Lat and Lon files
# - Find out information about the lat lon file
# - Using the gdal_translate tool convert the lat and lon hdr files into vrt format
# gdal.Translate(r"h00v01_lat.vrt",
@robintw
robintw / convert_to_pm25.py
Created Jul 21, 2016
Code to convert AOT data to PM2.5
View convert_to_pm25.py
import numbers
import gdal
import numpy as np
import os
import tempfile
from subprocess import call
import logging
from resample_image import resample_raster
You can’t perform that action at this time.