Skip to content

Instantly share code, notes, and snippets.

View matteodefelice's full-sized avatar
💭
Busy connecting things...

Matteo De Felice matteodefelice

💭
Busy connecting things...
View GitHub Profile

Keybase proof

I hereby claim:

  • I am matteodefelice on github.
  • I am matteodefelice (https://keybase.io/matteodefelice) on keybase.
  • I have a public key whose fingerprint is 438B DC6E F6CB 8095 4C67 72E3 3D6D 00F8 A83F FA3C

To claim this, I am signing this object:

@matteodefelice
matteodefelice / load_gen_data_entsoe.R
Created November 11, 2016 10:09
Code to batch load generation data saved locally from ENTSO-E data portal
library(dplyr)
library(lubridate)
PATH = '/opt/data/ENTSOE/'
DIR = 'AggregatedGenerationPerType'
YEARs = 2015:2016
MONTHs = 1:12
gen_data = list()
for (YEAR in YEARs) {
for (MONTH in MONTHs) {
filename = paste0(PATH, DIR , sprintf("/%d_%d_%s.csv", YEAR, MONTH, DIR))
## List of files to import
library(tidyverse)
library(stringr)
BASEPATH = '/opt/data/ECEM/ERA/adjusted/'
files = list.files(BASEPATH, pattern = "*csv", recursive = T)
orig_files = files[str_detect(files, 'output')]
# fix double __
files = stringr::str_replace(orig_files, '__', '_')
field_names = c('category', 'generation','originator',
## This script upload all the csv files into the DIR directory into a sqlite database
library(tidyr) # for data wrangling...
library(dplyr) # ...and connecting with the db
library(readr) # to read the csv files
# Create the db, delete any existing version before running this script
# if we want to use another dbms, there are dplyr::src_mysql, dplyr::src_postgres...
db_name = src_sqlite("ecem_data.sqlite", create = T)
##
library(tidyverse)
library(lubridate)
source("process_escii_csv.R")
files <- list.files(
path = "/opt/data/ECEM/DEMONSTRATOR/demonstrator-061217/DEM",
pattern = glob2rx("*PWR*01d*NT*"),
full.names = TRUE
)
@matteodefelice
matteodefelice / download-erain-moda.py
Created April 4, 2018 07:18
Download ERA-INTERIM data monthly stream (moda) year-by-year
#!/usr/bin/env python
from ecmwfapi import ECMWFDataServer
server = ECMWFDataServer()
for i in range(2017, 2018):
server.retrieve({
"class": "ei",
"dataset": "interim",
"date": str(i) + '0101/' + str(i) + '0201/' + str(i) + '0301/' + str(i) + '0401/' + str(i) + '0501/' + str(i) + '0601/' + str(i) + '0701/' + str(i) + '0801/' + str(i) + '0901/' + str(i) + '1001/' + str(i) + '1101/' + str(i) + '1201',
"expver": "1",
"grid": "0.75/0.75",
@matteodefelice
matteodefelice / matteo_wind_speed_100m.py
Last active June 26, 2018 09:31
My first CDS Toolbox app to visualise 100m wind speed from ERA5 reanalyses #CDSToolbox. To test the application add a new application in your toolbox workspace and copy & paste the following code.
import cdstoolbox as ct
@ct.application(title='100m Wind Speed')
@ct.input.dropdown('show_year', values=range(2008, 2018))
@ct.input.dropdown('region', values=['Europe', 'China'])
@ct.input.text('month', type = int, label='Month', default=7)
@ct.input.text('day', type = int, label='Day', default=1)
@ct.output.figure()
def plot_map(show_year, region, month, day):
@matteodefelice
matteodefelice / show_pv_potential.py
Last active June 27, 2018 09:00
Another CDS Toolbox app to visualise the PV Power Potential ERA5 reanalyses #CDSToolbox. To test the application add a new application in your toolbox workspace and copy & paste the following code. NOTE: the units in the colorbar are wrong (the potential is dimensionless) but I couldn't find a way to remove it.
import cdstoolbox as ct
@ct.application(title='PV Power Potential',
description = 'Computation of PV Power potential for a specific day using the dimensionless described in Mavromatakis et al. https://doi.org/10.1016/j.renene.2009.11.010',
abstract = 'Lorem Ipsum')
@ct.input.dropdown('show_year', values=range(2010, 2017))
@ct.input.dropdown('region', values=['Europe', 'China'])
@ct.input.text('month', type = int, label='Month', default=7)
@ct.input.text('day', type = int, label='Day', default=1)
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@matteodefelice
matteodefelice / era5_download_cds.py
Created February 22, 2019 14:23
A script to download ERA5 entire dataset for a set of selected surface variables.
import cdsapi
import os
c = cdsapi.Client()
VARS = ['100m_u_component_of_wind','100m_v_component_of_wind','10m_u_component_of_wind', '10m_v_component_of_wind', '2m_temperature','surface_solar_radiation_downward_clear_sky','surface_solar_radiation_downwards', '2m_dewpoint_temperature','mean_sea_level_pressure']
YEARS = [x for x in map(str, range(1979, 2019))]
for V in VARS: