Skip to content

Instantly share code, notes, and snippets.

View falkerl's full-sized avatar

Elena Savinova falkerl

View GitHub Profile
@falkerl
falkerl / test.scala
Last active March 9, 2021 19:54
Vaccine combinations
val df = spark.read.option("header", true)
.csv("/Users/elena/Downloads/vaccine_combinations.csv")
df.createTempView("data")
val diseases = df.columns.filter(_ != "ID")
diseases.map(d => df.where(col(d) === lit(1)).select(col("ID"), lit(d).as("disease")))
.reduce(_ union _)
.createTempView("vac2dis")
@falkerl
falkerl / gist:f62fd4d4b60eb2a36ed1e9073d9b17db
Created March 22, 2021 11:52
EEA & UK deaths: DIC and CVST
test("EEA_UK_deaths") {
val df = spark.read.option("header", true)
.csv("/Users/elena/Downloads/Morticd10_part*")
df.createTempView("data")
val code2country = spark.read.option("header", true)
.csv("/Users/elena/Downloads/WHO_country_codes.csv")