dcmoura

http https://data.giss.nasa.gov/gistemp/tabledata_v4/GLB.Ts+dSST.csv | sed 1d |
spyql "
  SELECT Year, cols[1:13] AS temps 
  FROM csv 
  TO json" |
spyql "
  SELECT json->Year + ((row_number-1)%12)/12 AS year, json->temps AS temp 
  FROM json 
  EXPLODE json->temps 
  WHERE json->temps is not Null 

pavlov99

// Based on following links:
// http://andrew.hedges.name/experiments/haversine/
// http://www.movable-type.co.uk/scripts/latlong.html
df
  .withColumn("a", pow(sin(toRadians($"destination_latitude" - $"origin_latitude") / 2), 2) + cos(toRadians($"origin_latitude")) * cos(toRadians($"destination_latitude")) * pow(sin(toRadians($"destination_longitude" - $"origin_longitude") / 2), 2))
  .withColumn("distance", atan2(sqrt($"a"), sqrt(-$"a" + 1)) * 2 * 6371)

>>>
+--------------+-------------------+-------------+----------------+---------------+----------------+--------------------+---------------------+--------------------+------------------+
|origin_airport|destination_airport|  origin_city|destination_city|origin_latitude|origin_longitude|destination_latitude|destination_longitude|                   a|          distance|

jimbojsb

Step 0:

Get Homebrew installed on your mac if you don't already have it

Step 1:

Install highlight. "brew install highlight". (This brings down Lua and Boost as well)

Step 2: