Skip to content

Instantly share code, notes, and snippets.

@KMarkert

KMarkert/julia_colab_notebook_earthengine.ipynb

Last active November 18, 2021 21:52
Show Gist options
  • Save KMarkert/0ceddee7bd139cc6c4757ee375ce143a to your computer and use it in GitHub Desktop.
Save KMarkert/0ceddee7bd139cc6c4757ee375ce143a to your computer and use it in GitHub Desktop.
Download ZIP
Julia_EarthEngine_GeoforGood21_Demo.ipynb
Display the source blob
Display the rendered blob
Raw
julia_colab_notebook_earthengine.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "Julia_EarthEngine_GeoforGood21_Demo.ipynb",
"provenance": [],
"collapsed_sections": [],
"include_colab_link": true
},
"kernelspec": {
"display_name": "Julia",
"language": "julia",
"name": "julia"
},
"language_info": {
"file_extension": ".jl",
"mimetype": "application/julia",
"name": "julia"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/KMarkert/0ceddee7bd139cc6c4757ee375ce143a/julia_colab_notebook_earthengine.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tQ1r1bbb0yBv"
},
"source": [
"# <img src=\"https://github.com/KMarkert/EarthEngine.jl/blob/main/docs/src/assets/logo-small.png?raw=true\" height=\"100\" /> Julia + Colab + Earth Engine\n",
"\n",
"This notebook was created from the awesome [Julia Colab Notebook Template](https://colab.research.google.com/github/ageron/julia_notebooks/blob/master/Julia_Colab_Notebook_Template.ipynb)\n",
"\n",
"## Instructions\n",
"\n",
"1. Execute the following cells in [Setup](#scrollTo=2nEX_r8YWQDN) to authenticate Earth Engine then install Julia, IJulia and other packages (if needed, update `JULIA_VERSION` and the other parameters). This takes a couple of minutes.\n",
"2. Reload this page (press Ctrl+R, or ⌘+R, or the F5 key) and continue to the next section.\n",
"\n",
"_Notes_:\n",
"* If your Colab Runtime gets reset (e.g., due to inactivity), repeat steps 1 and 2.\n",
"* After installation, if you want to change the Julia version or activate/deactivate the GPU, you will need to reset the Runtime: _Runtime_ > _Factory reset runtime_ and repeat steps."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2nEX_r8YWQDN"
},
"source": [
"# Setup\n",
"\n",
"To get started with using Earth Engine we will need credentials to interface with the service. We will initialize first using Python because after we load the Julia kernel we will not be able to do so in this environment."
]
},
{
"cell_type": "code",
"metadata": {
"id": "y_zUNa2cSGLk"
},
"source": [
"# import ee and save authentication token\n",
"import ee\n",
"ee.Authenticate()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "UmaLNPyGWuxN"
},
"source": [
"After you have successfully authenticated with Earth Engine, now it is time to venture into the world of Julia! The following cell will download Julia and install/setup the packages for working in a Jupyter environment."
]
},
{
"cell_type": "code",
"metadata": {
"id": "GIeFXS0F0zww"
},
"source": [
"%%shell\n",
"set -e\n",
"\n",
"#---------------------------------------------------#\n",
"JULIA_VERSION=\"1.6.1\" # any version ≥ 0.7.0\n",
"JULIA_PACKAGES=\"IJulia BenchmarkTools Plots EarthEngine\"\n",
"JULIA_PACKAGES_IF_GPU=\"CUDA\" # or CuArrays for older Julia versions\n",
"JULIA_NUM_THREADS=2\n",
"#---------------------------------------------------#\n",
"\n",
"if [ -n \"$COLAB_GPU\" ] && [ -z `which julia` ]; then\n",
" # Install Julia\n",
" JULIA_VER=`cut -d '.' -f -2 <<< \"$JULIA_VERSION\"`\n",
" echo \"Installing Julia $JULIA_VERSION on the current Colab Runtime...\"\n",
" BASE_URL=\"https://julialang-s3.julialang.org/bin/linux/x64\"\n",
" URL=\"$BASE_URL/$JULIA_VER/julia-$JULIA_VERSION-linux-x86_64.tar.gz\"\n",
" wget -nv $URL -O /tmp/julia.tar.gz # -nv means \"not verbose\"\n",
" tar -x -f /tmp/julia.tar.gz -C /usr/local --strip-components 1\n",
" rm /tmp/julia.tar.gz\n",
"\n",
" # Install Packages\n",
" if [ \"$COLAB_GPU\" = \"1\" ]; then\n",
" JULIA_PACKAGES=\"$JULIA_PACKAGES $JULIA_PACKAGES_IF_GPU\"\n",
" fi\n",
" for PKG in `echo $JULIA_PACKAGES`; do\n",
" echo \"Installing Julia package $PKG...\"\n",
" julia -e 'using Pkg; pkg\"add '$PKG'; precompile;\"' &> /dev/null\n",
" done\n",
"\n",
" # Install kernel and rename it to \"julia\"\n",
" echo \"Installing IJulia kernel...\"\n",
" julia -e 'using IJulia; IJulia.installkernel(\"julia\", env=Dict(\n",
" \"JULIA_NUM_THREADS\"=>\"'\"$JULIA_NUM_THREADS\"'\"))'\n",
" KERNEL_DIR=`julia -e \"using IJulia; print(IJulia.kerneldir())\"`\n",
" KERNEL_NAME=`ls -d \"$KERNEL_DIR\"/julia*`\n",
" mv -f $KERNEL_NAME \"$KERNEL_DIR\"/julia \n",
"\n",
" echo ''\n",
" echo \"Success! Please reload this page and jump to the next section.\"\n",
"fi"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "-OS3Ac017T1i"
},
"source": [
"## Checking the Installation\n",
"The `versioninfo()` function should print your Julia version and some other info about the system:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "EEzvvzCl1i0F"
},
"source": [
"versioninfo()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "dCFuqyPoXImX"
},
"source": [
"# Using the Earth Engine Julia API"
]
},
{
"cell_type": "code",
"metadata": {
"id": "3DB2Ox-IH6-J"
},
"source": [
"# import the package\n",
"using EarthEngine"
],
"execution_count": 1,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "L2d6x48VIfML"
},
"source": [
"# initialize an EE session and load the functions\n",
"Initialize()"
],
"execution_count": 2,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "rNf03gRPX7-K"
},
"source": [
"## Test the API\n",
"\n",
"Run the Mount Everest example to check and make sure everything is working properly"
]
},
{
"cell_type": "code",
"metadata": {
"id": "-Fl1mx1pX-AU"
},
"source": [
"# Print the elevation of Mount Everest.\n",
"dem = EE.Image(\"USGS/SRTMGL1_003\")\n",
"xy = Point(86.9250, 27.9881)\n",
"elev = getInfo(get(first(sample(dem, xy, 30)),\"elevation\"))\n",
"println(\"Mount Everest elevation (m): $elev\")"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "_IGwMk97Yc3T"
},
"source": [
"## Leveraging the Types and multiple dispatch\n",
"\n",
"Here we will go through an example of defining a function with multiple methods based on the type we are processing. We will use the remote sensing \"Hello World\" equivalent, calculating NDVI."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WFB-gKLMcATd"
},
"source": [
"First, a base NDVI function is defined that does the computation: $\\frac{(λ_{nir} - λ_{red})}{(λ_{nir} + λ_{red})}$"
]
},
{
"cell_type": "code",
"metadata": {
"id": "PCn6WZi-Yy3X"
},
"source": [
"# base ndvi function to do the calculation\n",
"# this is what all other methods call\n",
"function ndvi(n::EE.AbstractEEObject, r::EE.AbstractEEObject)\n",
" return (n - r) / (n + r)\n",
"end"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "XWshzReVcU8P"
},
"source": [
"Now we defined specific methods depeding on type. For an image, we need to extract the bands that correspond to NIR and Red then pass to `ndvi`"
]
},
{
"cell_type": "code",
"metadata": {
"id": "L8chg29NY_1g"
},
"source": [
"# ndvi function for image, will extract out the correct bands\n",
"# and then pass to base ndvi function\n",
"function ndvi(img::EE.Image)\n",
" n = select(img, [\"B5\"])\n",
" r = select(img, [\"B4\"])\n",
" ndvi(n,r)\n",
"end"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "5LON60dschWL"
},
"source": [
"For a feature, we need to select the properties that correspond to NIR and Red values and pass to `ndvi`"
]
},
{
"cell_type": "code",
"metadata": {
"id": "0Q4oeXI9ZEBg"
},
"source": [
"# ndvi function for image, will extract out the correct values\n",
"# and then pass to base ndvi function\n",
"function ndvi(feature::EE.Feature)\n",
" n = get(feature, \"B5\")\n",
" r = get(feature, \"B4\")\n",
" ndvi(n,r)\n",
"end"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "P9GitPijc0eM"
},
"source": [
"The methods have been defined, now we get some data to apply the methods on."
]
},
{
"cell_type": "code",
"metadata": {
"id": "MQW56BsGavz2"
},
"source": [
"# get a Landsat 8 image collection\n",
"ic = sort(\n",
" limit(\n",
" filterDate(\n",
" EE.ImageCollection(\"LANDSAT/LC08/C01/T1_SR\"),\n",
" \"2018-01-01\", \"2019-01-01\"\n",
" ),\n",
" 100, \"CLOUD_COVER\"),\n",
" \"system:time_start\", false\n",
")"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "cSV62j7ba1Jr"
},
"source": [
"# get fist image from collection\n",
"img = first(ic)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "UzNYMUGka4uX"
},
"source": [
"# sample the image so we have a feature collection\n",
"fc = sample(img; scale=30, numPixels=500)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "T4dNswXnc6nH"
},
"source": [
"Lastly, here we apply the methds to the different data types. Notice here how we alway call `ndvi` but we provide some type information to help determin which method to use when applying the method using `map()`."
]
},
{
"cell_type": "code",
"metadata": {
"id": "KlTbBjANa4rO"
},
"source": [
"# apply ndvi function on the feature collection\n",
"# we have to use @eefunc and provide the input type to force \n",
"# the Python side to do the correct thing with a mapped function\n",
"ndvi_fc = map(fc, @eefunc ndvi EE.Feature)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "Ce60LKtJa4lw"
},
"source": [
"# apply ndvi function on the image collection\n",
"ndvi_ic = map(ic, @eefunc ndvi EE.Image)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "ZaprYlbLa4il"
},
"source": [
"# apply ndvi on image\n",
"ndvi_img = ndvi(img)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "64NaIfYoa4U9"
},
"source": [
"n_value = EE.Number(0.1)\n",
"r_value = EE.Number(0.2)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "cT62rkoNbDYO"
},
"source": [
"# apply ndvi on ee.Number values\n",
"ndvi_value = ndvi(n_value, r_value)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "HBn0nLZubH9A"
},
"source": [
"getInfo(ndvi_value)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "moh-4CNDYeNO"
},
"source": [
"## Trendy lights example"
]
},
{
"cell_type": "code",
"metadata": {
"id": "SBY4JeXdTG_7"
},
"source": [
"# Add a band containing image date as years since 1990.\n",
"function createTimeBand(img)\n",
" img = EE.Image(img)\n",
" starttime = EE.Date(\"1990-01-01\")\n",
" year = difference(date(img), starttime, \"year\")\n",
" return float(addBands(year,img))\n",
"end"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "lq8QE7nNTTyx"
},
"source": [
"# get night light collection and add time band\n",
"collection = map(\n",
" select(\n",
" EE.ImageCollection(\"NOAA/DMSP-OLS/CALIBRATED_LIGHTS_V4\")\n",
" ,\"avg_vis\"\n",
" ), createTimeBand\n",
")"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "LAog5opVTWMl"
},
"source": [
"# Fit a linear trend to the nighttime lights collection.\n",
"fit = reduce(collection, linearFit())"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "DV9f8xIeTY6N"
},
"source": [
"# define a region to view results\n",
"pt = Point(15, 45)\n",
"region = bounds(buffer(pt,25e5));"
],
"execution_count": 19,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "CzOd14q0Ta5s"
},
"source": [
"# Display trend in red/blue, brightness in green.\n",
"trendylights = getThumbURL(fit, Dict(\n",
" :min => 0,\n",
" :max => (0.18, 20, -0.18,),\n",
" :bands => [\"scale\", \"offset\", \"scale\"],\n",
" :dimensions => 1024,\n",
" :region => region,\n",
"))"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "m9gqrnq4TeY5"
},
"source": [
"display(\"text/html\", \"<img src=\\\"$(trendylights)\\\">\")"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "k7TWg9e2bv2W"
},
"source": [
""
],
"execution_count": null,
"outputs": []
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment