Skip to content

Instantly share code, notes, and snippets.

@ColinTalbert
Last active October 17, 2018 17:10
Show Gist options
  • Save ColinTalbert/75019f691654d694a0cff0a6641e59de to your computer and use it in GitHub Desktop.
Save ColinTalbert/75019f691654d694a0cff0a6641e59de to your computer and use it in GitHub Desktop.
demo of how to generate a shapefile from a csv of GRTS cells
Display the source blob
Display the rendered blob
Raw
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Question 1\n",
"#### How do we install geopandas in our container?"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import geopandas as gpd\n",
"import pandas as pd"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Question 2\n",
"#### Can we have a local copy of the shapefile in this container?\n",
"#### Or would it make more sense to to hit a wfs?\n",
"#### Or would it make more sense to generate the geometries by code (would need the CONUS_10km as well as GRTS_ID)"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"grid_fname = r\"Z:\\TSH\\DD274_NABat\\SamplingGrid\\DATA\\DERIVED\\GridsWithGRTS\\Conus\\conus_mastersample_10km_GRTS.shp\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Once we have those answers the actual generation of the shapefile is relatively easy."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"grid = gpd.GeoDataFrame.from_file(grid_fname)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>CONUS_10KM</th>\n",
" <th>lat</th>\n",
" <th>long</th>\n",
" <th>GRTS_ID</th>\n",
" <th>geometry</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>132883</td>\n",
" <td>47.797518</td>\n",
" <td>-127.691696</td>\n",
" <td>1</td>\n",
" <td>POLYGON ((-2353000 3156000, -2343000 3156000, ...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>74208</td>\n",
" <td>39.293360</td>\n",
" <td>-108.770110</td>\n",
" <td>2</td>\n",
" <td>POLYGON ((-1093000 1886000, -1083000 1886000, ...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>520</td>\n",
" <td>24.145321</td>\n",
" <td>-113.527387</td>\n",
" <td>3</td>\n",
" <td>POLYGON ((-1803000 296000, -1793000 296000, -1...</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" CONUS_10KM lat long GRTS_ID \\\n",
"0 132883 47.797518 -127.691696 1 \n",
"1 74208 39.293360 -108.770110 2 \n",
"2 520 24.145321 -113.527387 3 \n",
"\n",
" geometry \n",
"0 POLYGON ((-2353000 3156000, -2343000 3156000, ... \n",
"1 POLYGON ((-1093000 1886000, -1083000 1886000, ... \n",
"2 POLYGON ((-1803000 296000, -1793000 296000, -1... "
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"grid.head(3)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### This is the csv that would get supplied to the microservice"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>GRTS_ID</th>\n",
" <th>project</th>\n",
" <th>species</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>1768</td>\n",
" <td>foo</td>\n",
" <td>MYLU</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>9960</td>\n",
" <td>foo</td>\n",
" <td>EPFU</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>33128</td>\n",
" <td>foo</td>\n",
" <td>EPFU</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>37224</td>\n",
" <td>bar</td>\n",
" <td>MYLU</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" GRTS_ID project species\n",
"0 1768 foo MYLU\n",
"1 9960 foo EPFU\n",
"2 33128 foo EPFU\n",
"3 37224 bar MYLU"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sample_csv = \"\"\"GRTS_ID,project,species\n",
"1768,foo,MYLU\n",
"9960,foo,EPFU\n",
"33128,foo,EPFU\n",
"37224,bar,MYLU\"\"\"\n",
"\n",
"import io\n",
"\n",
"df = pd.read_csv(io.StringIO(sample_csv))\n",
"df"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### A couple pandas/geopandas lines gives us a shapefile."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"selected_cells = grid[grid.GRTS_ID.isin(df.GRTS_ID)]"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"merged = selected_cells.merge(df)\n",
"merged.to_file(r\"C:\\temp\\out.shp\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Quick check of the output..."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div style=\"width:100%;\"><div style=\"position:relative;width:100%;height:0;padding-bottom:60%;\"><iframe src=\"data:text/html;charset=utf-8;base64,PCFET0NUWVBFIGh0bWw+CjxoZWFkPiAgICAKICAgIDxtZXRhIGh0dHAtZXF1aXY9ImNvbnRlbnQtdHlwZSIgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PVVURi04IiAvPgogICAgPHNjcmlwdD5MX1BSRUZFUl9DQU5WQVM9ZmFsc2U7IExfTk9fVE9VQ0g9ZmFsc2U7IExfRElTQUJMRV8zRD1mYWxzZTs8L3NjcmlwdD4KICAgIDxzY3JpcHQgc3JjPSJodHRwczovL2Nkbi5qc2RlbGl2ci5uZXQvbnBtL2xlYWZsZXRAMS4yLjAvZGlzdC9sZWFmbGV0LmpzIj48L3NjcmlwdD4KICAgIDxzY3JpcHQgc3JjPSJodHRwczovL2FqYXguZ29vZ2xlYXBpcy5jb20vYWpheC9saWJzL2pxdWVyeS8xLjExLjEvanF1ZXJ5Lm1pbi5qcyI+PC9zY3JpcHQ+CiAgICA8c2NyaXB0IHNyYz0iaHR0cHM6Ly9tYXhjZG4uYm9vdHN0cmFwY2RuLmNvbS9ib290c3RyYXAvMy4yLjAvanMvYm9vdHN0cmFwLm1pbi5qcyI+PC9zY3JpcHQ+CiAgICA8c2NyaXB0IHNyYz0iaHR0cHM6Ly9jZG5qcy5jbG91ZGZsYXJlLmNvbS9hamF4L2xpYnMvTGVhZmxldC5hd2Vzb21lLW1hcmtlcnMvMi4wLjIvbGVhZmxldC5hd2Vzb21lLW1hcmtlcnMuanMiPjwvc2NyaXB0PgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL2Nkbi5qc2RlbGl2ci5uZXQvbnBtL2xlYWZsZXRAMS4yLjAvZGlzdC9sZWFmbGV0LmNzcyIvPgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL21heGNkbi5ib290c3RyYXBjZG4uY29tL2Jvb3RzdHJhcC8zLjIuMC9jc3MvYm9vdHN0cmFwLm1pbi5jc3MiLz4KICAgIDxsaW5rIHJlbD0ic3R5bGVzaGVldCIgaHJlZj0iaHR0cHM6Ly9tYXhjZG4uYm9vdHN0cmFwY2RuLmNvbS9ib290c3RyYXAvMy4yLjAvY3NzL2Jvb3RzdHJhcC10aGVtZS5taW4uY3NzIi8+CiAgICA8bGluayByZWw9InN0eWxlc2hlZXQiIGhyZWY9Imh0dHBzOi8vbWF4Y2RuLmJvb3RzdHJhcGNkbi5jb20vZm9udC1hd2Vzb21lLzQuNi4zL2Nzcy9mb250LWF3ZXNvbWUubWluLmNzcyIvPgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL2NkbmpzLmNsb3VkZmxhcmUuY29tL2FqYXgvbGlicy9MZWFmbGV0LmF3ZXNvbWUtbWFya2Vycy8yLjAuMi9sZWFmbGV0LmF3ZXNvbWUtbWFya2Vycy5jc3MiLz4KICAgIDxsaW5rIHJlbD0ic3R5bGVzaGVldCIgaHJlZj0iaHR0cHM6Ly9yYXdnaXQuY29tL3B5dGhvbi12aXN1YWxpemF0aW9uL2ZvbGl1bS9tYXN0ZXIvZm9saXVtL3RlbXBsYXRlcy9sZWFmbGV0LmF3ZXNvbWUucm90YXRlLmNzcyIvPgogICAgPHN0eWxlPmh0bWwsIGJvZHkge3dpZHRoOiAxMDAlO2hlaWdodDogMTAwJTttYXJnaW46IDA7cGFkZGluZzogMDt9PC9zdHlsZT4KICAgIDxzdHlsZT4jbWFwIHtwb3NpdGlvbjphYnNvbHV0ZTt0b3A6MDtib3R0b206MDtyaWdodDowO2xlZnQ6MDt9PC9zdHlsZT4KICAgIAogICAgPHN0eWxlPiNtYXBfOWYzMWVhNWNmZmI1NGNlZDkwOWUxMDU5MzIyNTkwY2YgewogICAgICAgIHBvc2l0aW9uOiByZWxhdGl2ZTsKICAgICAgICB3aWR0aDogMTAwLjAlOwogICAgICAgIGhlaWdodDogMTAwLjAlOwogICAgICAgIGxlZnQ6IDAuMCU7CiAgICAgICAgdG9wOiAwLjAlOwogICAgICAgIH0KICAgIDwvc3R5bGU+CjwvaGVhZD4KPGJvZHk+ICAgIAogICAgCiAgICA8ZGl2IGNsYXNzPSJmb2xpdW0tbWFwIiBpZD0ibWFwXzlmMzFlYTVjZmZiNTRjZWQ5MDllMTA1OTMyMjU5MGNmIiA+PC9kaXY+CjwvYm9keT4KPHNjcmlwdD4gICAgCiAgICAKICAgIAogICAgICAgIHZhciBib3VuZHMgPSBudWxsOwogICAgCgogICAgdmFyIG1hcF85ZjMxZWE1Y2ZmYjU0Y2VkOTA5ZTEwNTkzMjI1OTBjZiA9IEwubWFwKAogICAgICAgICdtYXBfOWYzMWVhNWNmZmI1NGNlZDkwOWUxMDU5MzIyNTkwY2YnLCB7CiAgICAgICAgY2VudGVyOiBbMjQuNjg1NTM5MjQ5OTk5OTk4LCAtODEuNTY1MzU5MjVdLAogICAgICAgIHpvb206IDEwLAogICAgICAgIG1heEJvdW5kczogYm91bmRzLAogICAgICAgIGxheWVyczogW10sCiAgICAgICAgd29ybGRDb3B5SnVtcDogZmFsc2UsCiAgICAgICAgY3JzOiBMLkNSUy5FUFNHMzg1NywKICAgICAgICB6b29tQ29udHJvbDogdHJ1ZSwKICAgICAgICB9KTsKCiAgICAKICAgIAogICAgdmFyIHRpbGVfbGF5ZXJfYjY3N2E3Y2Q2NmYxNGQyY2FhYTg2YmNlYzMwMzZhNGYgPSBMLnRpbGVMYXllcigKICAgICAgICAnaHR0cHM6Ly97c30udGlsZS5vcGVuc3RyZWV0bWFwLm9yZy97en0ve3h9L3t5fS5wbmcnLAogICAgICAgIHsKICAgICAgICAiYXR0cmlidXRpb24iOiBudWxsLAogICAgICAgICJkZXRlY3RSZXRpbmEiOiBmYWxzZSwKICAgICAgICAibWF4TmF0aXZlWm9vbSI6IDE4LAogICAgICAgICJtYXhab29tIjogMTgsCiAgICAgICAgIm1pblpvb20iOiAwLAogICAgICAgICJub1dyYXAiOiBmYWxzZSwKICAgICAgICAic3ViZG9tYWlucyI6ICJhYmMiCn0pLmFkZFRvKG1hcF85ZjMxZWE1Y2ZmYjU0Y2VkOTA5ZTEwNTkzMjI1OTBjZik7CiAgICAKICAgICAgICAKICAgICAgICB2YXIgZ2VvX2pzb25fOTE5ZGFkODhkYWIyNDE2YTljNWQwM2ExZjNjYTNjMGEgPSBMLmdlb0pzb24oCiAgICAgICAgICAgIHsiYmJveCI6IFstODIuMjQ0OTQ1NTQyNTU4NCwgMjQuNDc0MDI5NzEwNzE0NTU3LCAtODEuMTQwODU1NTM0MTIxNjcsIDI0Ljk1MzkxODU5MzIwNDk5Nl0sICJmZWF0dXJlcyI6IFt7ImJib3giOiBbLTgxLjUxMTcyNDY4ODYwNjkyLCAyNC44NDk2NDQ4NzYyNDU1NzMsIC04MS40MDAyMDk5Mzc4MzM0NSwgMjQuOTUzOTE4NTkzMjA0OTk2XSwgImdlb21ldHJ5IjogeyJjb29yZGluYXRlcyI6IFtbWy04MS40OTY4NzA0MDE3MDg2NSwgMjQuOTUzOTE4NTkzMjA0OTk2XSwgWy04MS40MDAyMDk5Mzc4MzM0NSwgMjQuOTM5OTc5MDE4OTg2ODldLCBbLTgxLjQxNTE2MDEwNTk2MTAyLCAyNC44NDk2NDQ4NzYyNDU1NzNdLCBbLTgxLjUxMTcyNDY4ODYwNjkyLCAyNC44NjM1NzQ0NzYwNTAyMjVdLCBbLTgxLjQ5Njg3MDQwMTcwODY1LCAyNC45NTM5MTg1OTMyMDQ5OTZdXV0sICJ0eXBlIjogIlBvbHlnb24ifSwgImlkIjogIjAiLCAicHJvcGVydGllcyI6IHsiQ09OVVNfMTBLTSI6IDIyMzcsICJHUlRTX0lEIjogMTc2OCwgImhpZ2hsaWdodCI6IHt9LCAibGF0IjogMjQuOTAxNzk0LCAibG9uZyI6IC04MS40NTU5OTEsICJwcm9qZWN0IjogImZvbyIsICJzcGVjaWVzIjogIk1ZTFUiLCAic3R5bGUiOiB7fX0sICJ0eXBlIjogIkZlYXR1cmUifSwgeyJiYm94IjogWy04MS4yNTIzMTM0MTg2MDc5NywgMjQuNjI2NjA0ODQxMzIyOTcsIC04MS4xNDA4NTU1MzQxMjE2NywgMjQuNzMxMTY0ODA1MzYyMDAzXSwgImdlb21ldHJ5IjogeyJjb29yZGluYXRlcyI6IFtbWy04MS4yMzcyMzMxMDM0MDc1NiwgMjQuNzMxMTY0ODA1MzYyMDAzXSwgWy04MS4xNDA4NTU1MzQxMjE2NywgMjQuNzE2OTY4MDcwNjU3NTkzXSwgWy04MS4xNTYwMzEwODU1MzM2MywgMjQuNjI2NjA0ODQxMzIyOTddLCBbLTgxLjI1MjMxMzQxODYwNzk3LCAyNC42NDA3OTE1MTg0Mjc0NV0sIFstODEuMjM3MjMzMTAzNDA3NTYsIDI0LjczMTE2NDgwNTM2MjAwM11dXSwgInR5cGUiOiAiUG9seWdvbiJ9LCAiaWQiOiAiMSIsICJwcm9wZXJ0aWVzIjogeyJDT05VU18xMEtNIjogMTMxNCwgIkdSVFNfSUQiOiA5OTYwLCAiaGlnaGxpZ2h0Ijoge30sICJsYXQiOiAyNC42Nzg4OTgsICJsb25nIjogLTgxLjE5NjYwOCwgInByb2plY3QiOiAiZm9vIiwgInNwZWNpZXMiOiAiRVBGVSIsICJzdHlsZSI6IHt9fSwgInR5cGUiOiAiRmVhdHVyZSJ9LCB7ImJib3giOiBbLTgyLjI0NDk0NTU0MjU1ODQsIDI0LjU4MzMwNTEwNzg4NDA4MywgLTgyLjEzNDM5NjQ1NTAwMDU0LCAyNC42ODcwNzg2NTk1NzMzN10sICJnZW9tZXRyeSI6IHsiY29vcmRpbmF0ZXMiOiBbW1stODIuMjMwODc5ODUxNTIwNSwgMjQuNjg3MDc4NjU5NTczMzddLCBbLTgyLjEzNDM5NjQ1NTAwMDU0LCAyNC42NzM4MjU0MDExNTY1M10sIFstODIuMTQ4NTU3OTA4NDkyODgsIDI0LjU4MzMwNTEwNzg4NDA4M10sIFstODIuMjQ0OTQ1NTQyNTU4NCwgMjQuNTk2NTQ4OTc5ODYzMDEyXSwgWy04Mi4yMzA4Nzk4NTE1MjA1LCAyNC42ODcwNzg2NTk1NzMzN11dXSwgInR5cGUiOiAiUG9seWdvbiJ9LCAiaWQiOiAiMiIsICJwcm9wZXJ0aWVzIjogeyJDT05VU18xMEtNIjogMzc4LCAiR1JUU19JRCI6IDMzMTI4LCAiaGlnaGxpZ2h0Ijoge30sICJsYXQiOiAyNC42MzUyMDUsICJsb25nIjogLTgyLjE4OTY5NSwgInByb2plY3QiOiAiZm9vIiwgInNwZWNpZXMiOiAiRVBGVSIsICJzdHlsZSI6IHt9fSwgInR5cGUiOiAiRmVhdHVyZSJ9LCB7ImJib3giOiBbLTgxLjQ3NDY1ODIyNjc1ODkzLCAyNC40NzQwMjk3MTA3MTQ1NTcsIC04MS4zNjM1ODA5MjE3OTAwNSwgMjQuNTc4NDY1MDMwMTg4OTg3XSwgImdlb21ldHJ5IjogeyJjb29yZGluYXRlcyI6IFtbWy04MS40NTk4Mjg4OTY5MDI1NSwgMjQuNTc4NDY1MDMwMTg4OTg3XSwgWy04MS4zNjM1ODA5MjE3OTAwNSwgMjQuNTY0NDcyODEzNTUzMDhdLCBbLTgxLjM3ODUwNTMwNDc2NzU5LCAyNC40NzQwMjk3MTA3MTQ1NTddLCBbLTgxLjQ3NDY1ODIyNjc1ODkzLCAyNC40ODgwMTIwNzU5OTEyMjVdLCBbLTgxLjQ1OTgyODg5NjkwMjU1LCAyNC41Nzg0NjUwMzAxODg5ODddXV0sICJ0eXBlIjogIlBvbHlnb24ifSwgImlkIjogIjMiLCAicHJvcGVydGllcyI6IHsiQ09OVVNfMTBLTSI6IDM4NiwgIkdSVFNfSUQiOiAzNzIyNCwgImhpZ2hsaWdodCI6IHt9LCAibGF0IjogMjQuNTI2MjYsICJsb25nIjogLTgxLjQxOTE0MywgInByb2plY3QiOiAiYmFyIiwgInNwZWNpZXMiOiAiTVlMVSIsICJzdHlsZSI6IHt9fSwgInR5cGUiOiAiRmVhdHVyZSJ9XSwgInR5cGUiOiAiRmVhdHVyZUNvbGxlY3Rpb24ifQogICAgICAgICAgICAKICAgICAgICAgICAgKS5hZGRUbyhtYXBfOWYzMWVhNWNmZmI1NGNlZDkwOWUxMDU5MzIyNTkwY2YpOwogICAgICAgIGdlb19qc29uXzkxOWRhZDg4ZGFiMjQxNmE5YzVkMDNhMWYzY2EzYzBhLnNldFN0eWxlKGZ1bmN0aW9uKGZlYXR1cmUpIHtyZXR1cm4gZmVhdHVyZS5wcm9wZXJ0aWVzLnN0eWxlO30pOwogICAgICAgIAo8L3NjcmlwdD4=\" style=\"position:absolute;width:100%;height:100%;left:0;top:0;border:none !important;\" allowfullscreen webkitallowfullscreen mozallowfullscreen></iframe></div></div>"
],
"text/plain": [
"<folium.folium.Map at 0x18dd812ffd0>"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import folium\n",
"m = folium.Map([merged.lat.mean(), merged.long.mean()])\n",
"folium.GeoJson(merged.to_crs(epsg='4326')).add_to(m)\n",
"\n",
"m"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment