Created
April 14, 2021 16:51
-
-
Save barangerbenjamin/1f7f37fb5d937db97d59954a78695b3f to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"cells": [ | |
{ | |
"cell_type": "markdown", | |
"metadata": {}, | |
"source": [ | |
"# Reboot - Data Collection\n", | |
"\n", | |
"Let's practise calling an API and navigating a JSON response!\n", | |
"\n", | |
"---\n", | |
"\n", | |
"## 1. Simple API call\n", | |
"\n", | |
"There's an API for pretty much everything. Let's start with a super simple one: the [Breaking Bad quotes API](https://github.com/shevabam/breaking-bad-quotes).\n", | |
"\n", | |
"For your convenience, we've included the simple syntax to do an HTTP request in Python and convert the response to a Python object as boilerplate code.\n", | |
"\n", | |
"The goal here is to get a single, random Breaking Bad quote and print it out to the terminal. \n", | |
"\n", | |
"Go the [doc](https://github.com/shevabam/breaking-bad-quotes) to find out which URL you need to use. Make sure to carefully check out the data types you're getting back from the API and extract the string you need." | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 1, | |
"metadata": {}, | |
"outputs": [], | |
"source": [ | |
"import requests" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 8, | |
"metadata": {}, | |
"outputs": [ | |
{ | |
"data": { | |
"text/plain": [ | |
"'Jesse Pinkman said Seriously? \"Hello Kitty\"?'" | |
] | |
}, | |
"execution_count": 8, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"url = \"https://breaking-bad-quotes.herokuapp.com/v1/quotes\"\n", | |
"response = requests.get(url).json()[0]\n", | |
"f\"{response['author']} said {response['quote']}\"" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": {}, | |
"source": [ | |
"---\n", | |
"## 2. Looping over API results\n", | |
"\n", | |
"Often APIs will serve a list of results that we need to loop over and manipulate. For this next example, we will use the [Star Wars API](https://swapi.dev).\n", | |
"\n", | |
"First, go to the [documentation](https://swapi.dev/documentation) to find out which URL you're going to need to retrieve" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 9, | |
"metadata": {}, | |
"outputs": [], | |
"source": [ | |
"base_url = \"https://swapi.dev/api/\"" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": {}, | |
"source": [ | |
"Use `requests` to retrieve the people from the API. Store this in a `response` variable. What's the type of `response`?" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 11, | |
"metadata": {}, | |
"outputs": [ | |
{ | |
"data": { | |
"text/plain": [ | |
"dict" | |
] | |
}, | |
"execution_count": 11, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"url = base_url + \"people\"\n", | |
"response = requests.get(url).json()\n", | |
"type(response)" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"slideshow": { | |
"slide_type": "slide" | |
} | |
}, | |
"source": [ | |
"Loop over the **5 first people** and print a sentence for each following this template:\n", | |
"\n", | |
"- `Luke Skywalker has blue eyes`\n", | |
"- `C-3PO has yellow eyes`\n", | |
"- etc." | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 17, | |
"metadata": {}, | |
"outputs": [ | |
{ | |
"name": "stdout", | |
"output_type": "stream", | |
"text": [ | |
"Luke Skywalker has blue eyes\n", | |
"C-3PO has yellow eyes\n", | |
"R2-D2 has red eyes\n", | |
"Darth Vader has yellow eyes\n", | |
"Leia Organa has brown eyes\n" | |
] | |
} | |
], | |
"source": [ | |
"characters = response['results'][:5]\n", | |
"for character in characters:\n", | |
" name = character['name']\n", | |
" eye_color = character[\"eye_color\"]\n", | |
" print(f\"{name} has {eye_color} eyes\")" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": {}, | |
"source": [ | |
"---\n", | |
"## 3. Geocoding\n", | |
"\n", | |
"A geocoding service is a tool to convert addresses to geo-coordinates and vice versa. We use them every time we fire up our GPS apps and look for a destination.\n", | |
"\n", | |
"For this exercise, we will use the [Nominatim API](https://nominatim.openstreetmap.org/)\n", | |
"\n", | |
"First, get familiar with the documentation, [which you will find here](https://nominatim.org/release-docs/latest/api/Overview/). Take a couple of minutes to read through and become familiar with it. In this exercise, we will need to find a set of geo-coordinates for a given address, and find an address from a set of geo-coordinates." | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": {}, | |
"source": [ | |
"Use the API to find the latitude and longitude for this address: `10 Downing St, Westminster, London SW1A 2AA, United Kingdom`." | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 19, | |
"metadata": {}, | |
"outputs": [ | |
{ | |
"name": "stdout", | |
"output_type": "stream", | |
"text": [ | |
"51.50344025 -0.12770820958562096\n" | |
] | |
} | |
], | |
"source": [ | |
"address = \"10 Downing St, Westminster, London SW1A 2AA, United Kingdom\"\n", | |
"url = \"https://nominatim.openstreetmap.org/search\"\n", | |
"params = {\n", | |
" \"q\": address,\n", | |
" 'format': 'json'\n", | |
"}\n", | |
"response = requests.get(url, params=params).json()[0]\n", | |
"print(response['lat'], response['lon'])" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": {}, | |
"source": [ | |
"Try with another address of your choice!" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 20, | |
"metadata": {}, | |
"outputs": [], | |
"source": [ | |
"def coordinates(address):\n", | |
" url = \"https://nominatim.openstreetmap.org/search\"\n", | |
" params = {\n", | |
" \"q\": address,\n", | |
" 'format': 'json'\n", | |
" }\n", | |
" response = requests.get(url, params=params).json()[0]\n", | |
" lat = response['lat']\n", | |
" lon = response['lon']\n", | |
" return (lat, lon)" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 21, | |
"metadata": {}, | |
"outputs": [ | |
{ | |
"name": "stdout", | |
"output_type": "stream", | |
"text": [ | |
"('51.531662', '-0.0770283')\n" | |
] | |
} | |
], | |
"source": [ | |
"print(coordinates(\"138 kingsland road, e2 8dy, london\"))" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": {}, | |
"source": [ | |
"---\n", | |
"## 4. Reverse Geocoding\n", | |
"\n", | |
"Find the address belonging to this set of geo-coordinates: `{'lat': 38.8976633, 'lng': -77.036650 }`.\n", | |
"Print the sentence `I would like to visit X`, replacing X with the address you found." | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 25, | |
"metadata": {}, | |
"outputs": [ | |
{ | |
"data": { | |
"text/plain": [ | |
"'White House, 1600, Pennsylvania Avenue Northwest, Washington, District of Columbia, 20500, United States'" | |
] | |
}, | |
"execution_count": 25, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"coordinates = {'lat': 38.8976633, 'lng': -77.036650 }\n" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 26, | |
"metadata": {}, | |
"outputs": [], | |
"source": [ | |
"def reverse_coordinates(coordinates):\n", | |
" url = \"https://nominatim.openstreetmap.org/reverse\"\n", | |
" params = {\n", | |
" \"lat\": coordinates['lat'],\n", | |
" \"lon\": coordinates['lng'],\n", | |
" 'format': 'json'\n", | |
" }\n", | |
" response = requests.get(url, params=params).json()\n", | |
" return response['display_name']" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 27, | |
"metadata": {}, | |
"outputs": [ | |
{ | |
"name": "stdout", | |
"output_type": "stream", | |
"text": [ | |
"Museum of the Home, 136, Kingsland Road, Shoreditch, London Borough of Hackney, London, Greater London, England, E2 8EA, United Kingdom\n" | |
] | |
} | |
], | |
"source": [ | |
"print(reverse_coordinates({'lat': 51.531662, 'lng': -0.0770283}))" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": null, | |
"metadata": {}, | |
"outputs": [], | |
"source": [] | |
} | |
], | |
"metadata": { | |
"kernelspec": { | |
"display_name": "Python 3", | |
"language": "python", | |
"name": "python3" | |
}, | |
"language_info": { | |
"codemirror_mode": { | |
"name": "ipython", | |
"version": 3 | |
}, | |
"file_extension": ".py", | |
"mimetype": "text/x-python", | |
"name": "python", | |
"nbconvert_exporter": "python", | |
"pygments_lexer": "ipython3", | |
"version": "3.8.6" | |
}, | |
"toc": { | |
"base_numbering": 1, | |
"nav_menu": {}, | |
"number_sections": false, | |
"sideBar": true, | |
"skip_h1_title": false, | |
"title_cell": "Table of Contents", | |
"title_sidebar": "Contents", | |
"toc_cell": false, | |
"toc_position": {}, | |
"toc_section_display": true, | |
"toc_window_display": false | |
}, | |
"varInspector": { | |
"cols": { | |
"lenName": 16, | |
"lenType": 16, | |
"lenVar": 40 | |
}, | |
"kernels_config": { | |
"python": { | |
"delete_cmd_postfix": "", | |
"delete_cmd_prefix": "del ", | |
"library": "var_list.py", | |
"varRefreshCmd": "print(var_dic_list())" | |
}, | |
"r": { | |
"delete_cmd_postfix": ") ", | |
"delete_cmd_prefix": "rm(", | |
"library": "var_list.r", | |
"varRefreshCmd": "cat(var_dic_list()) " | |
} | |
}, | |
"types_to_exclude": [ | |
"module", | |
"function", | |
"builtin_function_or_method", | |
"instance", | |
"_Feature" | |
], | |
"window_display": false | |
} | |
}, | |
"nbformat": 4, | |
"nbformat_minor": 4 | |
} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment