Skip to content

Instantly share code, notes, and snippets.

@barangerbenjamin
Created April 14, 2021 16:51
Show Gist options
  • Save barangerbenjamin/1f7f37fb5d937db97d59954a78695b3f to your computer and use it in GitHub Desktop.
Save barangerbenjamin/1f7f37fb5d937db97d59954a78695b3f to your computer and use it in GitHub Desktop.
Display the source blob
Display the rendered blob
Raw
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Reboot - Data Collection\n",
"\n",
"Let's practise calling an API and navigating a JSON response!\n",
"\n",
"---\n",
"\n",
"## 1. Simple API call\n",
"\n",
"There's an API for pretty much everything. Let's start with a super simple one: the [Breaking Bad quotes API](https://github.com/shevabam/breaking-bad-quotes).\n",
"\n",
"For your convenience, we've included the simple syntax to do an HTTP request in Python and convert the response to a Python object as boilerplate code.\n",
"\n",
"The goal here is to get a single, random Breaking Bad quote and print it out to the terminal. \n",
"\n",
"Go the [doc](https://github.com/shevabam/breaking-bad-quotes) to find out which URL you need to use. Make sure to carefully check out the data types you're getting back from the API and extract the string you need."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import requests"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'Jesse Pinkman said Seriously? \"Hello Kitty\"?'"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"url = \"https://breaking-bad-quotes.herokuapp.com/v1/quotes\"\n",
"response = requests.get(url).json()[0]\n",
"f\"{response['author']} said {response['quote']}\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"## 2. Looping over API results\n",
"\n",
"Often APIs will serve a list of results that we need to loop over and manipulate. For this next example, we will use the [Star Wars API](https://swapi.dev).\n",
"\n",
"First, go to the [documentation](https://swapi.dev/documentation) to find out which URL you're going to need to retrieve"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"base_url = \"https://swapi.dev/api/\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Use `requests` to retrieve the people from the API. Store this in a `response` variable. What's the type of `response`?"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"dict"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"url = base_url + \"people\"\n",
"response = requests.get(url).json()\n",
"type(response)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Loop over the **5 first people** and print a sentence for each following this template:\n",
"\n",
"- `Luke Skywalker has blue eyes`\n",
"- `C-3PO has yellow eyes`\n",
"- etc."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Luke Skywalker has blue eyes\n",
"C-3PO has yellow eyes\n",
"R2-D2 has red eyes\n",
"Darth Vader has yellow eyes\n",
"Leia Organa has brown eyes\n"
]
}
],
"source": [
"characters = response['results'][:5]\n",
"for character in characters:\n",
" name = character['name']\n",
" eye_color = character[\"eye_color\"]\n",
" print(f\"{name} has {eye_color} eyes\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"## 3. Geocoding\n",
"\n",
"A geocoding service is a tool to convert addresses to geo-coordinates and vice versa. We use them every time we fire up our GPS apps and look for a destination.\n",
"\n",
"For this exercise, we will use the [Nominatim API](https://nominatim.openstreetmap.org/)\n",
"\n",
"First, get familiar with the documentation, [which you will find here](https://nominatim.org/release-docs/latest/api/Overview/). Take a couple of minutes to read through and become familiar with it. In this exercise, we will need to find a set of geo-coordinates for a given address, and find an address from a set of geo-coordinates."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Use the API to find the latitude and longitude for this address: `10 Downing St, Westminster, London SW1A 2AA, United Kingdom`."
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"51.50344025 -0.12770820958562096\n"
]
}
],
"source": [
"address = \"10 Downing St, Westminster, London SW1A 2AA, United Kingdom\"\n",
"url = \"https://nominatim.openstreetmap.org/search\"\n",
"params = {\n",
" \"q\": address,\n",
" 'format': 'json'\n",
"}\n",
"response = requests.get(url, params=params).json()[0]\n",
"print(response['lat'], response['lon'])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Try with another address of your choice!"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [],
"source": [
"def coordinates(address):\n",
" url = \"https://nominatim.openstreetmap.org/search\"\n",
" params = {\n",
" \"q\": address,\n",
" 'format': 'json'\n",
" }\n",
" response = requests.get(url, params=params).json()[0]\n",
" lat = response['lat']\n",
" lon = response['lon']\n",
" return (lat, lon)"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"('51.531662', '-0.0770283')\n"
]
}
],
"source": [
"print(coordinates(\"138 kingsland road, e2 8dy, london\"))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"## 4. Reverse Geocoding\n",
"\n",
"Find the address belonging to this set of geo-coordinates: `{'lat': 38.8976633, 'lng': -77.036650 }`.\n",
"Print the sentence `I would like to visit X`, replacing X with the address you found."
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'White House, 1600, Pennsylvania Avenue Northwest, Washington, District of Columbia, 20500, United States'"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"coordinates = {'lat': 38.8976633, 'lng': -77.036650 }\n"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [],
"source": [
"def reverse_coordinates(coordinates):\n",
" url = \"https://nominatim.openstreetmap.org/reverse\"\n",
" params = {\n",
" \"lat\": coordinates['lat'],\n",
" \"lon\": coordinates['lng'],\n",
" 'format': 'json'\n",
" }\n",
" response = requests.get(url, params=params).json()\n",
" return response['display_name']"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Museum of the Home, 136, Kingsland Road, Shoreditch, London Borough of Hackney, London, Greater London, England, E2 8EA, United Kingdom\n"
]
}
],
"source": [
"print(reverse_coordinates({'lat': 51.531662, 'lng': -0.0770283}))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.6"
},
"toc": {
"base_numbering": 1,
"nav_menu": {},
"number_sections": false,
"sideBar": true,
"skip_h1_title": false,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": false,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": false
},
"varInspector": {
"cols": {
"lenName": 16,
"lenType": 16,
"lenVar": 40
},
"kernels_config": {
"python": {
"delete_cmd_postfix": "",
"delete_cmd_prefix": "del ",
"library": "var_list.py",
"varRefreshCmd": "print(var_dic_list())"
},
"r": {
"delete_cmd_postfix": ") ",
"delete_cmd_prefix": "rm(",
"library": "var_list.r",
"varRefreshCmd": "cat(var_dic_list()) "
}
},
"types_to_exclude": [
"module",
"function",
"builtin_function_or_method",
"instance",
"_Feature"
],
"window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 4
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment