Created
June 7, 2020 12:18
-
-
Save adityantamarapu/4e5ab38f1df38f648c5b154db22907d3 to your computer and use it in GitHub Desktop.
Created on Skills Network Labs
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"cells": [ | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"<a href=\"https://cognitiveclass.ai\"><img src = \"https://ibm.box.com/shared/static/9gegpsmnsoo25ikkbl4qzlvlyjbgxs5x.png\" width = 400> </a>\n", | |
"\n", | |
"<h1 align=center><font size = 5>Learning FourSquare API with Python</font></h1>" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
" " | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"## Introduction\n", | |
"\n", | |
"In this lab, you will learn in details how to make calls to the Foursquare API for different purposes. You will learn how to construct a URL to send a request to the API to search for a specific type of venues, to explore a particular venue, to explore a Foursquare user, to explore a geographical location, and to get trending venues around a location. Also, you will learn how to use the visualization library, Folium, to visualize the results." | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"## Table of Contents\n", | |
"\n", | |
"1. <a href=\"#item1\">Foursquare API Search Function</a>\n", | |
"2. <a href=\"#item2\">Explore a Given Venue</a> \n", | |
"3. <a href=\"#item3\">Explore a User</a> \n", | |
"4. <a href=\"#item4\">Foursquare API Explore Function</a> \n", | |
"5. <a href=\"#item5\">Get Trending Venues</a> " | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"### Import necessary Libraries" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 1, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"name": "stdout", | |
"output_type": "stream", | |
"text": [ | |
"Collecting package metadata (current_repodata.json): done\n", | |
"Solving environment: done\n", | |
"\n", | |
"## Package Plan ##\n", | |
"\n", | |
" environment location: /home/jupyterlab/conda/envs/python\n", | |
"\n", | |
" added / updated specs:\n", | |
" - geopy\n", | |
"\n", | |
"\n", | |
"The following packages will be downloaded:\n", | |
"\n", | |
" package | build\n", | |
" ---------------------------|-----------------\n", | |
" geographiclib-1.50 | py_0 34 KB conda-forge\n", | |
" geopy-1.22.0 | pyh9f0ad1d_0 63 KB conda-forge\n", | |
" ------------------------------------------------------------\n", | |
" Total: 97 KB\n", | |
"\n", | |
"The following NEW packages will be INSTALLED:\n", | |
"\n", | |
" geographiclib conda-forge/noarch::geographiclib-1.50-py_0\n", | |
" geopy conda-forge/noarch::geopy-1.22.0-pyh9f0ad1d_0\n", | |
"\n", | |
"\n", | |
"\n", | |
"Downloading and Extracting Packages\n", | |
"geopy-1.22.0 | 63 KB | ##################################### | 100% \n", | |
"geographiclib-1.50 | 34 KB | ##################################### | 100% \n", | |
"Preparing transaction: done\n", | |
"Verifying transaction: done\n", | |
"Executing transaction: done\n", | |
"Collecting package metadata (current_repodata.json): done\n", | |
"Solving environment: failed with initial frozen solve. Retrying with flexible solve.\n", | |
"Collecting package metadata (repodata.json): done\n", | |
"Solving environment: done\n", | |
"\n", | |
"## Package Plan ##\n", | |
"\n", | |
" environment location: /home/jupyterlab/conda/envs/python\n", | |
"\n", | |
" added / updated specs:\n", | |
" - folium=0.5.0\n", | |
"\n", | |
"\n", | |
"The following packages will be downloaded:\n", | |
"\n", | |
" package | build\n", | |
" ---------------------------|-----------------\n", | |
" altair-4.1.0 | py_1 614 KB conda-forge\n", | |
" branca-0.4.1 | py_0 26 KB conda-forge\n", | |
" brotlipy-0.7.0 |py36h8c4c3a4_1000 346 KB conda-forge\n", | |
" chardet-3.0.4 |py36h9f0ad1d_1006 188 KB conda-forge\n", | |
" cryptography-2.9.2 | py36h45558ae_0 613 KB conda-forge\n", | |
" folium-0.5.0 | py_0 45 KB conda-forge\n", | |
" pandas-1.0.4 | py36h830a2c2_0 10.1 MB conda-forge\n", | |
" pysocks-1.7.1 | py36h9f0ad1d_1 27 KB conda-forge\n", | |
" toolz-0.10.0 | py_0 46 KB conda-forge\n", | |
" vincent-0.4.4 | py_1 28 KB conda-forge\n", | |
" ------------------------------------------------------------\n", | |
" Total: 12.0 MB\n", | |
"\n", | |
"The following NEW packages will be INSTALLED:\n", | |
"\n", | |
" altair conda-forge/noarch::altair-4.1.0-py_1\n", | |
" attrs conda-forge/noarch::attrs-19.3.0-py_0\n", | |
" branca conda-forge/noarch::branca-0.4.1-py_0\n", | |
" brotlipy conda-forge/linux-64::brotlipy-0.7.0-py36h8c4c3a4_1000\n", | |
" chardet conda-forge/linux-64::chardet-3.0.4-py36h9f0ad1d_1006\n", | |
" cryptography conda-forge/linux-64::cryptography-2.9.2-py36h45558ae_0\n", | |
" entrypoints conda-forge/linux-64::entrypoints-0.3-py36h9f0ad1d_1001\n", | |
" folium conda-forge/noarch::folium-0.5.0-py_0\n", | |
" idna conda-forge/noarch::idna-2.9-py_1\n", | |
" importlib_metadata conda-forge/noarch::importlib_metadata-1.6.0-0\n", | |
" jinja2 conda-forge/noarch::jinja2-2.11.2-pyh9f0ad1d_0\n", | |
" jsonschema conda-forge/linux-64::jsonschema-3.2.0-py36h9f0ad1d_1\n", | |
" markupsafe conda-forge/linux-64::markupsafe-1.1.1-py36h8c4c3a4_1\n", | |
" pandas conda-forge/linux-64::pandas-1.0.4-py36h830a2c2_0\n", | |
" pyopenssl conda-forge/noarch::pyopenssl-19.1.0-py_1\n", | |
" pyrsistent conda-forge/linux-64::pyrsistent-0.16.0-py36h8c4c3a4_0\n", | |
" pysocks conda-forge/linux-64::pysocks-1.7.1-py36h9f0ad1d_1\n", | |
" pytz conda-forge/noarch::pytz-2020.1-pyh9f0ad1d_0\n", | |
" requests conda-forge/noarch::requests-2.23.0-pyh8c360ce_2\n", | |
" toolz conda-forge/noarch::toolz-0.10.0-py_0\n", | |
" urllib3 conda-forge/noarch::urllib3-1.25.9-py_0\n", | |
" vincent conda-forge/noarch::vincent-0.4.4-py_1\n", | |
"\n", | |
"\n", | |
"\n", | |
"Downloading and Extracting Packages\n", | |
"pysocks-1.7.1 | 27 KB | ##################################### | 100% \n", | |
"toolz-0.10.0 | 46 KB | ##################################### | 100% \n", | |
"chardet-3.0.4 | 188 KB | ##################################### | 100% \n", | |
"pandas-1.0.4 | 10.1 MB | ##################################### | 100% \n", | |
"folium-0.5.0 | 45 KB | ##################################### | 100% \n", | |
"branca-0.4.1 | 26 KB | ##################################### | 100% \n", | |
"cryptography-2.9.2 | 613 KB | ##################################### | 100% \n", | |
"brotlipy-0.7.0 | 346 KB | ##################################### | 100% \n", | |
"altair-4.1.0 | 614 KB | ##################################### | 100% \n", | |
"vincent-0.4.4 | 28 KB | ##################################### | 100% \n", | |
"Preparing transaction: done\n", | |
"Verifying transaction: done\n", | |
"Executing transaction: done\n", | |
"Folium installed\n", | |
"Libraries imported.\n" | |
] | |
} | |
], | |
"source": [ | |
"import requests # library to handle requests\n", | |
"import pandas as pd # library for data analsysis\n", | |
"import numpy as np # library to handle data in a vectorized manner\n", | |
"import random # library for random number generation\n", | |
"\n", | |
"!conda install -c conda-forge geopy --yes \n", | |
"from geopy.geocoders import Nominatim # module to convert an address into latitude and longitude values\n", | |
"\n", | |
"# libraries for displaying images\n", | |
"from IPython.display import Image \n", | |
"from IPython.core.display import HTML \n", | |
" \n", | |
"# tranforming json file into a pandas dataframe library\n", | |
"from pandas.io.json import json_normalize\n", | |
"\n", | |
"!conda install -c conda-forge folium=0.5.0 --yes\n", | |
"import folium # plotting library\n", | |
"\n", | |
"print('Folium installed')\n", | |
"print('Libraries imported.')" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"### Define Foursquare Credentials and Version" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"##### Make sure that you have created a Foursquare developer account and have your credentials handy" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 3, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"name": "stdout", | |
"output_type": "stream", | |
"text": [ | |
"Your credentails:\n", | |
"CLIENT_ID: NM2ZJ4MDHZEN140JF3CDFT5JKTFY5XKPACFFJV1LR2ORR3W0\n", | |
"CLIENT_SECRET:GYL1KCRUWNUQKY51A5E4FKOLYSAEG4C4GIO2VEFUNITJYZFC\n" | |
] | |
} | |
], | |
"source": [ | |
"CLIENT_ID = 'NM2ZJ4MDHZEN140JF3CDFT5JKTFY5XKPACFFJV1LR2ORR3W0' # your Foursquare ID\n", | |
"CLIENT_SECRET = 'GYL1KCRUWNUQKY51A5E4FKOLYSAEG4C4GIO2VEFUNITJYZFC' # your Foursquare Secret\n", | |
"VERSION = '20180604'\n", | |
"LIMIT = 30\n", | |
"print('Your credentails:')\n", | |
"print('CLIENT_ID: ' + CLIENT_ID)\n", | |
"print('CLIENT_SECRET:' + CLIENT_SECRET)" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
" " | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### Let's again assume that you are staying at the Conrad hotel. So let's start by converting the Contrad Hotel's address to its latitude and longitude coordinates." | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": {}, | |
"source": [ | |
"In order to define an instance of the geocoder, we need to define a user_agent. We will name our agent <em>foursquare_agent</em>, as shown below." | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 4, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"name": "stdout", | |
"output_type": "stream", | |
"text": [ | |
"40.7151482 -74.0156573\n" | |
] | |
} | |
], | |
"source": [ | |
"address = '102 North End Ave, New York, NY'\n", | |
"\n", | |
"geolocator = Nominatim(user_agent=\"foursquare_agent\")\n", | |
"location = geolocator.geocode(address)\n", | |
"latitude = location.latitude\n", | |
"longitude = location.longitude\n", | |
"print(latitude, longitude)" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
" " | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"<a id=\"item1\"></a>" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"## 1. Search for a specific venue category\n", | |
"> `https://api.foursquare.com/v2/venues/`**search**`?client_id=`**CLIENT_ID**`&client_secret=`**CLIENT_SECRET**`&ll=`**LATITUDE**`,`**LONGITUDE**`&v=`**VERSION**`&query=`**QUERY**`&radius=`**RADIUS**`&limit=`**LIMIT**" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### Now, let's assume that it is lunch time, and you are craving Italian food. So, let's define a query to search for Italian food that is within 500 metres from the Conrad Hotel. " | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 5, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
}, | |
"scrolled": true | |
}, | |
"outputs": [ | |
{ | |
"name": "stdout", | |
"output_type": "stream", | |
"text": [ | |
"Italian .... OK!\n" | |
] | |
} | |
], | |
"source": [ | |
"search_query = 'Italian'\n", | |
"radius = 500\n", | |
"print(search_query + ' .... OK!')" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### Define the corresponding URL" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 6, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"data": { | |
"text/plain": [ | |
"'https://api.foursquare.com/v2/venues/search?client_id=NM2ZJ4MDHZEN140JF3CDFT5JKTFY5XKPACFFJV1LR2ORR3W0&client_secret=GYL1KCRUWNUQKY51A5E4FKOLYSAEG4C4GIO2VEFUNITJYZFC&ll=40.7151482,-74.0156573&v=20180604&query=Italian&radius=500&limit=30'" | |
] | |
}, | |
"execution_count": 6, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"url = 'https://api.foursquare.com/v2/venues/search?client_id={}&client_secret={}&ll={},{}&v={}&query={}&radius={}&limit={}'.format(CLIENT_ID, CLIENT_SECRET, latitude, longitude, VERSION, search_query, radius, LIMIT)\n", | |
"url" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### Send the GET Request and examine the results" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 7, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
}, | |
"scrolled": true | |
}, | |
"outputs": [ | |
{ | |
"data": { | |
"text/plain": [ | |
"{'meta': {'code': 200, 'requestId': '5ed643c877af03001b746437'},\n", | |
" 'response': {'venues': [{'id': '4fa862b3e4b0ebff2f749f06',\n", | |
" 'name': \"Harry's Italian Pizza Bar\",\n", | |
" 'location': {'address': '225 Murray St',\n", | |
" 'lat': 40.71521779064671,\n", | |
" 'lng': -74.01473940209351,\n", | |
" 'labeledLatLngs': [{'label': 'display',\n", | |
" 'lat': 40.71521779064671,\n", | |
" 'lng': -74.01473940209351},\n", | |
" {'label': 'entrance', 'lat': 40.715361, 'lng': -74.014975}],\n", | |
" 'distance': 77,\n", | |
" 'postalCode': '10282',\n", | |
" 'cc': 'US',\n", | |
" 'city': 'New York',\n", | |
" 'state': 'NY',\n", | |
" 'country': 'United States',\n", | |
" 'formattedAddress': ['225 Murray St',\n", | |
" 'New York, NY 10282',\n", | |
" 'United States']},\n", | |
" 'categories': [{'id': '4bf58dd8d48988d1ca941735',\n", | |
" 'name': 'Pizza Place',\n", | |
" 'pluralName': 'Pizza Places',\n", | |
" 'shortName': 'Pizza',\n", | |
" 'icon': {'prefix': 'https://ss3.4sqi.net/img/categories_v2/food/pizza_',\n", | |
" 'suffix': '.png'},\n", | |
" 'primary': True}],\n", | |
" 'referralId': 'v-1591100302',\n", | |
" 'hasPerk': False},\n", | |
" {'id': '4f3232e219836c91c7bfde94',\n", | |
" 'name': 'Conca Cucina Italian Restaurant',\n", | |
" 'location': {'address': '63 W Broadway',\n", | |
" 'lat': 40.714484000000006,\n", | |
" 'lng': -74.00980600000001,\n", | |
" 'labeledLatLngs': [{'label': 'display',\n", | |
" 'lat': 40.714484000000006,\n", | |
" 'lng': -74.00980600000001}],\n", | |
" 'distance': 499,\n", | |
" 'postalCode': '10007',\n", | |
" 'cc': 'US',\n", | |
" 'city': 'New York',\n", | |
" 'state': 'NY',\n", | |
" 'country': 'United States',\n", | |
" 'formattedAddress': ['63 W Broadway',\n", | |
" 'New York, NY 10007',\n", | |
" 'United States']},\n", | |
" 'categories': [{'id': '4d4b7105d754a06374d81259',\n", | |
" 'name': 'Food',\n", | |
" 'pluralName': 'Food',\n", | |
" 'shortName': 'Food',\n", | |
" 'icon': {'prefix': 'https://ss3.4sqi.net/img/categories_v2/food/default_',\n", | |
" 'suffix': '.png'},\n", | |
" 'primary': True}],\n", | |
" 'referralId': 'v-1591100302',\n", | |
" 'hasPerk': False}]}}" | |
] | |
}, | |
"execution_count": 7, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"results = requests.get(url).json()\n", | |
"results" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### Get relevant part of JSON and transform it into a *pandas* dataframe" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 8, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"name": "stderr", | |
"output_type": "stream", | |
"text": [ | |
"/home/jupyterlab/conda/envs/python/lib/python3.6/site-packages/ipykernel_launcher.py:5: FutureWarning: pandas.io.json.json_normalize is deprecated, use pandas.json_normalize instead\n", | |
" \"\"\"\n" | |
] | |
}, | |
{ | |
"data": { | |
"text/html": [ | |
"<div>\n", | |
"<style scoped>\n", | |
" .dataframe tbody tr th:only-of-type {\n", | |
" vertical-align: middle;\n", | |
" }\n", | |
"\n", | |
" .dataframe tbody tr th {\n", | |
" vertical-align: top;\n", | |
" }\n", | |
"\n", | |
" .dataframe thead th {\n", | |
" text-align: right;\n", | |
" }\n", | |
"</style>\n", | |
"<table border=\"1\" class=\"dataframe\">\n", | |
" <thead>\n", | |
" <tr style=\"text-align: right;\">\n", | |
" <th></th>\n", | |
" <th>id</th>\n", | |
" <th>name</th>\n", | |
" <th>categories</th>\n", | |
" <th>referralId</th>\n", | |
" <th>hasPerk</th>\n", | |
" <th>location.address</th>\n", | |
" <th>location.lat</th>\n", | |
" <th>location.lng</th>\n", | |
" <th>location.labeledLatLngs</th>\n", | |
" <th>location.distance</th>\n", | |
" <th>location.postalCode</th>\n", | |
" <th>location.cc</th>\n", | |
" <th>location.city</th>\n", | |
" <th>location.state</th>\n", | |
" <th>location.country</th>\n", | |
" <th>location.formattedAddress</th>\n", | |
" </tr>\n", | |
" </thead>\n", | |
" <tbody>\n", | |
" <tr>\n", | |
" <th>0</th>\n", | |
" <td>4fa862b3e4b0ebff2f749f06</td>\n", | |
" <td>Harry's Italian Pizza Bar</td>\n", | |
" <td>[{'id': '4bf58dd8d48988d1ca941735', 'name': 'P...</td>\n", | |
" <td>v-1591100302</td>\n", | |
" <td>False</td>\n", | |
" <td>225 Murray St</td>\n", | |
" <td>40.715218</td>\n", | |
" <td>-74.014739</td>\n", | |
" <td>[{'label': 'display', 'lat': 40.71521779064671...</td>\n", | |
" <td>77</td>\n", | |
" <td>10282</td>\n", | |
" <td>US</td>\n", | |
" <td>New York</td>\n", | |
" <td>NY</td>\n", | |
" <td>United States</td>\n", | |
" <td>[225 Murray St, New York, NY 10282, United Sta...</td>\n", | |
" </tr>\n", | |
" <tr>\n", | |
" <th>1</th>\n", | |
" <td>4f3232e219836c91c7bfde94</td>\n", | |
" <td>Conca Cucina Italian Restaurant</td>\n", | |
" <td>[{'id': '4d4b7105d754a06374d81259', 'name': 'F...</td>\n", | |
" <td>v-1591100302</td>\n", | |
" <td>False</td>\n", | |
" <td>63 W Broadway</td>\n", | |
" <td>40.714484</td>\n", | |
" <td>-74.009806</td>\n", | |
" <td>[{'label': 'display', 'lat': 40.71448400000000...</td>\n", | |
" <td>499</td>\n", | |
" <td>10007</td>\n", | |
" <td>US</td>\n", | |
" <td>New York</td>\n", | |
" <td>NY</td>\n", | |
" <td>United States</td>\n", | |
" <td>[63 W Broadway, New York, NY 10007, United Sta...</td>\n", | |
" </tr>\n", | |
" </tbody>\n", | |
"</table>\n", | |
"</div>" | |
], | |
"text/plain": [ | |
" id name \\\n", | |
"0 4fa862b3e4b0ebff2f749f06 Harry's Italian Pizza Bar \n", | |
"1 4f3232e219836c91c7bfde94 Conca Cucina Italian Restaurant \n", | |
"\n", | |
" categories referralId hasPerk \\\n", | |
"0 [{'id': '4bf58dd8d48988d1ca941735', 'name': 'P... v-1591100302 False \n", | |
"1 [{'id': '4d4b7105d754a06374d81259', 'name': 'F... v-1591100302 False \n", | |
"\n", | |
" location.address location.lat location.lng \\\n", | |
"0 225 Murray St 40.715218 -74.014739 \n", | |
"1 63 W Broadway 40.714484 -74.009806 \n", | |
"\n", | |
" location.labeledLatLngs location.distance \\\n", | |
"0 [{'label': 'display', 'lat': 40.71521779064671... 77 \n", | |
"1 [{'label': 'display', 'lat': 40.71448400000000... 499 \n", | |
"\n", | |
" location.postalCode location.cc location.city location.state \\\n", | |
"0 10282 US New York NY \n", | |
"1 10007 US New York NY \n", | |
"\n", | |
" location.country location.formattedAddress \n", | |
"0 United States [225 Murray St, New York, NY 10282, United Sta... \n", | |
"1 United States [63 W Broadway, New York, NY 10007, United Sta... " | |
] | |
}, | |
"execution_count": 8, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"# assign relevant part of JSON to venues\n", | |
"venues = results['response']['venues']\n", | |
"\n", | |
"# tranform venues into a dataframe\n", | |
"dataframe = json_normalize(venues)\n", | |
"dataframe.head()" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### Define information of interest and filter dataframe" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 9, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
}, | |
"scrolled": true | |
}, | |
"outputs": [ | |
{ | |
"data": { | |
"text/html": [ | |
"<div>\n", | |
"<style scoped>\n", | |
" .dataframe tbody tr th:only-of-type {\n", | |
" vertical-align: middle;\n", | |
" }\n", | |
"\n", | |
" .dataframe tbody tr th {\n", | |
" vertical-align: top;\n", | |
" }\n", | |
"\n", | |
" .dataframe thead th {\n", | |
" text-align: right;\n", | |
" }\n", | |
"</style>\n", | |
"<table border=\"1\" class=\"dataframe\">\n", | |
" <thead>\n", | |
" <tr style=\"text-align: right;\">\n", | |
" <th></th>\n", | |
" <th>name</th>\n", | |
" <th>categories</th>\n", | |
" <th>address</th>\n", | |
" <th>lat</th>\n", | |
" <th>lng</th>\n", | |
" <th>labeledLatLngs</th>\n", | |
" <th>distance</th>\n", | |
" <th>postalCode</th>\n", | |
" <th>cc</th>\n", | |
" <th>city</th>\n", | |
" <th>state</th>\n", | |
" <th>country</th>\n", | |
" <th>formattedAddress</th>\n", | |
" <th>id</th>\n", | |
" </tr>\n", | |
" </thead>\n", | |
" <tbody>\n", | |
" <tr>\n", | |
" <th>0</th>\n", | |
" <td>Harry's Italian Pizza Bar</td>\n", | |
" <td>Pizza Place</td>\n", | |
" <td>225 Murray St</td>\n", | |
" <td>40.715218</td>\n", | |
" <td>-74.014739</td>\n", | |
" <td>[{'label': 'display', 'lat': 40.71521779064671...</td>\n", | |
" <td>77</td>\n", | |
" <td>10282</td>\n", | |
" <td>US</td>\n", | |
" <td>New York</td>\n", | |
" <td>NY</td>\n", | |
" <td>United States</td>\n", | |
" <td>[225 Murray St, New York, NY 10282, United Sta...</td>\n", | |
" <td>4fa862b3e4b0ebff2f749f06</td>\n", | |
" </tr>\n", | |
" <tr>\n", | |
" <th>1</th>\n", | |
" <td>Conca Cucina Italian Restaurant</td>\n", | |
" <td>Food</td>\n", | |
" <td>63 W Broadway</td>\n", | |
" <td>40.714484</td>\n", | |
" <td>-74.009806</td>\n", | |
" <td>[{'label': 'display', 'lat': 40.71448400000000...</td>\n", | |
" <td>499</td>\n", | |
" <td>10007</td>\n", | |
" <td>US</td>\n", | |
" <td>New York</td>\n", | |
" <td>NY</td>\n", | |
" <td>United States</td>\n", | |
" <td>[63 W Broadway, New York, NY 10007, United Sta...</td>\n", | |
" <td>4f3232e219836c91c7bfde94</td>\n", | |
" </tr>\n", | |
" </tbody>\n", | |
"</table>\n", | |
"</div>" | |
], | |
"text/plain": [ | |
" name categories address lat \\\n", | |
"0 Harry's Italian Pizza Bar Pizza Place 225 Murray St 40.715218 \n", | |
"1 Conca Cucina Italian Restaurant Food 63 W Broadway 40.714484 \n", | |
"\n", | |
" lng labeledLatLngs distance \\\n", | |
"0 -74.014739 [{'label': 'display', 'lat': 40.71521779064671... 77 \n", | |
"1 -74.009806 [{'label': 'display', 'lat': 40.71448400000000... 499 \n", | |
"\n", | |
" postalCode cc city state country \\\n", | |
"0 10282 US New York NY United States \n", | |
"1 10007 US New York NY United States \n", | |
"\n", | |
" formattedAddress id \n", | |
"0 [225 Murray St, New York, NY 10282, United Sta... 4fa862b3e4b0ebff2f749f06 \n", | |
"1 [63 W Broadway, New York, NY 10007, United Sta... 4f3232e219836c91c7bfde94 " | |
] | |
}, | |
"execution_count": 9, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"# keep only columns that include venue name, and anything that is associated with location\n", | |
"filtered_columns = ['name', 'categories'] + [col for col in dataframe.columns if col.startswith('location.')] + ['id']\n", | |
"dataframe_filtered = dataframe.loc[:, filtered_columns]\n", | |
"\n", | |
"# function that extracts the category of the venue\n", | |
"def get_category_type(row):\n", | |
" try:\n", | |
" categories_list = row['categories']\n", | |
" except:\n", | |
" categories_list = row['venue.categories']\n", | |
" \n", | |
" if len(categories_list) == 0:\n", | |
" return None\n", | |
" else:\n", | |
" return categories_list[0]['name']\n", | |
"\n", | |
"# filter the category for each row\n", | |
"dataframe_filtered['categories'] = dataframe_filtered.apply(get_category_type, axis=1)\n", | |
"\n", | |
"# clean column names by keeping only last term\n", | |
"dataframe_filtered.columns = [column.split('.')[-1] for column in dataframe_filtered.columns]\n", | |
"\n", | |
"dataframe_filtered" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### Let's visualize the Italian restaurants that are nearby" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 10, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"data": { | |
"text/plain": [ | |
"0 Harry's Italian Pizza Bar\n", | |
"1 Conca Cucina Italian Restaurant\n", | |
"Name: name, dtype: object" | |
] | |
}, | |
"execution_count": 10, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"dataframe_filtered.name" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 11, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"data": { | |
"text/html": [ | |
"<div style=\"width:100%;\"><div style=\"position:relative;width:100%;height:0;padding-bottom:60%;\"><span style=\"color:#565656\">Make this Notebook Trusted to load map: File -> Trust Notebook</span><iframe src=\"about:blank\" style=\"position:absolute;width:100%;height:100%;left:0;top:0;border:none !important;\" data-html=PCFET0NUWVBFIGh0bWw+CjxoZWFkPiAgICAKICAgIDxtZXRhIGh0dHAtZXF1aXY9ImNvbnRlbnQtdHlwZSIgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PVVURi04IiAvPgogICAgPHNjcmlwdD5MX1BSRUZFUl9DQU5WQVMgPSBmYWxzZTsgTF9OT19UT1VDSCA9IGZhbHNlOyBMX0RJU0FCTEVfM0QgPSBmYWxzZTs8L3NjcmlwdD4KICAgIDxzY3JpcHQgc3JjPSJodHRwczovL2Nkbi5qc2RlbGl2ci5uZXQvbnBtL2xlYWZsZXRAMS4yLjAvZGlzdC9sZWFmbGV0LmpzIj48L3NjcmlwdD4KICAgIDxzY3JpcHQgc3JjPSJodHRwczovL2FqYXguZ29vZ2xlYXBpcy5jb20vYWpheC9saWJzL2pxdWVyeS8xLjExLjEvanF1ZXJ5Lm1pbi5qcyI+PC9zY3JpcHQ+CiAgICA8c2NyaXB0IHNyYz0iaHR0cHM6Ly9tYXhjZG4uYm9vdHN0cmFwY2RuLmNvbS9ib290c3RyYXAvMy4yLjAvanMvYm9vdHN0cmFwLm1pbi5qcyI+PC9zY3JpcHQ+CiAgICA8c2NyaXB0IHNyYz0iaHR0cHM6Ly9jZG5qcy5jbG91ZGZsYXJlLmNvbS9hamF4L2xpYnMvTGVhZmxldC5hd2Vzb21lLW1hcmtlcnMvMi4wLjIvbGVhZmxldC5hd2Vzb21lLW1hcmtlcnMuanMiPjwvc2NyaXB0PgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL2Nkbi5qc2RlbGl2ci5uZXQvbnBtL2xlYWZsZXRAMS4yLjAvZGlzdC9sZWFmbGV0LmNzcyIvPgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL21heGNkbi5ib290c3RyYXBjZG4uY29tL2Jvb3RzdHJhcC8zLjIuMC9jc3MvYm9vdHN0cmFwLm1pbi5jc3MiLz4KICAgIDxsaW5rIHJlbD0ic3R5bGVzaGVldCIgaHJlZj0iaHR0cHM6Ly9tYXhjZG4uYm9vdHN0cmFwY2RuLmNvbS9ib290c3RyYXAvMy4yLjAvY3NzL2Jvb3RzdHJhcC10aGVtZS5taW4uY3NzIi8+CiAgICA8bGluayByZWw9InN0eWxlc2hlZXQiIGhyZWY9Imh0dHBzOi8vbWF4Y2RuLmJvb3RzdHJhcGNkbi5jb20vZm9udC1hd2Vzb21lLzQuNi4zL2Nzcy9mb250LWF3ZXNvbWUubWluLmNzcyIvPgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL2NkbmpzLmNsb3VkZmxhcmUuY29tL2FqYXgvbGlicy9MZWFmbGV0LmF3ZXNvbWUtbWFya2Vycy8yLjAuMi9sZWFmbGV0LmF3ZXNvbWUtbWFya2Vycy5jc3MiLz4KICAgIDxsaW5rIHJlbD0ic3R5bGVzaGVldCIgaHJlZj0iaHR0cHM6Ly9yYXdnaXQuY29tL3B5dGhvbi12aXN1YWxpemF0aW9uL2ZvbGl1bS9tYXN0ZXIvZm9saXVtL3RlbXBsYXRlcy9sZWFmbGV0LmF3ZXNvbWUucm90YXRlLmNzcyIvPgogICAgPHN0eWxlPmh0bWwsIGJvZHkge3dpZHRoOiAxMDAlO2hlaWdodDogMTAwJTttYXJnaW46IDA7cGFkZGluZzogMDt9PC9zdHlsZT4KICAgIDxzdHlsZT4jbWFwIHtwb3NpdGlvbjphYnNvbHV0ZTt0b3A6MDtib3R0b206MDtyaWdodDowO2xlZnQ6MDt9PC9zdHlsZT4KICAgIAogICAgICAgICAgICA8c3R5bGU+ICNtYXBfNWYwYjJiYmUyYjBkNDY5NGE4NWRjZmI0Y2JkZWM1Y2YgewogICAgICAgICAgICAgICAgcG9zaXRpb24gOiByZWxhdGl2ZTsKICAgICAgICAgICAgICAgIHdpZHRoIDogMTAwLjAlOwogICAgICAgICAgICAgICAgaGVpZ2h0OiAxMDAuMCU7CiAgICAgICAgICAgICAgICBsZWZ0OiAwLjAlOwogICAgICAgICAgICAgICAgdG9wOiAwLjAlOwogICAgICAgICAgICAgICAgfQogICAgICAgICAgICA8L3N0eWxlPgogICAgICAgIAo8L2hlYWQ+Cjxib2R5PiAgICAKICAgIAogICAgICAgICAgICA8ZGl2IGNsYXNzPSJmb2xpdW0tbWFwIiBpZD0ibWFwXzVmMGIyYmJlMmIwZDQ2OTRhODVkY2ZiNGNiZGVjNWNmIiA+PC9kaXY+CiAgICAgICAgCjwvYm9keT4KPHNjcmlwdD4gICAgCiAgICAKCiAgICAgICAgICAgIAogICAgICAgICAgICAgICAgdmFyIGJvdW5kcyA9IG51bGw7CiAgICAgICAgICAgIAoKICAgICAgICAgICAgdmFyIG1hcF81ZjBiMmJiZTJiMGQ0Njk0YTg1ZGNmYjRjYmRlYzVjZiA9IEwubWFwKAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgJ21hcF81ZjBiMmJiZTJiMGQ0Njk0YTg1ZGNmYjRjYmRlYzVjZicsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB7Y2VudGVyOiBbNDAuNzE1MTQ4MiwtNzQuMDE1NjU3M10sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB6b29tOiAxMywKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG1heEJvdW5kczogYm91bmRzLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbGF5ZXJzOiBbXSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHdvcmxkQ29weUp1bXA6IGZhbHNlLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY3JzOiBMLkNSUy5FUFNHMzg1NwogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB9KTsKICAgICAgICAgICAgCiAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIHRpbGVfbGF5ZXJfNmIzZmUyODMxNzI3NGEwNmI4NjQ3ZTI5MjIwY2UxMGIgPSBMLnRpbGVMYXllcigKICAgICAgICAgICAgICAgICdodHRwczovL3tzfS50aWxlLm9wZW5zdHJlZXRtYXAub3JnL3t6fS97eH0ve3l9LnBuZycsCiAgICAgICAgICAgICAgICB7CiAgImF0dHJpYnV0aW9uIjogbnVsbCwKICAiZGV0ZWN0UmV0aW5hIjogZmFsc2UsCiAgIm1heFpvb20iOiAxOCwKICAibWluWm9vbSI6IDEsCiAgIm5vV3JhcCI6IGZhbHNlLAogICJzdWJkb21haW5zIjogImFiYyIKfQogICAgICAgICAgICAgICAgKS5hZGRUbyhtYXBfNWYwYjJiYmUyYjBkNDY5NGE4NWRjZmI0Y2JkZWM1Y2YpOwogICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBjaXJjbGVfbWFya2VyXzJjZmU5ZTMwZTUxODQwNjg5MDhmMDJlYTY0ZjRhMGIwID0gTC5jaXJjbGVNYXJrZXIoCiAgICAgICAgICAgICAgICBbNDAuNzE1MTQ4MiwtNzQuMDE1NjU3M10sCiAgICAgICAgICAgICAgICB7CiAgImJ1YmJsaW5nTW91c2VFdmVudHMiOiB0cnVlLAogICJjb2xvciI6ICJyZWQiLAogICJkYXNoQXJyYXkiOiBudWxsLAogICJkYXNoT2Zmc2V0IjogbnVsbCwKICAiZmlsbCI6IHRydWUsCiAgImZpbGxDb2xvciI6ICJyZWQiLAogICJmaWxsT3BhY2l0eSI6IDAuNiwKICAiZmlsbFJ1bGUiOiAiZXZlbm9kZCIsCiAgImxpbmVDYXAiOiAicm91bmQiLAogICJsaW5lSm9pbiI6ICJyb3VuZCIsCiAgIm9wYWNpdHkiOiAxLjAsCiAgInJhZGl1cyI6IDEwLAogICJzdHJva2UiOiB0cnVlLAogICJ3ZWlnaHQiOiAzCn0KICAgICAgICAgICAgICAgICkuYWRkVG8obWFwXzVmMGIyYmJlMmIwZDQ2OTRhODVkY2ZiNGNiZGVjNWNmKTsKICAgICAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIHBvcHVwXzZlZjMzYTU5MDcwMjQwODM5ZGNlZTE4NzA5ZWI0NThlID0gTC5wb3B1cCh7bWF4V2lkdGg6ICczMDAnfSk7CgogICAgICAgICAgICAKICAgICAgICAgICAgICAgIHZhciBodG1sXzA5MTNhMzFhN2E2ZDQxMjRhZWIzMDc3YzBhMjNiYzQxID0gJCgnPGRpdiBpZD0iaHRtbF8wOTEzYTMxYTdhNmQ0MTI0YWViMzA3N2MwYTIzYmM0MSIgc3R5bGU9IndpZHRoOiAxMDAuMCU7IGhlaWdodDogMTAwLjAlOyI+Q29ucmFkIEhvdGVsPC9kaXY+JylbMF07CiAgICAgICAgICAgICAgICBwb3B1cF82ZWYzM2E1OTA3MDI0MDgzOWRjZWUxODcwOWViNDU4ZS5zZXRDb250ZW50KGh0bWxfMDkxM2EzMWE3YTZkNDEyNGFlYjMwNzdjMGEyM2JjNDEpOwogICAgICAgICAgICAKCiAgICAgICAgICAgIGNpcmNsZV9tYXJrZXJfMmNmZTllMzBlNTE4NDA2ODkwOGYwMmVhNjRmNGEwYjAuYmluZFBvcHVwKHBvcHVwXzZlZjMzYTU5MDcwMjQwODM5ZGNlZTE4NzA5ZWI0NThlKTsKCiAgICAgICAgICAgIAogICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBjaXJjbGVfbWFya2VyX2ZhNTA0MGY2NzRiZjQ5YWRhMDgxNTQyYjVlMzEyYzFhID0gTC5jaXJjbGVNYXJrZXIoCiAgICAgICAgICAgICAgICBbNDAuNzE1MjE3NzkwNjQ2NzEsLTc0LjAxNDczOTQwMjA5MzUxXSwKICAgICAgICAgICAgICAgIHsKICAiYnViYmxpbmdNb3VzZUV2ZW50cyI6IHRydWUsCiAgImNvbG9yIjogImJsdWUiLAogICJkYXNoQXJyYXkiOiBudWxsLAogICJkYXNoT2Zmc2V0IjogbnVsbCwKICAiZmlsbCI6IHRydWUsCiAgImZpbGxDb2xvciI6ICJibHVlIiwKICAiZmlsbE9wYWNpdHkiOiAwLjYsCiAgImZpbGxSdWxlIjogImV2ZW5vZGQiLAogICJsaW5lQ2FwIjogInJvdW5kIiwKICAibGluZUpvaW4iOiAicm91bmQiLAogICJvcGFjaXR5IjogMS4wLAogICJyYWRpdXMiOiA1LAogICJzdHJva2UiOiB0cnVlLAogICJ3ZWlnaHQiOiAzCn0KICAgICAgICAgICAgICAgICkuYWRkVG8obWFwXzVmMGIyYmJlMmIwZDQ2OTRhODVkY2ZiNGNiZGVjNWNmKTsKICAgICAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIHBvcHVwXzUwZmQ3OTVkZDg1YzQ2MTdhNTVhMmZlZWI0YzIzNmJkID0gTC5wb3B1cCh7bWF4V2lkdGg6ICczMDAnfSk7CgogICAgICAgICAgICAKICAgICAgICAgICAgICAgIHZhciBodG1sX2QzODRiMTU2NzcyYTQ3YmFiM2Q4NDc1NTc4Yzg4YWFjID0gJCgnPGRpdiBpZD0iaHRtbF9kMzg0YjE1Njc3MmE0N2JhYjNkODQ3NTU3OGM4OGFhYyIgc3R5bGU9IndpZHRoOiAxMDAuMCU7IGhlaWdodDogMTAwLjAlOyI+UGl6emEgUGxhY2U8L2Rpdj4nKVswXTsKICAgICAgICAgICAgICAgIHBvcHVwXzUwZmQ3OTVkZDg1YzQ2MTdhNTVhMmZlZWI0YzIzNmJkLnNldENvbnRlbnQoaHRtbF9kMzg0YjE1Njc3MmE0N2JhYjNkODQ3NTU3OGM4OGFhYyk7CiAgICAgICAgICAgIAoKICAgICAgICAgICAgY2lyY2xlX21hcmtlcl9mYTUwNDBmNjc0YmY0OWFkYTA4MTU0MmI1ZTMxMmMxYS5iaW5kUG9wdXAocG9wdXBfNTBmZDc5NWRkODVjNDYxN2E1NWEyZmVlYjRjMjM2YmQpOwoKICAgICAgICAgICAgCiAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIGNpcmNsZV9tYXJrZXJfZDg0MDI4YWZlZDZiNDUzZjkzZTE0ZDM1NmFkMzkwOTAgPSBMLmNpcmNsZU1hcmtlcigKICAgICAgICAgICAgICAgIFs0MC43MTQ0ODQwMDAwMDAwMDYsLTc0LjAwOTgwNjAwMDAwMDAxXSwKICAgICAgICAgICAgICAgIHsKICAiYnViYmxpbmdNb3VzZUV2ZW50cyI6IHRydWUsCiAgImNvbG9yIjogImJsdWUiLAogICJkYXNoQXJyYXkiOiBudWxsLAogICJkYXNoT2Zmc2V0IjogbnVsbCwKICAiZmlsbCI6IHRydWUsCiAgImZpbGxDb2xvciI6ICJibHVlIiwKICAiZmlsbE9wYWNpdHkiOiAwLjYsCiAgImZpbGxSdWxlIjogImV2ZW5vZGQiLAogICJsaW5lQ2FwIjogInJvdW5kIiwKICAibGluZUpvaW4iOiAicm91bmQiLAogICJvcGFjaXR5IjogMS4wLAogICJyYWRpdXMiOiA1LAogICJzdHJva2UiOiB0cnVlLAogICJ3ZWlnaHQiOiAzCn0KICAgICAgICAgICAgICAgICkuYWRkVG8obWFwXzVmMGIyYmJlMmIwZDQ2OTRhODVkY2ZiNGNiZGVjNWNmKTsKICAgICAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIHBvcHVwXzdmOTBiMTY3YmZjZDQ3M2Q5M2NhNjBmZGMxZjFhOGFkID0gTC5wb3B1cCh7bWF4V2lkdGg6ICczMDAnfSk7CgogICAgICAgICAgICAKICAgICAgICAgICAgICAgIHZhciBodG1sXzFmOGFjM2UyM2U4ZTQyYmZiZjcwNjA5NDk3MDdmOGY1ID0gJCgnPGRpdiBpZD0iaHRtbF8xZjhhYzNlMjNlOGU0MmJmYmY3MDYwOTQ5NzA3ZjhmNSIgc3R5bGU9IndpZHRoOiAxMDAuMCU7IGhlaWdodDogMTAwLjAlOyI+Rm9vZDwvZGl2PicpWzBdOwogICAgICAgICAgICAgICAgcG9wdXBfN2Y5MGIxNjdiZmNkNDczZDkzY2E2MGZkYzFmMWE4YWQuc2V0Q29udGVudChodG1sXzFmOGFjM2UyM2U4ZTQyYmZiZjcwNjA5NDk3MDdmOGY1KTsKICAgICAgICAgICAgCgogICAgICAgICAgICBjaXJjbGVfbWFya2VyX2Q4NDAyOGFmZWQ2YjQ1M2Y5M2UxNGQzNTZhZDM5MDkwLmJpbmRQb3B1cChwb3B1cF83ZjkwYjE2N2JmY2Q0NzNkOTNjYTYwZmRjMWYxYThhZCk7CgogICAgICAgICAgICAKICAgICAgICAKPC9zY3JpcHQ+ onload=\"this.contentDocument.open();this.contentDocument.write(atob(this.getAttribute('data-html')));this.contentDocument.close();\" allowfullscreen webkitallowfullscreen mozallowfullscreen></iframe></div></div>" | |
], | |
"text/plain": [ | |
"<folium.folium.Map at 0x7f9e1ef845c0>" | |
] | |
}, | |
"execution_count": 11, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"venues_map = folium.Map(location=[latitude, longitude], zoom_start=13) # generate map centred around the Conrad Hotel\n", | |
"\n", | |
"# add a red circle marker to represent the Conrad Hotel\n", | |
"folium.features.CircleMarker(\n", | |
" [latitude, longitude],\n", | |
" radius=10,\n", | |
" color='red',\n", | |
" popup='Conrad Hotel',\n", | |
" fill = True,\n", | |
" fill_color = 'red',\n", | |
" fill_opacity = 0.6\n", | |
").add_to(venues_map)\n", | |
"\n", | |
"# add the Italian restaurants as blue circle markers\n", | |
"for lat, lng, label in zip(dataframe_filtered.lat, dataframe_filtered.lng, dataframe_filtered.categories):\n", | |
" folium.features.CircleMarker(\n", | |
" [lat, lng],\n", | |
" radius=5,\n", | |
" color='blue',\n", | |
" popup=label,\n", | |
" fill = True,\n", | |
" fill_color='blue',\n", | |
" fill_opacity=0.6\n", | |
" ).add_to(venues_map)\n", | |
"\n", | |
"# display map\n", | |
"venues_map" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
" " | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"<a id=\"item2\"></a>" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"## 2. Explore a Given Venue\n", | |
"> `https://api.foursquare.com/v2/venues/`**VENUE_ID**`?client_id=`**CLIENT_ID**`&client_secret=`**CLIENT_SECRET**`&v=`**VERSION**" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"### A. Let's explore the closest Italian restaurant -- _Harry's Italian Pizza Bar_" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 12, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"data": { | |
"text/plain": [ | |
"'https://api.foursquare.com/v2/venues/4fa862b3e4b0ebff2f749f06?client_id=NM2ZJ4MDHZEN140JF3CDFT5JKTFY5XKPACFFJV1LR2ORR3W0&client_secret=GYL1KCRUWNUQKY51A5E4FKOLYSAEG4C4GIO2VEFUNITJYZFC&v=20180604'" | |
] | |
}, | |
"execution_count": 12, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"venue_id = '4fa862b3e4b0ebff2f749f06' # ID of Harry's Italian Pizza Bar\n", | |
"url = 'https://api.foursquare.com/v2/venues/{}?client_id={}&client_secret={}&v={}'.format(venue_id, CLIENT_ID, CLIENT_SECRET, VERSION)\n", | |
"url" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### Send GET request for result" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 13, | |
"metadata": { | |
"button": false, | |
"collapsed": true, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": true | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"name": "stdout", | |
"output_type": "stream", | |
"text": [ | |
"dict_keys(['id', 'name', 'contact', 'location', 'canonicalUrl', 'categories', 'verified', 'stats', 'url', 'price', 'hasMenu', 'likes', 'dislike', 'ok', 'rating', 'ratingColor', 'ratingSignals', 'menu', 'allowMenuUrlEdit', 'beenHere', 'specials', 'photos', 'reasons', 'hereNow', 'createdAt', 'tips', 'shortUrl', 'timeZone', 'listed', 'hours', 'popular', 'seasonalHours', 'defaultHours', 'pageUpdates', 'inbox', 'attributes', 'bestPhoto', 'colors'])\n" | |
] | |
}, | |
{ | |
"data": { | |
"text/plain": [ | |
"{'id': '4fa862b3e4b0ebff2f749f06',\n", | |
" 'name': \"Harry's Italian Pizza Bar\",\n", | |
" 'contact': {'phone': '2126081007', 'formattedPhone': '(212) 608-1007'},\n", | |
" 'location': {'address': '225 Murray St',\n", | |
" 'lat': 40.71521779064671,\n", | |
" 'lng': -74.01473940209351,\n", | |
" 'labeledLatLngs': [{'label': 'display',\n", | |
" 'lat': 40.71521779064671,\n", | |
" 'lng': -74.01473940209351},\n", | |
" {'label': 'entrance', 'lat': 40.715361, 'lng': -74.014975}],\n", | |
" 'postalCode': '10282',\n", | |
" 'cc': 'US',\n", | |
" 'city': 'New York',\n", | |
" 'state': 'NY',\n", | |
" 'country': 'United States',\n", | |
" 'formattedAddress': ['225 Murray St',\n", | |
" 'New York, NY 10282',\n", | |
" 'United States']},\n", | |
" 'canonicalUrl': 'https://foursquare.com/v/harrys-italian-pizza-bar/4fa862b3e4b0ebff2f749f06',\n", | |
" 'categories': [{'id': '4bf58dd8d48988d1ca941735',\n", | |
" 'name': 'Pizza Place',\n", | |
" 'pluralName': 'Pizza Places',\n", | |
" 'shortName': 'Pizza',\n", | |
" 'icon': {'prefix': 'https://ss3.4sqi.net/img/categories_v2/food/pizza_',\n", | |
" 'suffix': '.png'},\n", | |
" 'primary': True},\n", | |
" {'id': '4bf58dd8d48988d110941735',\n", | |
" 'name': 'Italian Restaurant',\n", | |
" 'pluralName': 'Italian Restaurants',\n", | |
" 'shortName': 'Italian',\n", | |
" 'icon': {'prefix': 'https://ss3.4sqi.net/img/categories_v2/food/italian_',\n", | |
" 'suffix': '.png'}}],\n", | |
" 'verified': False,\n", | |
" 'stats': {'tipCount': 57},\n", | |
" 'url': 'http://harrysitalian.com',\n", | |
" 'price': {'tier': 2, 'message': 'Moderate', 'currency': '$'},\n", | |
" 'hasMenu': True,\n", | |
" 'likes': {'count': 120,\n", | |
" 'groups': [{'type': 'others', 'count': 120, 'items': []}],\n", | |
" 'summary': '120 Likes'},\n", | |
" 'dislike': False,\n", | |
" 'ok': False,\n", | |
" 'rating': 6.4,\n", | |
" 'ratingColor': 'FFC800',\n", | |
" 'ratingSignals': 212,\n", | |
" 'menu': {'type': 'Menu',\n", | |
" 'label': 'Menu',\n", | |
" 'anchor': 'View Menu',\n", | |
" 'url': 'https://foursquare.com/v/harrys-italian-pizza-bar/4fa862b3e4b0ebff2f749f06/menu',\n", | |
" 'mobileUrl': 'https://foursquare.com/v/4fa862b3e4b0ebff2f749f06/device_menu'},\n", | |
" 'allowMenuUrlEdit': True,\n", | |
" 'beenHere': {'count': 0,\n", | |
" 'unconfirmedCount': 0,\n", | |
" 'marked': False,\n", | |
" 'lastCheckinExpiredAt': 0},\n", | |
" 'specials': {'count': 0, 'items': []},\n", | |
" 'photos': {'count': 146,\n", | |
" 'groups': [{'type': 'venue',\n", | |
" 'name': 'Venue photos',\n", | |
" 'count': 146,\n", | |
" 'items': [{'id': '4fad980de4b091b4626c3633',\n", | |
" 'createdAt': 1336776717,\n", | |
" 'source': {'name': 'Foursquare for Android',\n", | |
" 'url': 'https://foursquare.com/download/#/android'},\n", | |
" 'prefix': 'https://fastly.4sqi.net/img/general/',\n", | |
" 'suffix': '/ya1iQFI7pLjuIJp1PGDKlrZS3OJdHCF7tpILMmjv_2w.jpg',\n", | |
" 'width': 480,\n", | |
" 'height': 640,\n", | |
" 'user': {'id': '13676709',\n", | |
" 'firstName': 'Leony',\n", | |
" 'lastName': 'N',\n", | |
" 'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n", | |
" 'suffix': '/T0ANFNGNMCHUDEUE.jpg'}},\n", | |
" 'visibility': 'public'}]}]},\n", | |
" 'reasons': {'count': 1,\n", | |
" 'items': [{'summary': 'Lots of people like this place',\n", | |
" 'type': 'general',\n", | |
" 'reasonName': 'rawLikesReason'}]},\n", | |
" 'hereNow': {'count': 0, 'summary': 'Nobody here', 'groups': []},\n", | |
" 'createdAt': 1336435379,\n", | |
" 'tips': {'count': 57,\n", | |
" 'groups': [{'type': 'others',\n", | |
" 'name': 'All tips',\n", | |
" 'count': 57,\n", | |
" 'items': [{'id': '53d27909498e0523841340b6',\n", | |
" 'createdAt': 1406302473,\n", | |
" 'text': \"Harry's Italian Pizza bar is known for it's amazing pizza, but did you know that the brunches here are amazing too? Try the Nutella French toast and we know you'll be sold.\",\n", | |
" 'type': 'user',\n", | |
" 'canonicalUrl': 'https://foursquare.com/item/53d27909498e0523841340b6',\n", | |
" 'lang': 'en',\n", | |
" 'likes': {'count': 4,\n", | |
" 'groups': [{'type': 'others',\n", | |
" 'count': 4,\n", | |
" 'items': [{'id': '369426',\n", | |
" 'firstName': 'P.',\n", | |
" 'lastName': 'M',\n", | |
" 'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n", | |
" 'suffix': '/JPQYUWJKUT0H2OO4.jpg'}},\n", | |
" {'id': '87587879',\n", | |
" 'firstName': 'Diane',\n", | |
" 'lastName': 'D',\n", | |
" 'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n", | |
" 'suffix': '/87587879-ESLRSZLQ2CBE2P4W.jpg'}},\n", | |
" {'id': '87591341',\n", | |
" 'firstName': 'Tim',\n", | |
" 'lastName': 'S',\n", | |
" 'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n", | |
" 'suffix': '/-Z4YK4VKE0JSVXIY1.jpg'}},\n", | |
" {'id': '87473404',\n", | |
" 'firstName': 'TenantKing.com',\n", | |
" 'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n", | |
" 'suffix': '/87473404-HI5DTBTK0HX401CA.png'},\n", | |
" 'type': 'page'}]}],\n", | |
" 'summary': '4 likes'},\n", | |
" 'logView': True,\n", | |
" 'agreeCount': 4,\n", | |
" 'disagreeCount': 0,\n", | |
" 'todo': {'count': 0},\n", | |
" 'user': {'id': '87473404',\n", | |
" 'firstName': 'TenantKing.com',\n", | |
" 'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n", | |
" 'suffix': '/87473404-HI5DTBTK0HX401CA.png'},\n", | |
" 'type': 'page'}}]}]},\n", | |
" 'shortUrl': 'http://4sq.com/JNblHV',\n", | |
" 'timeZone': 'America/New_York',\n", | |
" 'listed': {'count': 54,\n", | |
" 'groups': [{'type': 'others',\n", | |
" 'name': 'Lists from other people',\n", | |
" 'count': 54,\n", | |
" 'items': [{'id': '4fa32fd0e4b04193744746b1',\n", | |
" 'name': 'Manhattan Haunts',\n", | |
" 'description': '',\n", | |
" 'type': 'others',\n", | |
" 'user': {'id': '24592223',\n", | |
" 'firstName': 'Becca',\n", | |
" 'lastName': 'M',\n", | |
" 'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n", | |
" 'suffix': '/24592223-RAW2UYM0GIB1U40K.jpg'}},\n", | |
" 'editable': False,\n", | |
" 'public': True,\n", | |
" 'collaborative': False,\n", | |
" 'url': '/becca_mcarthur/list/manhattan-haunts',\n", | |
" 'canonicalUrl': 'https://foursquare.com/becca_mcarthur/list/manhattan-haunts',\n", | |
" 'createdAt': 1336094672,\n", | |
" 'updatedAt': 1380845377,\n", | |
" 'photo': {'id': '4e8cc9461081e3b3544e12e5',\n", | |
" 'createdAt': 1317849414,\n", | |
" 'prefix': 'https://fastly.4sqi.net/img/general/',\n", | |
" 'suffix': '/0NLVU2HC1JF4DXIMKWUFW3QBUT31DC11EFNYYHMJG3NDWAPS.jpg',\n", | |
" 'width': 492,\n", | |
" 'height': 330,\n", | |
" 'user': {'id': '742542',\n", | |
" 'firstName': 'Time Out New York',\n", | |
" 'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n", | |
" 'suffix': '/XXHKCBSQHBORZNSR.jpg'},\n", | |
" 'type': 'page'},\n", | |
" 'visibility': 'public'},\n", | |
" 'followers': {'count': 22},\n", | |
" 'listItems': {'count': 187,\n", | |
" 'items': [{'id': 'v4fa862b3e4b0ebff2f749f06',\n", | |
" 'createdAt': 1342934485}]}},\n", | |
" {'id': '4fae817be4b085f6b2a74d19',\n", | |
" 'name': 'USA NYC MAN FiDi',\n", | |
" 'description': 'Where to go for decent eats in the restaurant wasteland of Downtown NYC aka FiDi, along with Tribeca & Battery Park City.',\n", | |
" 'type': 'others',\n", | |
" 'user': {'id': '12113441',\n", | |
" 'firstName': 'Kino',\n", | |
" 'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n", | |
" 'suffix': '/12113441-K5HTHFLU2MUCM0CM.jpg'}},\n", | |
" 'editable': False,\n", | |
" 'public': True,\n", | |
" 'collaborative': False,\n", | |
" 'url': '/kinosfault/list/usa-nyc-man-fidi',\n", | |
" 'canonicalUrl': 'https://foursquare.com/kinosfault/list/usa-nyc-man-fidi',\n", | |
" 'createdAt': 1336836475,\n", | |
" 'updatedAt': 1556754919,\n", | |
" 'photo': {'id': '55984992498e13ba75e353bb',\n", | |
" 'createdAt': 1436043666,\n", | |
" 'prefix': 'https://fastly.4sqi.net/img/general/',\n", | |
" 'suffix': '/12113441_iOa6Uh-Xi8bhj2-gpzkkw8MKiAIs7RmOcz_RM7m8ink.jpg',\n", | |
" 'width': 540,\n", | |
" 'height': 960,\n", | |
" 'user': {'id': '12113441',\n", | |
" 'firstName': 'Kino',\n", | |
" 'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n", | |
" 'suffix': '/12113441-K5HTHFLU2MUCM0CM.jpg'}},\n", | |
" 'visibility': 'public'},\n", | |
" 'followers': {'count': 20},\n", | |
" 'listItems': {'count': 273,\n", | |
" 'items': [{'id': 'v4fa862b3e4b0ebff2f749f06',\n", | |
" 'createdAt': 1373909433}]}},\n", | |
" {'id': '4fddeff0e4b0e078037ac0d3',\n", | |
" 'name': 'NYC Resturants',\n", | |
" 'description': '',\n", | |
" 'type': 'others',\n", | |
" 'user': {'id': '21563126',\n", | |
" 'firstName': 'Richard',\n", | |
" 'lastName': 'R',\n", | |
" 'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n", | |
" 'suffix': '/21563126_v05J1KPw_SVj6Ehq9g8B9jeAGjFUMsU5QGl-NZ8inUQ7pKQm5bKplW37EmR7jS2A7GYPBBAtl.jpg'}},\n", | |
" 'editable': False,\n", | |
" 'public': True,\n", | |
" 'collaborative': True,\n", | |
" 'url': '/rickr7/list/nyc-resturants',\n", | |
" 'canonicalUrl': 'https://foursquare.com/rickr7/list/nyc-resturants',\n", | |
" 'createdAt': 1339944944,\n", | |
" 'updatedAt': 1589017010,\n", | |
" 'photo': {'id': '5072dd13e4b09145cdf782d1',\n", | |
" 'createdAt': 1349704979,\n", | |
" 'prefix': 'https://fastly.4sqi.net/img/general/',\n", | |
" 'suffix': '/208205_fGh2OuAZ9qJ4agbAA5wMVNOSIm9kNUlRtNwj1N-adqg.jpg',\n", | |
" 'width': 800,\n", | |
" 'height': 800,\n", | |
" 'user': {'id': '208205',\n", | |
" 'firstName': 'Thalia',\n", | |
" 'lastName': 'K',\n", | |
" 'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n", | |
" 'suffix': '/SNOOLCAW2AG04ZKD.jpg'}},\n", | |
" 'visibility': 'public'},\n", | |
" 'followers': {'count': 12},\n", | |
" 'listItems': {'count': 192,\n", | |
" 'items': [{'id': 'v4fa862b3e4b0ebff2f749f06',\n", | |
" 'createdAt': 1581655865}]}},\n", | |
" {'id': '5266c68a498e7c667807fe09',\n", | |
" 'name': 'Foodie Love in NY - 02',\n", | |
" 'description': '',\n", | |
" 'type': 'others',\n", | |
" 'user': {'id': '547977',\n", | |
" 'firstName': 'WiLL',\n", | |
" 'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n", | |
" 'suffix': '/-Q5NYGDMFDMOITQRR.jpg'}},\n", | |
" 'editable': False,\n", | |
" 'public': True,\n", | |
" 'collaborative': False,\n", | |
" 'url': '/sweetiewill/list/foodie-love-in-ny--02',\n", | |
" 'canonicalUrl': 'https://foursquare.com/sweetiewill/list/foodie-love-in-ny--02',\n", | |
" 'createdAt': 1382467210,\n", | |
" 'updatedAt': 1391995585,\n", | |
" 'followers': {'count': 7},\n", | |
" 'listItems': {'count': 200,\n", | |
" 'items': [{'id': 'v4fa862b3e4b0ebff2f749f06',\n", | |
" 'createdAt': 1386809936}]}}]}]},\n", | |
" 'hours': {'status': 'Closed until 11:30 AM',\n", | |
" 'richStatus': {'entities': [], 'text': 'Closed until 11:30 AM'},\n", | |
" 'isOpen': False,\n", | |
" 'isLocalHoliday': False,\n", | |
" 'dayData': [],\n", | |
" 'timeframes': [{'days': 'Mon–Wed, Sun',\n", | |
" 'includesToday': True,\n", | |
" 'open': [{'renderedTime': '11:30 AM–11:00 PM'}],\n", | |
" 'segments': []},\n", | |
" {'days': 'Thu–Sat',\n", | |
" 'open': [{'renderedTime': '11:30 AM–Midnight'}],\n", | |
" 'segments': []}]},\n", | |
" 'popular': {'isOpen': False,\n", | |
" 'isLocalHoliday': False,\n", | |
" 'timeframes': [{'days': 'Today',\n", | |
" 'includesToday': True,\n", | |
" 'open': [{'renderedTime': 'Noon–2:00 PM'},\n", | |
" {'renderedTime': '5:00 PM–10:00 PM'}],\n", | |
" 'segments': []},\n", | |
" {'days': 'Wed–Thu',\n", | |
" 'open': [{'renderedTime': 'Noon–2:00 PM'},\n", | |
" {'renderedTime': '5:00 PM–10:00 PM'}],\n", | |
" 'segments': []},\n", | |
" {'days': 'Fri',\n", | |
" 'open': [{'renderedTime': 'Noon–3:00 PM'},\n", | |
" {'renderedTime': '5:00 PM–11:00 PM'}],\n", | |
" 'segments': []},\n", | |
" {'days': 'Sat',\n", | |
" 'open': [{'renderedTime': 'Noon–11:00 PM'}],\n", | |
" 'segments': []},\n", | |
" {'days': 'Sun',\n", | |
" 'open': [{'renderedTime': 'Noon–3:00 PM'},\n", | |
" {'renderedTime': '5:00 PM–8:00 PM'}],\n", | |
" 'segments': []},\n", | |
" {'days': 'Mon',\n", | |
" 'open': [{'renderedTime': 'Noon–2:00 PM'},\n", | |
" {'renderedTime': '6:00 PM–8:00 PM'}],\n", | |
" 'segments': []}]},\n", | |
" 'seasonalHours': [],\n", | |
" 'defaultHours': {'status': 'Closed until 11:30 AM',\n", | |
" 'richStatus': {'entities': [], 'text': 'Closed until 11:30 AM'},\n", | |
" 'isOpen': False,\n", | |
" 'isLocalHoliday': False,\n", | |
" 'dayData': [],\n", | |
" 'timeframes': [{'days': 'Mon–Wed, Sun',\n", | |
" 'includesToday': True,\n", | |
" 'open': [{'renderedTime': '11:30 AM–11:00 PM'}],\n", | |
" 'segments': []},\n", | |
" {'days': 'Thu–Sat',\n", | |
" 'open': [{'renderedTime': '11:30 AM–Midnight'}],\n", | |
" 'segments': []}]},\n", | |
" 'pageUpdates': {'count': 0, 'items': []},\n", | |
" 'inbox': {'count': 0, 'items': []},\n", | |
" 'attributes': {'groups': [{'type': 'price',\n", | |
" 'name': 'Price',\n", | |
" 'summary': '$$',\n", | |
" 'count': 1,\n", | |
" 'items': [{'displayName': 'Price', 'displayValue': '$$', 'priceTier': 2}]},\n", | |
" {'type': 'payments',\n", | |
" 'name': 'Credit Cards',\n", | |
" 'summary': 'Credit Cards',\n", | |
" 'count': 7,\n", | |
" 'items': [{'displayName': 'Credit Cards',\n", | |
" 'displayValue': 'Yes (incl. American Express)'}]},\n", | |
" {'type': 'outdoorSeating',\n", | |
" 'name': 'Outdoor Seating',\n", | |
" 'summary': 'Outdoor Seating',\n", | |
" 'count': 1,\n", | |
" 'items': [{'displayName': 'Outdoor Seating', 'displayValue': 'Yes'}]},\n", | |
" {'type': 'serves',\n", | |
" 'name': 'Menus',\n", | |
" 'summary': 'Happy Hour, Brunch & more',\n", | |
" 'count': 8,\n", | |
" 'items': [{'displayName': 'Brunch', 'displayValue': 'Brunch'},\n", | |
" {'displayName': 'Lunch', 'displayValue': 'Lunch'},\n", | |
" {'displayName': 'Dinner', 'displayValue': 'Dinner'},\n", | |
" {'displayName': 'Happy Hour', 'displayValue': 'Happy Hour'}]},\n", | |
" {'type': 'drinks',\n", | |
" 'name': 'Drinks',\n", | |
" 'summary': 'Beer, Wine & Cocktails',\n", | |
" 'count': 5,\n", | |
" 'items': [{'displayName': 'Beer', 'displayValue': 'Beer'},\n", | |
" {'displayName': 'Wine', 'displayValue': 'Wine'},\n", | |
" {'displayName': 'Cocktails', 'displayValue': 'Cocktails'}]},\n", | |
" {'type': 'diningOptions',\n", | |
" 'name': 'Dining Options',\n", | |
" 'summary': 'Delivery',\n", | |
" 'count': 5,\n", | |
" 'items': [{'displayName': 'Delivery', 'displayValue': 'Delivery'}]}]},\n", | |
" 'bestPhoto': {'id': '4fad980de4b091b4626c3633',\n", | |
" 'createdAt': 1336776717,\n", | |
" 'source': {'name': 'Foursquare for Android',\n", | |
" 'url': 'https://foursquare.com/download/#/android'},\n", | |
" 'prefix': 'https://fastly.4sqi.net/img/general/',\n", | |
" 'suffix': '/ya1iQFI7pLjuIJp1PGDKlrZS3OJdHCF7tpILMmjv_2w.jpg',\n", | |
" 'width': 480,\n", | |
" 'height': 640,\n", | |
" 'visibility': 'public'},\n", | |
" 'colors': {'highlightColor': {'photoId': '4fad980de4b091b4626c3633',\n", | |
" 'value': -13619152},\n", | |
" 'highlightTextColor': {'photoId': '4fad980de4b091b4626c3633', 'value': -1},\n", | |
" 'algoVersion': 3}}" | |
] | |
}, | |
"execution_count": 13, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"result = requests.get(url).json()\n", | |
"print(result['response']['venue'].keys())\n", | |
"result['response']['venue']" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"### B. Get the venue's overall rating" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 14, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"name": "stdout", | |
"output_type": "stream", | |
"text": [ | |
"6.4\n" | |
] | |
} | |
], | |
"source": [ | |
"try:\n", | |
" print(result['response']['venue']['rating'])\n", | |
"except:\n", | |
" print('This venue has not been rated yet.')" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"That is not a very good rating. Let's check the rating of the second closest Italian restaurant." | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 15, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"name": "stdout", | |
"output_type": "stream", | |
"text": [ | |
"This venue has not been rated yet.\n" | |
] | |
} | |
], | |
"source": [ | |
"venue_id = '4f3232e219836c91c7bfde94' # ID of Conca Cucina Italian Restaurant\n", | |
"url = 'https://api.foursquare.com/v2/venues/{}?client_id={}&client_secret={}&v={}'.format(venue_id, CLIENT_ID, CLIENT_SECRET, VERSION)\n", | |
"\n", | |
"result = requests.get(url).json()\n", | |
"try:\n", | |
" print(result['response']['venue']['rating'])\n", | |
"except:\n", | |
" print('This venue has not been rated yet.')" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"Since this restaurant has no ratings, let's check the third restaurant." | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 16, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"name": "stdout", | |
"output_type": "stream", | |
"text": [ | |
"7.4\n" | |
] | |
} | |
], | |
"source": [ | |
"venue_id = '3fd66200f964a520f4e41ee3' # ID of Ecco\n", | |
"url = 'https://api.foursquare.com/v2/venues/{}?client_id={}&client_secret={}&v={}'.format(venue_id, CLIENT_ID, CLIENT_SECRET, VERSION)\n", | |
"\n", | |
"result = requests.get(url).json()\n", | |
"try:\n", | |
" print(result['response']['venue']['rating'])\n", | |
"except:\n", | |
" print('This venue has not been rated yet.')" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"Since this restaurant has a slightly better rating, let's explore it further." | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"### C. Get the number of tips" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 17, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"data": { | |
"text/plain": [ | |
"19" | |
] | |
}, | |
"execution_count": 17, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"result['response']['venue']['tips']['count']" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"### D. Get the venue's tips\n", | |
"> `https://api.foursquare.com/v2/venues/`**VENUE_ID**`/tips?client_id=`**CLIENT_ID**`&client_secret=`**CLIENT_SECRET**`&v=`**VERSION**`&limit=`**LIMIT**" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### Create URL and send GET request. Make sure to set limit to get all tips" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 18, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"data": { | |
"text/plain": [ | |
"{'meta': {'code': 200, 'requestId': '5ed643dafb34b5001b5ee6f8'},\n", | |
" 'response': {'tips': {'count': 19,\n", | |
" 'items': [{'id': '5ab1cb46c9a517174651d3fe',\n", | |
" 'createdAt': 1521601350,\n", | |
" 'text': 'A+ Italian food! Trust me on this: my mom’s side of the family is 100% Italian. I was born and bred to know good pasta when I see it, and Ecco is one of my all-time NYC favorites',\n", | |
" 'type': 'user',\n", | |
" 'canonicalUrl': 'https://foursquare.com/item/5ab1cb46c9a517174651d3fe',\n", | |
" 'lang': 'en',\n", | |
" 'likes': {'count': 0, 'groups': []},\n", | |
" 'logView': True,\n", | |
" 'agreeCount': 4,\n", | |
" 'disagreeCount': 0,\n", | |
" 'todo': {'count': 0},\n", | |
" 'user': {'id': '484542633',\n", | |
" 'firstName': 'Nick',\n", | |
" 'lastName': 'E',\n", | |
" 'photo': {'prefix': 'https://fastly.4sqi.net/img/user/',\n", | |
" 'suffix': '/484542633_unymNUmw_FdPs3GjXHujmHcYnN4hf8kEPADlOZuIrdcdm97VX3tFqL7fFNMNA_8Gl9NlU1GYg.jpg'}},\n", | |
" 'authorInteractionType': 'liked'}]}}}" | |
] | |
}, | |
"execution_count": 18, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"## Ecco Tips\n", | |
"limit = 15 # set limit to be greater than or equal to the total number of tips\n", | |
"url = 'https://api.foursquare.com/v2/venues/{}/tips?client_id={}&client_secret={}&v={}&limit={}'.format(venue_id, CLIENT_ID, CLIENT_SECRET, VERSION, limit)\n", | |
"\n", | |
"results = requests.get(url).json()\n", | |
"results" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### Get tips and list of associated features" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 19, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"data": { | |
"text/plain": [ | |
"dict_keys(['id', 'createdAt', 'text', 'type', 'canonicalUrl', 'lang', 'likes', 'logView', 'agreeCount', 'disagreeCount', 'todo', 'user', 'authorInteractionType'])" | |
] | |
}, | |
"execution_count": 19, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"tips = results['response']['tips']['items']\n", | |
"\n", | |
"tip = results['response']['tips']['items'][0]\n", | |
"tip.keys()" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### Format column width and display all tips" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 20, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"name": "stderr", | |
"output_type": "stream", | |
"text": [ | |
"/home/jupyterlab/conda/envs/python/lib/python3.6/site-packages/ipykernel_launcher.py:1: FutureWarning: Passing a negative integer is deprecated in version 1.0 and will not be supported in future version. Instead, use None to not limit the column width.\n", | |
" \"\"\"Entry point for launching an IPython kernel.\n", | |
"/home/jupyterlab/conda/envs/python/lib/python3.6/site-packages/ipykernel_launcher.py:3: FutureWarning: pandas.io.json.json_normalize is deprecated, use pandas.json_normalize instead\n", | |
" This is separate from the ipykernel package so we can avoid doing imports until\n" | |
] | |
}, | |
{ | |
"ename": "KeyError", | |
"evalue": "'Passing list-likes to .loc or [] with any missing labels is no longer supported, see https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#deprecate-loc-reindex-listlike'", | |
"output_type": "error", | |
"traceback": [ | |
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", | |
"\u001b[0;31mKeyError\u001b[0m Traceback (most recent call last)", | |
"\u001b[0;32m<ipython-input-20-fe9c7711676c>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0;31m# columns to keep\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 6\u001b[0m \u001b[0mfiltered_columns\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m'text'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'agreeCount'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'disagreeCount'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'id'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'user.firstName'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'user.lastName'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'user.gender'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'user.id'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 7\u001b[0;31m \u001b[0mtips_filtered\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtips_df\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mloc\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfiltered_columns\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 8\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 9\u001b[0m \u001b[0;31m# display tips\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", | |
"\u001b[0;32m~/conda/envs/python/lib/python3.6/site-packages/pandas/core/indexing.py\u001b[0m in \u001b[0;36m__getitem__\u001b[0;34m(self, key)\u001b[0m\n\u001b[1;32m 1760\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mKeyError\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mIndexError\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mAttributeError\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1761\u001b[0m \u001b[0;32mpass\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1762\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_getitem_tuple\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1763\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1764\u001b[0m \u001b[0;31m# we by definition only have the 0th axis\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", | |
"\u001b[0;32m~/conda/envs/python/lib/python3.6/site-packages/pandas/core/indexing.py\u001b[0m in \u001b[0;36m_getitem_tuple\u001b[0;34m(self, tup)\u001b[0m\n\u001b[1;32m 1287\u001b[0m \u001b[0;32mcontinue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1288\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1289\u001b[0;31m \u001b[0mretval\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mgetattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mretval\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_getitem_axis\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1290\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1291\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mretval\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", | |
"\u001b[0;32m~/conda/envs/python/lib/python3.6/site-packages/pandas/core/indexing.py\u001b[0m in \u001b[0;36m_getitem_axis\u001b[0;34m(self, key, axis)\u001b[0m\n\u001b[1;32m 1952\u001b[0m \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Cannot index with multidimensional key\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1953\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1954\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_getitem_iterable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0maxis\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1955\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1956\u001b[0m \u001b[0;31m# nested tuple slicing\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", | |
"\u001b[0;32m~/conda/envs/python/lib/python3.6/site-packages/pandas/core/indexing.py\u001b[0m in \u001b[0;36m_getitem_iterable\u001b[0;34m(self, key, axis)\u001b[0m\n\u001b[1;32m 1593\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1594\u001b[0m \u001b[0;31m# A collection of keys\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1595\u001b[0;31m \u001b[0mkeyarr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindexer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_get_listlike_indexer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mraise_missing\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1596\u001b[0m return self.obj._reindex_with_indexers(\n\u001b[1;32m 1597\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0maxis\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mkeyarr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindexer\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcopy\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mallow_dups\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", | |
"\u001b[0;32m~/conda/envs/python/lib/python3.6/site-packages/pandas/core/indexing.py\u001b[0m in \u001b[0;36m_get_listlike_indexer\u001b[0;34m(self, key, axis, raise_missing)\u001b[0m\n\u001b[1;32m 1551\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1552\u001b[0m self._validate_read_indexer(\n\u001b[0;32m-> 1553\u001b[0;31m \u001b[0mkeyarr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindexer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mo\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_get_axis_number\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maxis\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mraise_missing\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mraise_missing\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1554\u001b[0m )\n\u001b[1;32m 1555\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mkeyarr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindexer\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", | |
"\u001b[0;32m~/conda/envs/python/lib/python3.6/site-packages/pandas/core/indexing.py\u001b[0m in \u001b[0;36m_validate_read_indexer\u001b[0;34m(self, key, indexer, axis, raise_missing)\u001b[0m\n\u001b[1;32m 1653\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0max\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mis_categorical\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0max\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mis_interval\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1654\u001b[0m raise KeyError(\n\u001b[0;32m-> 1655\u001b[0;31m \u001b[0;34m\"Passing list-likes to .loc or [] with any missing labels \"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1656\u001b[0m \u001b[0;34m\"is no longer supported, see \"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1657\u001b[0m \u001b[0;34m\"https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#deprecate-loc-reindex-listlike\"\u001b[0m \u001b[0;31m# noqa:E501\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", | |
"\u001b[0;31mKeyError\u001b[0m: 'Passing list-likes to .loc or [] with any missing labels is no longer supported, see https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#deprecate-loc-reindex-listlike'" | |
] | |
} | |
], | |
"source": [ | |
"pd.set_option('display.max_colwidth', -1)\n", | |
"\n", | |
"tips_df = json_normalize(tips) # json normalize tips\n", | |
"\n", | |
"# columns to keep\n", | |
"filtered_columns = ['text', 'agreeCount', 'disagreeCount', 'id', 'user.firstName', 'user.lastName', 'user.gender', 'user.id']\n", | |
"tips_filtered = tips_df.loc[:, filtered_columns]\n", | |
"\n", | |
"# display tips\n", | |
"tips_filtered" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"Now remember that because we are using a personal developer account, then we can access only 2 of the restaurant's tips, instead of all 15 tips." | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
" " | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"<a id=\"item3\"></a>" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"## 3. Search a Foursquare User\n", | |
"> `https://api.foursquare.com/v2/users/`**USER_ID**`?client_id=`**CLIENT_ID**`&client_secret=`**CLIENT_SECRET**`&v=`**VERSION**" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"### Define URL, send GET request and display features associated with user" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 21, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"ename": "KeyError", | |
"evalue": "'user'", | |
"output_type": "error", | |
"traceback": [ | |
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", | |
"\u001b[0;31mKeyError\u001b[0m Traceback (most recent call last)", | |
"\u001b[0;32m<ipython-input-21-91224d45fd37>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0;31m# send GET request\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 6\u001b[0m \u001b[0mresults\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mrequests\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0murl\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mjson\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 7\u001b[0;31m \u001b[0muser_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mresults\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'response'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'user'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 8\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 9\u001b[0m \u001b[0;31m# display features associated with user\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", | |
"\u001b[0;31mKeyError\u001b[0m: 'user'" | |
] | |
} | |
], | |
"source": [ | |
"user_id = '484542633' # user ID with most agree counts and complete profile\n", | |
"\n", | |
"url = 'https://api.foursquare.com/v2/users/{}?client_id={}&client_secret={}&v={}'.format(user_id, CLIENT_ID, CLIENT_SECRET, VERSION) # define URL\n", | |
"\n", | |
"# send GET request\n", | |
"results = requests.get(url).json()\n", | |
"user_data = results['response']['user']\n", | |
"\n", | |
"# display features associated with user\n", | |
"user_data.keys()" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": null, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [], | |
"source": [ | |
"print('First Name: ' + user_data['firstName'])\n", | |
"print('Last Name: ' + user_data['lastName'])\n", | |
"print('Home City: ' + user_data['homeCity'])" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### How many tips has this user submitted?" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": null, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [], | |
"source": [ | |
"user_data['tips']" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"Wow! So it turns out that Nick is a very active Foursquare user, with more than 250 tips." | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"### Get User's tips" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": null, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [], | |
"source": [ | |
"# define tips URL\n", | |
"url = 'https://api.foursquare.com/v2/users/{}/tips?client_id={}&client_secret={}&v={}&limit={}'.format(user_id, CLIENT_ID, CLIENT_SECRET, VERSION, limit)\n", | |
"\n", | |
"# send GET request and get user's tips\n", | |
"results = requests.get(url).json()\n", | |
"tips = results['response']['tips']['items']\n", | |
"\n", | |
"# format column width\n", | |
"pd.set_option('display.max_colwidth', -1)\n", | |
"\n", | |
"tips_df = json_normalize(tips)\n", | |
"\n", | |
"# filter columns\n", | |
"filtered_columns = ['text', 'agreeCount', 'disagreeCount', 'id']\n", | |
"tips_filtered = tips_df.loc[:, filtered_columns]\n", | |
"\n", | |
"# display user's tips\n", | |
"tips_filtered" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### Let's get the venue for the tip with the greatest number of agree counts" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": null, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [], | |
"source": [ | |
"tip_id = '5ab5575d73fe2516ad8f363b' # tip id\n", | |
"\n", | |
"# define URL\n", | |
"url = 'http://api.foursquare.com/v2/tips/{}?client_id={}&client_secret={}&v={}'.format(tip_id, CLIENT_ID, CLIENT_SECRET, VERSION)\n", | |
"\n", | |
"# send GET Request and examine results\n", | |
"result = requests.get(url).json()\n", | |
"print(result['response']['tip']['venue']['name'])\n", | |
"print(result['response']['tip']['venue']['location'])" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"### Get User's friends" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": null, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [], | |
"source": [ | |
"user_friends = json_normalize(user_data['friends']['groups'][0]['items'])\n", | |
"user_friends" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"Interesting. Despite being very active, it turns out that Nick does not have any friends on Foursquare. This might definitely change in the future." | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"### Retrieve the User's Profile Image" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": null, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
}, | |
"scrolled": true | |
}, | |
"outputs": [], | |
"source": [ | |
"user_data" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": null, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [], | |
"source": [ | |
"# 1. grab prefix of photo\n", | |
"# 2. grab suffix of photo\n", | |
"# 3. concatenate them using the image size \n", | |
"Image(url='https://igx.4sqi.net/img/user/300x300/484542633_mK2Yum7T_7Tn9fWpndidJsmw2Hof_6T5vJBKCHPLMK5OL-U5ZiJGj51iwBstcpDLYa3Zvhvis.jpg')" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
" " | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"<a id=\"item4\"></a>" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"## 4. Explore a location\n", | |
"> `https://api.foursquare.com/v2/venues/`**explore**`?client_id=`**CLIENT_ID**`&client_secret=`**CLIENT_SECRET**`&ll=`**LATITUDE**`,`**LONGITUDE**`&v=`**VERSION**`&limit=`**LIMIT**" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### So, you just finished your gourmet dish at Ecco, and are just curious about the popular spots around the restaurant. In order to explore the area, let's start by getting the latitude and longitude values of Ecco Restaurant." | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 22, | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [], | |
"source": [ | |
"latitude = 40.715337\n", | |
"longitude = -74.008848" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### Define URL" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 23, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"data": { | |
"text/plain": [ | |
"'https://api.foursquare.com/v2/venues/explore?client_id=NM2ZJ4MDHZEN140JF3CDFT5JKTFY5XKPACFFJV1LR2ORR3W0&client_secret=GYL1KCRUWNUQKY51A5E4FKOLYSAEG4C4GIO2VEFUNITJYZFC&ll=40.715337,-74.008848&v=20180604&radius=500&limit=30'" | |
] | |
}, | |
"execution_count": 23, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"url = 'https://api.foursquare.com/v2/venues/explore?client_id={}&client_secret={}&ll={},{}&v={}&radius={}&limit={}'.format(CLIENT_ID, CLIENT_SECRET, latitude, longitude, VERSION, radius, LIMIT)\n", | |
"url" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### Send GET request and examine results" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 24, | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [], | |
"source": [ | |
"import requests" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 25, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"data": { | |
"text/plain": [ | |
"'There are 30 around Ecco restaurant.'" | |
] | |
}, | |
"execution_count": 25, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"results = requests.get(url).json()\n", | |
"'There are {} around Ecco restaurant.'.format(len(results['response']['groups'][0]['items']))" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### Get relevant part of JSON" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 26, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"data": { | |
"text/plain": [ | |
"{'reasons': {'count': 0,\n", | |
" 'items': [{'summary': 'This spot is popular',\n", | |
" 'type': 'general',\n", | |
" 'reasonName': 'globalInteractionReason'}]},\n", | |
" 'venue': {'id': '54148bc6498ea7bb8c05b70a',\n", | |
" 'name': 'Juice Press',\n", | |
" 'location': {'address': '83 Murray St',\n", | |
" 'crossStreet': 'btwn Greenwich St & W Broadway',\n", | |
" 'lat': 40.71478769908051,\n", | |
" 'lng': -74.0111317502157,\n", | |
" 'labeledLatLngs': [{'label': 'display',\n", | |
" 'lat': 40.71478769908051,\n", | |
" 'lng': -74.0111317502157}],\n", | |
" 'distance': 202,\n", | |
" 'postalCode': '10007',\n", | |
" 'cc': 'US',\n", | |
" 'city': 'New York',\n", | |
" 'state': 'NY',\n", | |
" 'country': 'United States',\n", | |
" 'formattedAddress': ['83 Murray St (btwn Greenwich St & W Broadway)',\n", | |
" 'New York, NY 10007',\n", | |
" 'United States']},\n", | |
" 'categories': [{'id': '4bf58dd8d48988d1d3941735',\n", | |
" 'name': 'Vegetarian / Vegan Restaurant',\n", | |
" 'pluralName': 'Vegetarian / Vegan Restaurants',\n", | |
" 'shortName': 'Vegetarian / Vegan',\n", | |
" 'icon': {'prefix': 'https://ss3.4sqi.net/img/categories_v2/food/vegetarian_',\n", | |
" 'suffix': '.png'},\n", | |
" 'primary': True}],\n", | |
" 'photos': {'count': 0, 'groups': []}},\n", | |
" 'referralId': 'e-0-54148bc6498ea7bb8c05b70a-0'}" | |
] | |
}, | |
"execution_count": 26, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"items = results['response']['groups'][0]['items']\n", | |
"items[0]" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### Process JSON and convert it to a clean dataframe" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 27, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"name": "stderr", | |
"output_type": "stream", | |
"text": [ | |
"/home/jupyterlab/conda/envs/python/lib/python3.6/site-packages/ipykernel_launcher.py:1: FutureWarning: pandas.io.json.json_normalize is deprecated, use pandas.json_normalize instead\n", | |
" \"\"\"Entry point for launching an IPython kernel.\n" | |
] | |
}, | |
{ | |
"data": { | |
"text/html": [ | |
"<div>\n", | |
"<style scoped>\n", | |
" .dataframe tbody tr th:only-of-type {\n", | |
" vertical-align: middle;\n", | |
" }\n", | |
"\n", | |
" .dataframe tbody tr th {\n", | |
" vertical-align: top;\n", | |
" }\n", | |
"\n", | |
" .dataframe thead th {\n", | |
" text-align: right;\n", | |
" }\n", | |
"</style>\n", | |
"<table border=\"1\" class=\"dataframe\">\n", | |
" <thead>\n", | |
" <tr style=\"text-align: right;\">\n", | |
" <th></th>\n", | |
" <th>name</th>\n", | |
" <th>categories</th>\n", | |
" <th>address</th>\n", | |
" <th>crossStreet</th>\n", | |
" <th>lat</th>\n", | |
" <th>lng</th>\n", | |
" <th>labeledLatLngs</th>\n", | |
" <th>distance</th>\n", | |
" <th>postalCode</th>\n", | |
" <th>cc</th>\n", | |
" <th>city</th>\n", | |
" <th>state</th>\n", | |
" <th>country</th>\n", | |
" <th>formattedAddress</th>\n", | |
" <th>neighborhood</th>\n", | |
" <th>id</th>\n", | |
" </tr>\n", | |
" </thead>\n", | |
" <tbody>\n", | |
" <tr>\n", | |
" <th>0</th>\n", | |
" <td>Juice Press</td>\n", | |
" <td>Vegetarian / Vegan Restaurant</td>\n", | |
" <td>83 Murray St</td>\n", | |
" <td>btwn Greenwich St & W Broadway</td>\n", | |
" <td>40.714788</td>\n", | |
" <td>-74.011132</td>\n", | |
" <td>[{'label': 'display', 'lat': 40.71478769908051, 'lng': -74.0111317502157}]</td>\n", | |
" <td>202</td>\n", | |
" <td>10007</td>\n", | |
" <td>US</td>\n", | |
" <td>New York</td>\n", | |
" <td>NY</td>\n", | |
" <td>United States</td>\n", | |
" <td>[83 Murray St (btwn Greenwich St & W Broadway), New York, NY 10007, United States]</td>\n", | |
" <td>NaN</td>\n", | |
" <td>54148bc6498ea7bb8c05b70a</td>\n", | |
" </tr>\n", | |
" <tr>\n", | |
" <th>1</th>\n", | |
" <td>Korin</td>\n", | |
" <td>Furniture / Home Store</td>\n", | |
" <td>57 Warren St</td>\n", | |
" <td>Church St</td>\n", | |
" <td>40.714824</td>\n", | |
" <td>-74.009404</td>\n", | |
" <td>[{'label': 'display', 'lat': 40.71482437714839, 'lng': -74.00940425461492}, {'label': 'entrance', 'lat': 40.714727, 'lng': -74.009399}]</td>\n", | |
" <td>73</td>\n", | |
" <td>10007</td>\n", | |
" <td>US</td>\n", | |
" <td>New York</td>\n", | |
" <td>NY</td>\n", | |
" <td>United States</td>\n", | |
" <td>[57 Warren St (Church St), New York, NY 10007, United States]</td>\n", | |
" <td>Tribeca</td>\n", | |
" <td>4af5d65ff964a52091fd21e3</td>\n", | |
" </tr>\n", | |
" <tr>\n", | |
" <th>2</th>\n", | |
" <td>Los Tacos No. 1</td>\n", | |
" <td>Taco Place</td>\n", | |
" <td>136 Church St</td>\n", | |
" <td>NaN</td>\n", | |
" <td>40.714267</td>\n", | |
" <td>-74.008756</td>\n", | |
" <td>[{'label': 'display', 'lat': 40.714267, 'lng': -74.008756}]</td>\n", | |
" <td>119</td>\n", | |
" <td>10007</td>\n", | |
" <td>US</td>\n", | |
" <td>New York</td>\n", | |
" <td>NY</td>\n", | |
" <td>United States</td>\n", | |
" <td>[136 Church St, New York, NY 10007, United States]</td>\n", | |
" <td>NaN</td>\n", | |
" <td>5d5f24ec09484500079aee00</td>\n", | |
" </tr>\n", | |
" <tr>\n", | |
" <th>3</th>\n", | |
" <td>Takahachi Bakery</td>\n", | |
" <td>Bakery</td>\n", | |
" <td>25 Murray St</td>\n", | |
" <td>at Church St</td>\n", | |
" <td>40.713653</td>\n", | |
" <td>-74.008804</td>\n", | |
" <td>[{'label': 'display', 'lat': 40.713652845301894, 'lng': -74.0088038953017}, {'label': 'entrance', 'lat': 40.713716, 'lng': -74.008443}]</td>\n", | |
" <td>187</td>\n", | |
" <td>10007</td>\n", | |
" <td>US</td>\n", | |
" <td>New York</td>\n", | |
" <td>NY</td>\n", | |
" <td>United States</td>\n", | |
" <td>[25 Murray St (at Church St), New York, NY 10007, United States]</td>\n", | |
" <td>NaN</td>\n", | |
" <td>4c154c9a77cea593c401d260</td>\n", | |
" </tr>\n", | |
" <tr>\n", | |
" <th>4</th>\n", | |
" <td>Takahachi</td>\n", | |
" <td>Sushi Restaurant</td>\n", | |
" <td>145 Duane St</td>\n", | |
" <td>btwn W Broadway & Church St</td>\n", | |
" <td>40.716526</td>\n", | |
" <td>-74.008101</td>\n", | |
" <td>[{'label': 'display', 'lat': 40.71652647412374, 'lng': -74.00810108466207}, {'label': 'entrance', 'lat': 40.716508, 'lng': -74.007989}]</td>\n", | |
" <td>146</td>\n", | |
" <td>10013</td>\n", | |
" <td>US</td>\n", | |
" <td>New York</td>\n", | |
" <td>NY</td>\n", | |
" <td>United States</td>\n", | |
" <td>[145 Duane St (btwn W Broadway & Church St), New York, NY 10013, United States]</td>\n", | |
" <td>NaN</td>\n", | |
" <td>4a8f2f39f964a520471420e3</td>\n", | |
" </tr>\n", | |
" <tr>\n", | |
" <th>5</th>\n", | |
" <td>Exceed Physical Culture</td>\n", | |
" <td>Gym / Fitness Center</td>\n", | |
" <td>97 Reade St</td>\n", | |
" <td>bet W Broadway & Church St</td>\n", | |
" <td>40.715629</td>\n", | |
" <td>-74.007992</td>\n", | |
" <td>[{'label': 'display', 'lat': 40.7156286200256, 'lng': -74.0079922583853}, {'label': 'entrance', 'lat': 40.715589, 'lng': -74.008105}]</td>\n", | |
" <td>79</td>\n", | |
" <td>10013</td>\n", | |
" <td>US</td>\n", | |
" <td>New York</td>\n", | |
" <td>NY</td>\n", | |
" <td>United States</td>\n", | |
" <td>[97 Reade St (bet W Broadway & Church St), New York, NY 10013, United States]</td>\n", | |
" <td>Tribeca</td>\n", | |
" <td>53910ac3498e57a5dc0eb160</td>\n", | |
" </tr>\n", | |
" <tr>\n", | |
" <th>6</th>\n", | |
" <td>Equinox Tribeca</td>\n", | |
" <td>Gym</td>\n", | |
" <td>54 Murray St</td>\n", | |
" <td>at W Broadway</td>\n", | |
" <td>40.714099</td>\n", | |
" <td>-74.009686</td>\n", | |
" <td>[{'label': 'display', 'lat': 40.71409860726041, 'lng': -74.0096857179283}]</td>\n", | |
" <td>154</td>\n", | |
" <td>10007</td>\n", | |
" <td>US</td>\n", | |
" <td>New York</td>\n", | |
" <td>NY</td>\n", | |
" <td>United States</td>\n", | |
" <td>[54 Murray St (at W Broadway), New York, NY 10007, United States]</td>\n", | |
" <td>NaN</td>\n", | |
" <td>4a6e331af964a52031d41fe3</td>\n", | |
" </tr>\n", | |
" <tr>\n", | |
" <th>7</th>\n", | |
" <td>Whole Foods Market</td>\n", | |
" <td>Grocery Store</td>\n", | |
" <td>270 Greenwich Street</td>\n", | |
" <td>at Warren St</td>\n", | |
" <td>40.715579</td>\n", | |
" <td>-74.011368</td>\n", | |
" <td>[{'label': 'display', 'lat': 40.715579155420606, 'lng': -74.01136823958119}]</td>\n", | |
" <td>214</td>\n", | |
" <td>10007</td>\n", | |
" <td>US</td>\n", | |
" <td>New York</td>\n", | |
" <td>NY</td>\n", | |
" <td>United States</td>\n", | |
" <td>[270 Greenwich Street (at Warren St), New York, NY 10007, United States]</td>\n", | |
" <td>Tribeca</td>\n", | |
" <td>49bc3b0af964a52020541fe3</td>\n", | |
" </tr>\n", | |
" <tr>\n", | |
" <th>8</th>\n", | |
" <td>Nish Nūsh</td>\n", | |
" <td>Falafel Restaurant</td>\n", | |
" <td>88 Reade St</td>\n", | |
" <td>at Church St</td>\n", | |
" <td>40.715537</td>\n", | |
" <td>-74.007725</td>\n", | |
" <td>[{'label': 'display', 'lat': 40.71553710116416, 'lng': -74.00772452925565}, {'label': 'entrance', 'lat': 40.715615, 'lng': -74.00773}]</td>\n", | |
" <td>97</td>\n", | |
" <td>10013</td>\n", | |
" <td>US</td>\n", | |
" <td>New York</td>\n", | |
" <td>NY</td>\n", | |
" <td>United States</td>\n", | |
" <td>[88 Reade St (at Church St), New York, NY 10013, United States]</td>\n", | |
" <td>NaN</td>\n", | |
" <td>50ba9119e4b071a4bae6dc10</td>\n", | |
" </tr>\n", | |
" <tr>\n", | |
" <th>9</th>\n", | |
" <td>Philip Williams Posters</td>\n", | |
" <td>Antique Shop</td>\n", | |
" <td>122 Chambers St</td>\n", | |
" <td>NaN</td>\n", | |
" <td>40.715284</td>\n", | |
" <td>-74.008781</td>\n", | |
" <td>[{'label': 'display', 'lat': 40.71528423132827, 'lng': -74.00878093952018}, {'label': 'entrance', 'lat': 40.715188, 'lng': -74.008747}]</td>\n", | |
" <td>8</td>\n", | |
" <td>10007</td>\n", | |
" <td>US</td>\n", | |
" <td>New York</td>\n", | |
" <td>NY</td>\n", | |
" <td>United States</td>\n", | |
" <td>[122 Chambers St, New York, NY 10007, United States]</td>\n", | |
" <td>NaN</td>\n", | |
" <td>4b747291f964a52042dd2de3</td>\n", | |
" </tr>\n", | |
" </tbody>\n", | |
"</table>\n", | |
"</div>" | |
], | |
"text/plain": [ | |
" name categories \\\n", | |
"0 Juice Press Vegetarian / Vegan Restaurant \n", | |
"1 Korin Furniture / Home Store \n", | |
"2 Los Tacos No. 1 Taco Place \n", | |
"3 Takahachi Bakery Bakery \n", | |
"4 Takahachi Sushi Restaurant \n", | |
"5 Exceed Physical Culture Gym / Fitness Center \n", | |
"6 Equinox Tribeca Gym \n", | |
"7 Whole Foods Market Grocery Store \n", | |
"8 Nish Nūsh Falafel Restaurant \n", | |
"9 Philip Williams Posters Antique Shop \n", | |
"\n", | |
" address crossStreet lat lng \\\n", | |
"0 83 Murray St btwn Greenwich St & W Broadway 40.714788 -74.011132 \n", | |
"1 57 Warren St Church St 40.714824 -74.009404 \n", | |
"2 136 Church St NaN 40.714267 -74.008756 \n", | |
"3 25 Murray St at Church St 40.713653 -74.008804 \n", | |
"4 145 Duane St btwn W Broadway & Church St 40.716526 -74.008101 \n", | |
"5 97 Reade St bet W Broadway & Church St 40.715629 -74.007992 \n", | |
"6 54 Murray St at W Broadway 40.714099 -74.009686 \n", | |
"7 270 Greenwich Street at Warren St 40.715579 -74.011368 \n", | |
"8 88 Reade St at Church St 40.715537 -74.007725 \n", | |
"9 122 Chambers St NaN 40.715284 -74.008781 \n", | |
"\n", | |
" labeledLatLngs \\\n", | |
"0 [{'label': 'display', 'lat': 40.71478769908051, 'lng': -74.0111317502157}] \n", | |
"1 [{'label': 'display', 'lat': 40.71482437714839, 'lng': -74.00940425461492}, {'label': 'entrance', 'lat': 40.714727, 'lng': -74.009399}] \n", | |
"2 [{'label': 'display', 'lat': 40.714267, 'lng': -74.008756}] \n", | |
"3 [{'label': 'display', 'lat': 40.713652845301894, 'lng': -74.0088038953017}, {'label': 'entrance', 'lat': 40.713716, 'lng': -74.008443}] \n", | |
"4 [{'label': 'display', 'lat': 40.71652647412374, 'lng': -74.00810108466207}, {'label': 'entrance', 'lat': 40.716508, 'lng': -74.007989}] \n", | |
"5 [{'label': 'display', 'lat': 40.7156286200256, 'lng': -74.0079922583853}, {'label': 'entrance', 'lat': 40.715589, 'lng': -74.008105}] \n", | |
"6 [{'label': 'display', 'lat': 40.71409860726041, 'lng': -74.0096857179283}] \n", | |
"7 [{'label': 'display', 'lat': 40.715579155420606, 'lng': -74.01136823958119}] \n", | |
"8 [{'label': 'display', 'lat': 40.71553710116416, 'lng': -74.00772452925565}, {'label': 'entrance', 'lat': 40.715615, 'lng': -74.00773}] \n", | |
"9 [{'label': 'display', 'lat': 40.71528423132827, 'lng': -74.00878093952018}, {'label': 'entrance', 'lat': 40.715188, 'lng': -74.008747}] \n", | |
"\n", | |
" distance postalCode cc city state country \\\n", | |
"0 202 10007 US New York NY United States \n", | |
"1 73 10007 US New York NY United States \n", | |
"2 119 10007 US New York NY United States \n", | |
"3 187 10007 US New York NY United States \n", | |
"4 146 10013 US New York NY United States \n", | |
"5 79 10013 US New York NY United States \n", | |
"6 154 10007 US New York NY United States \n", | |
"7 214 10007 US New York NY United States \n", | |
"8 97 10013 US New York NY United States \n", | |
"9 8 10007 US New York NY United States \n", | |
"\n", | |
" formattedAddress \\\n", | |
"0 [83 Murray St (btwn Greenwich St & W Broadway), New York, NY 10007, United States] \n", | |
"1 [57 Warren St (Church St), New York, NY 10007, United States] \n", | |
"2 [136 Church St, New York, NY 10007, United States] \n", | |
"3 [25 Murray St (at Church St), New York, NY 10007, United States] \n", | |
"4 [145 Duane St (btwn W Broadway & Church St), New York, NY 10013, United States] \n", | |
"5 [97 Reade St (bet W Broadway & Church St), New York, NY 10013, United States] \n", | |
"6 [54 Murray St (at W Broadway), New York, NY 10007, United States] \n", | |
"7 [270 Greenwich Street (at Warren St), New York, NY 10007, United States] \n", | |
"8 [88 Reade St (at Church St), New York, NY 10013, United States] \n", | |
"9 [122 Chambers St, New York, NY 10007, United States] \n", | |
"\n", | |
" neighborhood id \n", | |
"0 NaN 54148bc6498ea7bb8c05b70a \n", | |
"1 Tribeca 4af5d65ff964a52091fd21e3 \n", | |
"2 NaN 5d5f24ec09484500079aee00 \n", | |
"3 NaN 4c154c9a77cea593c401d260 \n", | |
"4 NaN 4a8f2f39f964a520471420e3 \n", | |
"5 Tribeca 53910ac3498e57a5dc0eb160 \n", | |
"6 NaN 4a6e331af964a52031d41fe3 \n", | |
"7 Tribeca 49bc3b0af964a52020541fe3 \n", | |
"8 NaN 50ba9119e4b071a4bae6dc10 \n", | |
"9 NaN 4b747291f964a52042dd2de3 " | |
] | |
}, | |
"execution_count": 27, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"dataframe = json_normalize(items) # flatten JSON\n", | |
"\n", | |
"# filter columns\n", | |
"filtered_columns = ['venue.name', 'venue.categories'] + [col for col in dataframe.columns if col.startswith('venue.location.')] + ['venue.id']\n", | |
"dataframe_filtered = dataframe.loc[:, filtered_columns]\n", | |
"\n", | |
"# filter the category for each row\n", | |
"dataframe_filtered['venue.categories'] = dataframe_filtered.apply(get_category_type, axis=1)\n", | |
"\n", | |
"# clean columns\n", | |
"dataframe_filtered.columns = [col.split('.')[-1] for col in dataframe_filtered.columns]\n", | |
"\n", | |
"dataframe_filtered.head(10)" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### Let's visualize these items on the map around our location" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 28, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"data": { | |
"text/html": [ | |
"<div style=\"width:100%;\"><div style=\"position:relative;width:100%;height:0;padding-bottom:60%;\"><span style=\"color:#565656\">Make this Notebook Trusted to load map: File -> Trust Notebook</span><iframe src=\"about:blank\" style=\"position:absolute;width:100%;height:100%;left:0;top:0;border:none !important;\" data-html=PCFET0NUWVBFIGh0bWw+CjxoZWFkPiAgICAKICAgIDxtZXRhIGh0dHAtZXF1aXY9ImNvbnRlbnQtdHlwZSIgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PVVURi04IiAvPgogICAgPHNjcmlwdD5MX1BSRUZFUl9DQU5WQVMgPSBmYWxzZTsgTF9OT19UT1VDSCA9IGZhbHNlOyBMX0RJU0FCTEVfM0QgPSBmYWxzZTs8L3NjcmlwdD4KICAgIDxzY3JpcHQgc3JjPSJodHRwczovL2Nkbi5qc2RlbGl2ci5uZXQvbnBtL2xlYWZsZXRAMS4yLjAvZGlzdC9sZWFmbGV0LmpzIj48L3NjcmlwdD4KICAgIDxzY3JpcHQgc3JjPSJodHRwczovL2FqYXguZ29vZ2xlYXBpcy5jb20vYWpheC9saWJzL2pxdWVyeS8xLjExLjEvanF1ZXJ5Lm1pbi5qcyI+PC9zY3JpcHQ+CiAgICA8c2NyaXB0IHNyYz0iaHR0cHM6Ly9tYXhjZG4uYm9vdHN0cmFwY2RuLmNvbS9ib290c3RyYXAvMy4yLjAvanMvYm9vdHN0cmFwLm1pbi5qcyI+PC9zY3JpcHQ+CiAgICA8c2NyaXB0IHNyYz0iaHR0cHM6Ly9jZG5qcy5jbG91ZGZsYXJlLmNvbS9hamF4L2xpYnMvTGVhZmxldC5hd2Vzb21lLW1hcmtlcnMvMi4wLjIvbGVhZmxldC5hd2Vzb21lLW1hcmtlcnMuanMiPjwvc2NyaXB0PgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL2Nkbi5qc2RlbGl2ci5uZXQvbnBtL2xlYWZsZXRAMS4yLjAvZGlzdC9sZWFmbGV0LmNzcyIvPgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL21heGNkbi5ib290c3RyYXBjZG4uY29tL2Jvb3RzdHJhcC8zLjIuMC9jc3MvYm9vdHN0cmFwLm1pbi5jc3MiLz4KICAgIDxsaW5rIHJlbD0ic3R5bGVzaGVldCIgaHJlZj0iaHR0cHM6Ly9tYXhjZG4uYm9vdHN0cmFwY2RuLmNvbS9ib290c3RyYXAvMy4yLjAvY3NzL2Jvb3RzdHJhcC10aGVtZS5taW4uY3NzIi8+CiAgICA8bGluayByZWw9InN0eWxlc2hlZXQiIGhyZWY9Imh0dHBzOi8vbWF4Y2RuLmJvb3RzdHJhcGNkbi5jb20vZm9udC1hd2Vzb21lLzQuNi4zL2Nzcy9mb250LWF3ZXNvbWUubWluLmNzcyIvPgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL2NkbmpzLmNsb3VkZmxhcmUuY29tL2FqYXgvbGlicy9MZWFmbGV0LmF3ZXNvbWUtbWFya2Vycy8yLjAuMi9sZWFmbGV0LmF3ZXNvbWUtbWFya2Vycy5jc3MiLz4KICAgIDxsaW5rIHJlbD0ic3R5bGVzaGVldCIgaHJlZj0iaHR0cHM6Ly9yYXdnaXQuY29tL3B5dGhvbi12aXN1YWxpemF0aW9uL2ZvbGl1bS9tYXN0ZXIvZm9saXVtL3RlbXBsYXRlcy9sZWFmbGV0LmF3ZXNvbWUucm90YXRlLmNzcyIvPgogICAgPHN0eWxlPmh0bWwsIGJvZHkge3dpZHRoOiAxMDAlO2hlaWdodDogMTAwJTttYXJnaW46IDA7cGFkZGluZzogMDt9PC9zdHlsZT4KICAgIDxzdHlsZT4jbWFwIHtwb3NpdGlvbjphYnNvbHV0ZTt0b3A6MDtib3R0b206MDtyaWdodDowO2xlZnQ6MDt9PC9zdHlsZT4KICAgIAogICAgICAgICAgICA8c3R5bGU+ICNtYXBfYWNjMWI1N2E5Nzg4NDJlMDg4MTBlNzQ2NjE3YjgwNmQgewogICAgICAgICAgICAgICAgcG9zaXRpb24gOiByZWxhdGl2ZTsKICAgICAgICAgICAgICAgIHdpZHRoIDogMTAwLjAlOwogICAgICAgICAgICAgICAgaGVpZ2h0OiAxMDAuMCU7CiAgICAgICAgICAgICAgICBsZWZ0OiAwLjAlOwogICAgICAgICAgICAgICAgdG9wOiAwLjAlOwogICAgICAgICAgICAgICAgfQogICAgICAgICAgICA8L3N0eWxlPgogICAgICAgIAo8L2hlYWQ+Cjxib2R5PiAgICAKICAgIAogICAgICAgICAgICA8ZGl2IGNsYXNzPSJmb2xpdW0tbWFwIiBpZD0ibWFwX2FjYzFiNTdhOTc4ODQyZTA4ODEwZTc0NjYxN2I4MDZkIiA+PC9kaXY+CiAgICAgICAgCjwvYm9keT4KPHNjcmlwdD4gICAgCiAgICAKCiAgICAgICAgICAgIAogICAgICAgICAgICAgICAgdmFyIGJvdW5kcyA9IG51bGw7CiAgICAgICAgICAgIAoKICAgICAgICAgICAgdmFyIG1hcF9hY2MxYjU3YTk3ODg0MmUwODgxMGU3NDY2MTdiODA2ZCA9IEwubWFwKAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgJ21hcF9hY2MxYjU3YTk3ODg0MmUwODgxMGU3NDY2MTdiODA2ZCcsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB7Y2VudGVyOiBbNDAuNzE1MzM3LC03NC4wMDg4NDhdLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgem9vbTogMTUsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBtYXhCb3VuZHM6IGJvdW5kcywKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGxheWVyczogW10sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB3b3JsZENvcHlKdW1wOiBmYWxzZSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNyczogTC5DUlMuRVBTRzM4NTcKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfSk7CiAgICAgICAgICAgIAogICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciB0aWxlX2xheWVyX2M4ZGI0M2M2MjRjNjQwZTJiZDhlOTNjOWMyZjM3ODVhID0gTC50aWxlTGF5ZXIoCiAgICAgICAgICAgICAgICAnaHR0cHM6Ly97c30udGlsZS5vcGVuc3RyZWV0bWFwLm9yZy97en0ve3h9L3t5fS5wbmcnLAogICAgICAgICAgICAgICAgewogICJhdHRyaWJ1dGlvbiI6IG51bGwsCiAgImRldGVjdFJldGluYSI6IGZhbHNlLAogICJtYXhab29tIjogMTgsCiAgIm1pblpvb20iOiAxLAogICJub1dyYXAiOiBmYWxzZSwKICAic3ViZG9tYWlucyI6ICJhYmMiCn0KICAgICAgICAgICAgICAgICkuYWRkVG8obWFwX2FjYzFiNTdhOTc4ODQyZTA4ODEwZTc0NjYxN2I4MDZkKTsKICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgY2lyY2xlX21hcmtlcl8yM2FiNjk0MWVmOWM0MWU5OWYyMzgyOTEzN2JmNjc5NCA9IEwuY2lyY2xlTWFya2VyKAogICAgICAgICAgICAgICAgWzQwLjcxNTMzNywtNzQuMDA4ODQ4XSwKICAgICAgICAgICAgICAgIHsKICAiYnViYmxpbmdNb3VzZUV2ZW50cyI6IHRydWUsCiAgImNvbG9yIjogInJlZCIsCiAgImRhc2hBcnJheSI6IG51bGwsCiAgImRhc2hPZmZzZXQiOiBudWxsLAogICJmaWxsIjogdHJ1ZSwKICAiZmlsbENvbG9yIjogInJlZCIsCiAgImZpbGxPcGFjaXR5IjogMC42LAogICJmaWxsUnVsZSI6ICJldmVub2RkIiwKICAibGluZUNhcCI6ICJyb3VuZCIsCiAgImxpbmVKb2luIjogInJvdW5kIiwKICAib3BhY2l0eSI6IDEuMCwKICAicmFkaXVzIjogMTAsCiAgInN0cm9rZSI6IHRydWUsCiAgIndlaWdodCI6IDMKfQogICAgICAgICAgICAgICAgKS5hZGRUbyhtYXBfYWNjMWI1N2E5Nzg4NDJlMDg4MTBlNzQ2NjE3YjgwNmQpOwogICAgICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgcG9wdXBfMWRlMWJjYzJmMWQyNDE2MGIwYzMzNzViYzcwNjAzZDAgPSBMLnBvcHVwKHttYXhXaWR0aDogJzMwMCd9KTsKCiAgICAgICAgICAgIAogICAgICAgICAgICAgICAgdmFyIGh0bWxfODdmYWQyNzgzMTQxNDI4N2IxYTBlOTA5NzVmMDYwOTMgPSAkKCc8ZGl2IGlkPSJodG1sXzg3ZmFkMjc4MzE0MTQyODdiMWEwZTkwOTc1ZjA2MDkzIiBzdHlsZT0id2lkdGg6IDEwMC4wJTsgaGVpZ2h0OiAxMDAuMCU7Ij5FY2NvPC9kaXY+JylbMF07CiAgICAgICAgICAgICAgICBwb3B1cF8xZGUxYmNjMmYxZDI0MTYwYjBjMzM3NWJjNzA2MDNkMC5zZXRDb250ZW50KGh0bWxfODdmYWQyNzgzMTQxNDI4N2IxYTBlOTA5NzVmMDYwOTMpOwogICAgICAgICAgICAKCiAgICAgICAgICAgIGNpcmNsZV9tYXJrZXJfMjNhYjY5NDFlZjljNDFlOTlmMjM4MjkxMzdiZjY3OTQuYmluZFBvcHVwKHBvcHVwXzFkZTFiY2MyZjFkMjQxNjBiMGMzMzc1YmM3MDYwM2QwKTsKCiAgICAgICAgICAgIAogICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBjaXJjbGVfbWFya2VyX2YyZjg0MTA1YTMwNjQ2MWNhYWRlNmEyNGMyY2JkYmZhID0gTC5jaXJjbGVNYXJrZXIoCiAgICAgICAgICAgICAgICBbNDAuNzE0Nzg3Njk5MDgwNTEsLTc0LjAxMTEzMTc1MDIxNTddLAogICAgICAgICAgICAgICAgewogICJidWJibGluZ01vdXNlRXZlbnRzIjogdHJ1ZSwKICAiY29sb3IiOiAiYmx1ZSIsCiAgImRhc2hBcnJheSI6IG51bGwsCiAgImRhc2hPZmZzZXQiOiBudWxsLAogICJmaWxsIjogdHJ1ZSwKICAiZmlsbENvbG9yIjogImJsdWUiLAogICJmaWxsT3BhY2l0eSI6IDAuNiwKICAiZmlsbFJ1bGUiOiAiZXZlbm9kZCIsCiAgImxpbmVDYXAiOiAicm91bmQiLAogICJsaW5lSm9pbiI6ICJyb3VuZCIsCiAgIm9wYWNpdHkiOiAxLjAsCiAgInJhZGl1cyI6IDUsCiAgInN0cm9rZSI6IHRydWUsCiAgIndlaWdodCI6IDMKfQogICAgICAgICAgICAgICAgKS5hZGRUbyhtYXBfYWNjMWI1N2E5Nzg4NDJlMDg4MTBlNzQ2NjE3YjgwNmQpOwogICAgICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgcG9wdXBfMzgyMGRhN2E3NjA5NGFmNWE2ZjYzNDE4MmQ0MDRjMzIgPSBMLnBvcHVwKHttYXhXaWR0aDogJzMwMCd9KTsKCiAgICAgICAgICAgIAogICAgICAgICAgICAgICAgdmFyIGh0bWxfYjkwYWFmMDJhM2MxNDY3ZWFkMTA0NWI4MjEwYTc4OGMgPSAkKCc8ZGl2IGlkPSJodG1sX2I5MGFhZjAyYTNjMTQ2N2VhZDEwNDViODIxMGE3ODhjIiBzdHlsZT0id2lkdGg6IDEwMC4wJTsgaGVpZ2h0OiAxMDAuMCU7Ij5WZWdldGFyaWFuIC8gVmVnYW4gUmVzdGF1cmFudDwvZGl2PicpWzBdOwogICAgICAgICAgICAgICAgcG9wdXBfMzgyMGRhN2E3NjA5NGFmNWE2ZjYzNDE4MmQ0MDRjMzIuc2V0Q29udGVudChodG1sX2I5MGFhZjAyYTNjMTQ2N2VhZDEwNDViODIxMGE3ODhjKTsKICAgICAgICAgICAgCgogICAgICAgICAgICBjaXJjbGVfbWFya2VyX2YyZjg0MTA1YTMwNjQ2MWNhYWRlNmEyNGMyY2JkYmZhLmJpbmRQb3B1cChwb3B1cF8zODIwZGE3YTc2MDk0YWY1YTZmNjM0MTgyZDQwNGMzMik7CgogICAgICAgICAgICAKICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgY2lyY2xlX21hcmtlcl9mOGI3OTcwNWJmNzk0NTUzYmY5MmMzZDEyNGQ4Y2NmOSA9IEwuY2lyY2xlTWFya2VyKAogICAgICAgICAgICAgICAgWzQwLjcxNDgyNDM3NzE0ODM5LC03NC4wMDk0MDQyNTQ2MTQ5Ml0sCiAgICAgICAgICAgICAgICB7CiAgImJ1YmJsaW5nTW91c2VFdmVudHMiOiB0cnVlLAogICJjb2xvciI6ICJibHVlIiwKICAiZGFzaEFycmF5IjogbnVsbCwKICAiZGFzaE9mZnNldCI6IG51bGwsCiAgImZpbGwiOiB0cnVlLAogICJmaWxsQ29sb3IiOiAiYmx1ZSIsCiAgImZpbGxPcGFjaXR5IjogMC42LAogICJmaWxsUnVsZSI6ICJldmVub2RkIiwKICAibGluZUNhcCI6ICJyb3VuZCIsCiAgImxpbmVKb2luIjogInJvdW5kIiwKICAib3BhY2l0eSI6IDEuMCwKICAicmFkaXVzIjogNSwKICAic3Ryb2tlIjogdHJ1ZSwKICAid2VpZ2h0IjogMwp9CiAgICAgICAgICAgICAgICApLmFkZFRvKG1hcF9hY2MxYjU3YTk3ODg0MmUwODgxMGU3NDY2MTdiODA2ZCk7CiAgICAgICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBwb3B1cF82Mzc0ZGM0YTBlNzE0YzcwOWI1YTM3NzZjMGQ2NGFkYiA9IEwucG9wdXAoe21heFdpZHRoOiAnMzAwJ30pOwoKICAgICAgICAgICAgCiAgICAgICAgICAgICAgICB2YXIgaHRtbF81OWJmZjU0OGY1ZGE0MjBlODY5ZjYwNTdmMTI0NTUzNCA9ICQoJzxkaXYgaWQ9Imh0bWxfNTliZmY1NDhmNWRhNDIwZTg2OWY2MDU3ZjEyNDU1MzQiIHN0eWxlPSJ3aWR0aDogMTAwLjAlOyBoZWlnaHQ6IDEwMC4wJTsiPkZ1cm5pdHVyZSAvIEhvbWUgU3RvcmU8L2Rpdj4nKVswXTsKICAgICAgICAgICAgICAgIHBvcHVwXzYzNzRkYzRhMGU3MTRjNzA5YjVhMzc3NmMwZDY0YWRiLnNldENvbnRlbnQoaHRtbF81OWJmZjU0OGY1ZGE0MjBlODY5ZjYwNTdmMTI0NTUzNCk7CiAgICAgICAgICAgIAoKICAgICAgICAgICAgY2lyY2xlX21hcmtlcl9mOGI3OTcwNWJmNzk0NTUzYmY5MmMzZDEyNGQ4Y2NmOS5iaW5kUG9wdXAocG9wdXBfNjM3NGRjNGEwZTcxNGM3MDliNWEzNzc2YzBkNjRhZGIpOwoKICAgICAgICAgICAgCiAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIGNpcmNsZV9tYXJrZXJfOGRhMzc3ZDlkZmQzNGQzZjgyYzkyODc4NGU5NmM0MDMgPSBMLmNpcmNsZU1hcmtlcigKICAgICAgICAgICAgICAgIFs0MC43MTQyNjcsLTc0LjAwODc1Nl0sCiAgICAgICAgICAgICAgICB7CiAgImJ1YmJsaW5nTW91c2VFdmVudHMiOiB0cnVlLAogICJjb2xvciI6ICJibHVlIiwKICAiZGFzaEFycmF5IjogbnVsbCwKICAiZGFzaE9mZnNldCI6IG51bGwsCiAgImZpbGwiOiB0cnVlLAogICJmaWxsQ29sb3IiOiAiYmx1ZSIsCiAgImZpbGxPcGFjaXR5IjogMC42LAogICJmaWxsUnVsZSI6ICJldmVub2RkIiwKICAibGluZUNhcCI6ICJyb3VuZCIsCiAgImxpbmVKb2luIjogInJvdW5kIiwKICAib3BhY2l0eSI6IDEuMCwKICAicmFkaXVzIjogNSwKICAic3Ryb2tlIjogdHJ1ZSwKICAid2VpZ2h0IjogMwp9CiAgICAgICAgICAgICAgICApLmFkZFRvKG1hcF9hY2MxYjU3YTk3ODg0MmUwODgxMGU3NDY2MTdiODA2ZCk7CiAgICAgICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBwb3B1cF8yN2UwMjBjMWVhYWU0ZDNiOGI4ZWU4NWY2ZGNlOTgwZSA9IEwucG9wdXAoe21heFdpZHRoOiAnMzAwJ30pOwoKICAgICAgICAgICAgCiAgICAgICAgICAgICAgICB2YXIgaHRtbF82NjViMzUzZTgyYTQ0YTJhYmE5MWMwNGI2MzFmNThmNiA9ICQoJzxkaXYgaWQ9Imh0bWxfNjY1YjM1M2U4MmE0NGEyYWJhOTFjMDRiNjMxZjU4ZjYiIHN0eWxlPSJ3aWR0aDogMTAwLjAlOyBoZWlnaHQ6IDEwMC4wJTsiPlRhY28gUGxhY2U8L2Rpdj4nKVswXTsKICAgICAgICAgICAgICAgIHBvcHVwXzI3ZTAyMGMxZWFhZTRkM2I4YjhlZTg1ZjZkY2U5ODBlLnNldENvbnRlbnQoaHRtbF82NjViMzUzZTgyYTQ0YTJhYmE5MWMwNGI2MzFmNThmNik7CiAgICAgICAgICAgIAoKICAgICAgICAgICAgY2lyY2xlX21hcmtlcl84ZGEzNzdkOWRmZDM0ZDNmODJjOTI4Nzg0ZTk2YzQwMy5iaW5kUG9wdXAocG9wdXBfMjdlMDIwYzFlYWFlNGQzYjhiOGVlODVmNmRjZTk4MGUpOwoKICAgICAgICAgICAgCiAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIGNpcmNsZV9tYXJrZXJfZGNiNWQ0NmVmZDFlNDk3MDg0OTYxNmY4MzdiYzA5ODYgPSBMLmNpcmNsZU1hcmtlcigKICAgICAgICAgICAgICAgIFs0MC43MTM2NTI4NDUzMDE4OTQsLTc0LjAwODgwMzg5NTMwMTddLAogICAgICAgICAgICAgICAgewogICJidWJibGluZ01vdXNlRXZlbnRzIjogdHJ1ZSwKICAiY29sb3IiOiAiYmx1ZSIsCiAgImRhc2hBcnJheSI6IG51bGwsCiAgImRhc2hPZmZzZXQiOiBudWxsLAogICJmaWxsIjogdHJ1ZSwKICAiZmlsbENvbG9yIjogImJsdWUiLAogICJmaWxsT3BhY2l0eSI6IDAuNiwKICAiZmlsbFJ1bGUiOiAiZXZlbm9kZCIsCiAgImxpbmVDYXAiOiAicm91bmQiLAogICJsaW5lSm9pbiI6ICJyb3VuZCIsCiAgIm9wYWNpdHkiOiAxLjAsCiAgInJhZGl1cyI6IDUsCiAgInN0cm9rZSI6IHRydWUsCiAgIndlaWdodCI6IDMKfQogICAgICAgICAgICAgICAgKS5hZGRUbyhtYXBfYWNjMWI1N2E5Nzg4NDJlMDg4MTBlNzQ2NjE3YjgwNmQpOwogICAgICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgcG9wdXBfNzliM2FkZTc5ZjQ0NDA3N2FkNTlkOWFkNmZiYTVmOWMgPSBMLnBvcHVwKHttYXhXaWR0aDogJzMwMCd9KTsKCiAgICAgICAgICAgIAogICAgICAgICAgICAgICAgdmFyIGh0bWxfODM5M2I3ZjE5MTQ5NDFiNDk0ODJhZjdkOWYyZDhhNzcgPSAkKCc8ZGl2IGlkPSJodG1sXzgzOTNiN2YxOTE0OTQxYjQ5NDgyYWY3ZDlmMmQ4YTc3IiBzdHlsZT0id2lkdGg6IDEwMC4wJTsgaGVpZ2h0OiAxMDAuMCU7Ij5CYWtlcnk8L2Rpdj4nKVswXTsKICAgICAgICAgICAgICAgIHBvcHVwXzc5YjNhZGU3OWY0NDQwNzdhZDU5ZDlhZDZmYmE1ZjljLnNldENvbnRlbnQoaHRtbF84MzkzYjdmMTkxNDk0MWI0OTQ4MmFmN2Q5ZjJkOGE3Nyk7CiAgICAgICAgICAgIAoKICAgICAgICAgICAgY2lyY2xlX21hcmtlcl9kY2I1ZDQ2ZWZkMWU0OTcwODQ5NjE2ZjgzN2JjMDk4Ni5iaW5kUG9wdXAocG9wdXBfNzliM2FkZTc5ZjQ0NDA3N2FkNTlkOWFkNmZiYTVmOWMpOwoKICAgICAgICAgICAgCiAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIGNpcmNsZV9tYXJrZXJfOGYzZjRhMjcyZmFjNDAwNThhYjExMTY5OTY0ZDg1ZjIgPSBMLmNpcmNsZU1hcmtlcigKICAgICAgICAgICAgICAgIFs0MC43MTY1MjY0NzQxMjM3NCwtNzQuMDA4MTAxMDg0NjYyMDddLAogICAgICAgICAgICAgICAgewogICJidWJibGluZ01vdXNlRXZlbnRzIjogdHJ1ZSwKICAiY29sb3IiOiAiYmx1ZSIsCiAgImRhc2hBcnJheSI6IG51bGwsCiAgImRhc2hPZmZzZXQiOiBudWxsLAogICJmaWxsIjogdHJ1ZSwKICAiZmlsbENvbG9yIjogImJsdWUiLAogICJmaWxsT3BhY2l0eSI6IDAuNiwKICAiZmlsbFJ1bGUiOiAiZXZlbm9kZCIsCiAgImxpbmVDYXAiOiAicm91bmQiLAogICJsaW5lSm9pbiI6ICJyb3VuZCIsCiAgIm9wYWNpdHkiOiAxLjAsCiAgInJhZGl1cyI6IDUsCiAgInN0cm9rZSI6IHRydWUsCiAgIndlaWdodCI6IDMKfQogICAgICAgICAgICAgICAgKS5hZGRUbyhtYXBfYWNjMWI1N2E5Nzg4NDJlMDg4MTBlNzQ2NjE3YjgwNmQpOwogICAgICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgcG9wdXBfMzQ4YmM1NjllOTY2NDAyMjlmZDI2NzFiZWQyYmEyYjQgPSBMLnBvcHVwKHttYXhXaWR0aDogJzMwMCd9KTsKCiAgICAgICAgICAgIAogICAgICAgICAgICAgICAgdmFyIGh0bWxfODI4MDZmNzdkNTVmNDcyYjkzNjYzYTQyYzFjNmM2MDIgPSAkKCc8ZGl2IGlkPSJodG1sXzgyODA2Zjc3ZDU1ZjQ3MmI5MzY2M2E0MmMxYzZjNjAyIiBzdHlsZT0id2lkdGg6IDEwMC4wJTsgaGVpZ2h0OiAxMDAuMCU7Ij5TdXNoaSBSZXN0YXVyYW50PC9kaXY+JylbMF07CiAgICAgICAgICAgICAgICBwb3B1cF8zNDhiYzU2OWU5NjY0MDIyOWZkMjY3MWJlZDJiYTJiNC5zZXRDb250ZW50KGh0bWxfODI4MDZmNzdkNTVmNDcyYjkzNjYzYTQyYzFjNmM2MDIpOwogICAgICAgICAgICAKCiAgICAgICAgICAgIGNpcmNsZV9tYXJrZXJfOGYzZjRhMjcyZmFjNDAwNThhYjExMTY5OTY0ZDg1ZjIuYmluZFBvcHVwKHBvcHVwXzM0OGJjNTY5ZTk2NjQwMjI5ZmQyNjcxYmVkMmJhMmI0KTsKCiAgICAgICAgICAgIAogICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBjaXJjbGVfbWFya2VyXzQxMjA2NmI2ZjU0ODRiNWE5ZTdkZDEzODFkNDZmNDA3ID0gTC5jaXJjbGVNYXJrZXIoCiAgICAgICAgICAgICAgICBbNDAuNzE1NjI4NjIwMDI1NiwtNzQuMDA3OTkyMjU4Mzg1M10sCiAgICAgICAgICAgICAgICB7CiAgImJ1YmJsaW5nTW91c2VFdmVudHMiOiB0cnVlLAogICJjb2xvciI6ICJibHVlIiwKICAiZGFzaEFycmF5IjogbnVsbCwKICAiZGFzaE9mZnNldCI6IG51bGwsCiAgImZpbGwiOiB0cnVlLAogICJmaWxsQ29sb3IiOiAiYmx1ZSIsCiAgImZpbGxPcGFjaXR5IjogMC42LAogICJmaWxsUnVsZSI6ICJldmVub2RkIiwKICAibGluZUNhcCI6ICJyb3VuZCIsCiAgImxpbmVKb2luIjogInJvdW5kIiwKICAib3BhY2l0eSI6IDEuMCwKICAicmFkaXVzIjogNSwKICAic3Ryb2tlIjogdHJ1ZSwKICAid2VpZ2h0IjogMwp9CiAgICAgICAgICAgICAgICApLmFkZFRvKG1hcF9hY2MxYjU3YTk3ODg0MmUwODgxMGU3NDY2MTdiODA2ZCk7CiAgICAgICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBwb3B1cF9hMzBlZmNjYjk3MDk0MzQxOTY1NzE4NmI3ZDg2NzIxZiA9IEwucG9wdXAoe21heFdpZHRoOiAnMzAwJ30pOwoKICAgICAgICAgICAgCiAgICAgICAgICAgICAgICB2YXIgaHRtbF81YTZlM2Y0MWI5NzQ0NTA1YjVjM2U1NjE4OTcyOTEyZSA9ICQoJzxkaXYgaWQ9Imh0bWxfNWE2ZTNmNDFiOTc0NDUwNWI1YzNlNTYxODk3MjkxMmUiIHN0eWxlPSJ3aWR0aDogMTAwLjAlOyBoZWlnaHQ6IDEwMC4wJTsiPkd5bSAvIEZpdG5lc3MgQ2VudGVyPC9kaXY+JylbMF07CiAgICAgICAgICAgICAgICBwb3B1cF9hMzBlZmNjYjk3MDk0MzQxOTY1NzE4NmI3ZDg2NzIxZi5zZXRDb250ZW50KGh0bWxfNWE2ZTNmNDFiOTc0NDUwNWI1YzNlNTYxODk3MjkxMmUpOwogICAgICAgICAgICAKCiAgICAgICAgICAgIGNpcmNsZV9tYXJrZXJfNDEyMDY2YjZmNTQ4NGI1YTllN2RkMTM4MWQ0NmY0MDcuYmluZFBvcHVwKHBvcHVwX2EzMGVmY2NiOTcwOTQzNDE5NjU3MTg2YjdkODY3MjFmKTsKCiAgICAgICAgICAgIAogICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBjaXJjbGVfbWFya2VyXzNiZTE2NzEwNjgxZTQ4MzZiY2IxZjMxNTRiOGQyNGJkID0gTC5jaXJjbGVNYXJrZXIoCiAgICAgICAgICAgICAgICBbNDAuNzE0MDk4NjA3MjYwNDEsLTc0LjAwOTY4NTcxNzkyODNdLAogICAgICAgICAgICAgICAgewogICJidWJibGluZ01vdXNlRXZlbnRzIjogdHJ1ZSwKICAiY29sb3IiOiAiYmx1ZSIsCiAgImRhc2hBcnJheSI6IG51bGwsCiAgImRhc2hPZmZzZXQiOiBudWxsLAogICJmaWxsIjogdHJ1ZSwKICAiZmlsbENvbG9yIjogImJsdWUiLAogICJmaWxsT3BhY2l0eSI6IDAuNiwKICAiZmlsbFJ1bGUiOiAiZXZlbm9kZCIsCiAgImxpbmVDYXAiOiAicm91bmQiLAogICJsaW5lSm9pbiI6ICJyb3VuZCIsCiAgIm9wYWNpdHkiOiAxLjAsCiAgInJhZGl1cyI6IDUsCiAgInN0cm9rZSI6IHRydWUsCiAgIndlaWdodCI6IDMKfQogICAgICAgICAgICAgICAgKS5hZGRUbyhtYXBfYWNjMWI1N2E5Nzg4NDJlMDg4MTBlNzQ2NjE3YjgwNmQpOwogICAgICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgcG9wdXBfYWJhMDY4MzUyZjk4NDhmN2FkMDgxYzZjNzVjODNjZmYgPSBMLnBvcHVwKHttYXhXaWR0aDogJzMwMCd9KTsKCiAgICAgICAgICAgIAogICAgICAgICAgICAgICAgdmFyIGh0bWxfYTQ5YzhjODc2OTZjNDJjNTk5MmMwMjJlOThhZjM2NGUgPSAkKCc8ZGl2IGlkPSJodG1sX2E0OWM4Yzg3Njk2YzQyYzU5OTJjMDIyZTk4YWYzNjRlIiBzdHlsZT0id2lkdGg6IDEwMC4wJTsgaGVpZ2h0OiAxMDAuMCU7Ij5HeW08L2Rpdj4nKVswXTsKICAgICAgICAgICAgICAgIHBvcHVwX2FiYTA2ODM1MmY5ODQ4ZjdhZDA4MWM2Yzc1YzgzY2ZmLnNldENvbnRlbnQoaHRtbF9hNDljOGM4NzY5NmM0MmM1OTkyYzAyMmU5OGFmMzY0ZSk7CiAgICAgICAgICAgIAoKICAgICAgICAgICAgY2lyY2xlX21hcmtlcl8zYmUxNjcxMDY4MWU0ODM2YmNiMWYzMTU0YjhkMjRiZC5iaW5kUG9wdXAocG9wdXBfYWJhMDY4MzUyZjk4NDhmN2FkMDgxYzZjNzVjODNjZmYpOwoKICAgICAgICAgICAgCiAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIGNpcmNsZV9tYXJrZXJfMDQ3NWIyNjhmZmJkNDEyYTgyNzA1Yzk5Njc5YzVkNzEgPSBMLmNpcmNsZU1hcmtlcigKICAgICAgICAgICAgICAgIFs0MC43MTU1NzkxNTU0MjA2MDYsLTc0LjAxMTM2ODIzOTU4MTE5XSwKICAgICAgICAgICAgICAgIHsKICAiYnViYmxpbmdNb3VzZUV2ZW50cyI6IHRydWUsCiAgImNvbG9yIjogImJsdWUiLAogICJkYXNoQXJyYXkiOiBudWxsLAogICJkYXNoT2Zmc2V0IjogbnVsbCwKICAiZmlsbCI6IHRydWUsCiAgImZpbGxDb2xvciI6ICJibHVlIiwKICAiZmlsbE9wYWNpdHkiOiAwLjYsCiAgImZpbGxSdWxlIjogImV2ZW5vZGQiLAogICJsaW5lQ2FwIjogInJvdW5kIiwKICAibGluZUpvaW4iOiAicm91bmQiLAogICJvcGFjaXR5IjogMS4wLAogICJyYWRpdXMiOiA1LAogICJzdHJva2UiOiB0cnVlLAogICJ3ZWlnaHQiOiAzCn0KICAgICAgICAgICAgICAgICkuYWRkVG8obWFwX2FjYzFiNTdhOTc4ODQyZTA4ODEwZTc0NjYxN2I4MDZkKTsKICAgICAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIHBvcHVwXzk5ZTI5YTdjZTlmNTQxZjg4MjY3OTI3NjdmZTAzMmQ5ID0gTC5wb3B1cCh7bWF4V2lkdGg6ICczMDAnfSk7CgogICAgICAgICAgICAKICAgICAgICAgICAgICAgIHZhciBodG1sXzhiZGY3ZDhlY2E2ZTQ4YzRiMDg0ODU1MDYwMTNhNGM0ID0gJCgnPGRpdiBpZD0iaHRtbF84YmRmN2Q4ZWNhNmU0OGM0YjA4NDg1NTA2MDEzYTRjNCIgc3R5bGU9IndpZHRoOiAxMDAuMCU7IGhlaWdodDogMTAwLjAlOyI+R3JvY2VyeSBTdG9yZTwvZGl2PicpWzBdOwogICAgICAgICAgICAgICAgcG9wdXBfOTllMjlhN2NlOWY1NDFmODgyNjc5Mjc2N2ZlMDMyZDkuc2V0Q29udGVudChodG1sXzhiZGY3ZDhlY2E2ZTQ4YzRiMDg0ODU1MDYwMTNhNGM0KTsKICAgICAgICAgICAgCgogICAgICAgICAgICBjaXJjbGVfbWFya2VyXzA0NzViMjY4ZmZiZDQxMmE4MjcwNWM5OTY3OWM1ZDcxLmJpbmRQb3B1cChwb3B1cF85OWUyOWE3Y2U5ZjU0MWY4ODI2NzkyNzY3ZmUwMzJkOSk7CgogICAgICAgICAgICAKICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgY2lyY2xlX21hcmtlcl83MWYwZjhlNWI2ZWQ0OTMxOTgzNzliOTk5MzViNTg4ZiA9IEwuY2lyY2xlTWFya2VyKAogICAgICAgICAgICAgICAgWzQwLjcxNTUzNzEwMTE2NDE2LC03NC4wMDc3MjQ1MjkyNTU2NV0sCiAgICAgICAgICAgICAgICB7CiAgImJ1YmJsaW5nTW91c2VFdmVudHMiOiB0cnVlLAogICJjb2xvciI6ICJibHVlIiwKICAiZGFzaEFycmF5IjogbnVsbCwKICAiZGFzaE9mZnNldCI6IG51bGwsCiAgImZpbGwiOiB0cnVlLAogICJmaWxsQ29sb3IiOiAiYmx1ZSIsCiAgImZpbGxPcGFjaXR5IjogMC42LAogICJmaWxsUnVsZSI6ICJldmVub2RkIiwKICAibGluZUNhcCI6ICJyb3VuZCIsCiAgImxpbmVKb2luIjogInJvdW5kIiwKICAib3BhY2l0eSI6IDEuMCwKICAicmFkaXVzIjogNSwKICAic3Ryb2tlIjogdHJ1ZSwKICAid2VpZ2h0IjogMwp9CiAgICAgICAgICAgICAgICApLmFkZFRvKG1hcF9hY2MxYjU3YTk3ODg0MmUwODgxMGU3NDY2MTdiODA2ZCk7CiAgICAgICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBwb3B1cF9lYWMwM2U3MmRjNDg0NWFiOGZlM2IxYjVkNWJiMDU4ZSA9IEwucG9wdXAoe21heFdpZHRoOiAnMzAwJ30pOwoKICAgICAgICAgICAgCiAgICAgICAgICAgICAgICB2YXIgaHRtbF9jZjA0NWEwMjhkZTQ0NTgyYjAyMzM1ZmE2YjI1NDU2NCA9ICQoJzxkaXYgaWQ9Imh0bWxfY2YwNDVhMDI4ZGU0NDU4MmIwMjMzNWZhNmIyNTQ1NjQiIHN0eWxlPSJ3aWR0aDogMTAwLjAlOyBoZWlnaHQ6IDEwMC4wJTsiPkZhbGFmZWwgUmVzdGF1cmFudDwvZGl2PicpWzBdOwogICAgICAgICAgICAgICAgcG9wdXBfZWFjMDNlNzJkYzQ4NDVhYjhmZTNiMWI1ZDViYjA1OGUuc2V0Q29udGVudChodG1sX2NmMDQ1YTAyOGRlNDQ1ODJiMDIzMzVmYTZiMjU0NTY0KTsKICAgICAgICAgICAgCgogICAgICAgICAgICBjaXJjbGVfbWFya2VyXzcxZjBmOGU1YjZlZDQ5MzE5ODM3OWI5OTkzNWI1ODhmLmJpbmRQb3B1cChwb3B1cF9lYWMwM2U3MmRjNDg0NWFiOGZlM2IxYjVkNWJiMDU4ZSk7CgogICAgICAgICAgICAKICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgY2lyY2xlX21hcmtlcl84YTdkNTJmMGI2Zjk0ZWNkOWJiOWEyOTJjYTFjODZjZCA9IEwuY2lyY2xlTWFya2VyKAogICAgICAgICAgICAgICAgWzQwLjcxNTI4NDIzMTMyODI3LC03NC4wMDg3ODA5Mzk1MjAxOF0sCiAgICAgICAgICAgICAgICB7CiAgImJ1YmJsaW5nTW91c2VFdmVudHMiOiB0cnVlLAogICJjb2xvciI6ICJibHVlIiwKICAiZGFzaEFycmF5IjogbnVsbCwKICAiZGFzaE9mZnNldCI6IG51bGwsCiAgImZpbGwiOiB0cnVlLAogICJmaWxsQ29sb3IiOiAiYmx1ZSIsCiAgImZpbGxPcGFjaXR5IjogMC42LAogICJmaWxsUnVsZSI6ICJldmVub2RkIiwKICAibGluZUNhcCI6ICJyb3VuZCIsCiAgImxpbmVKb2luIjogInJvdW5kIiwKICAib3BhY2l0eSI6IDEuMCwKICAicmFkaXVzIjogNSwKICAic3Ryb2tlIjogdHJ1ZSwKICAid2VpZ2h0IjogMwp9CiAgICAgICAgICAgICAgICApLmFkZFRvKG1hcF9hY2MxYjU3YTk3ODg0MmUwODgxMGU3NDY2MTdiODA2ZCk7CiAgICAgICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBwb3B1cF81YzJmOWZlZjBkZTg0MTJkODUyZmQ2Y2I5YzhmYTBhOCA9IEwucG9wdXAoe21heFdpZHRoOiAnMzAwJ30pOwoKICAgICAgICAgICAgCiAgICAgICAgICAgICAgICB2YXIgaHRtbF9iMmM2NTY2NTliZDk0Njc0ODkzYTg2ZjczODcwMWQzZCA9ICQoJzxkaXYgaWQ9Imh0bWxfYjJjNjU2NjU5YmQ5NDY3NDg5M2E4NmY3Mzg3MDFkM2QiIHN0eWxlPSJ3aWR0aDogMTAwLjAlOyBoZWlnaHQ6IDEwMC4wJTsiPkFudGlxdWUgU2hvcDwvZGl2PicpWzBdOwogICAgICAgICAgICAgICAgcG9wdXBfNWMyZjlmZWYwZGU4NDEyZDg1MmZkNmNiOWM4ZmEwYTguc2V0Q29udGVudChodG1sX2IyYzY1NjY1OWJkOTQ2NzQ4OTNhODZmNzM4NzAxZDNkKTsKICAgICAgICAgICAgCgogICAgICAgICAgICBjaXJjbGVfbWFya2VyXzhhN2Q1MmYwYjZmOTRlY2Q5YmI5YTI5MmNhMWM4NmNkLmJpbmRQb3B1cChwb3B1cF81YzJmOWZlZjBkZTg0MTJkODUyZmQ2Y2I5YzhmYTBhOCk7CgogICAgICAgICAgICAKICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgY2lyY2xlX21hcmtlcl8zYTk0ZWNhYjRhOWM0Zjk4YTU2ODZiZjU3ZDg2MjVlNyA9IEwuY2lyY2xlTWFya2VyKAogICAgICAgICAgICAgICAgWzQwLjcxNzAxMDExNDA5OTA2LC03NC4wMDgwNDI0NDU2MjIyNV0sCiAgICAgICAgICAgICAgICB7CiAgImJ1YmJsaW5nTW91c2VFdmVudHMiOiB0cnVlLAogICJjb2xvciI6ICJibHVlIiwKICAiZGFzaEFycmF5IjogbnVsbCwKICAiZGFzaE9mZnNldCI6IG51bGwsCiAgImZpbGwiOiB0cnVlLAogICJmaWxsQ29sb3IiOiAiYmx1ZSIsCiAgImZpbGxPcGFjaXR5IjogMC42LAogICJmaWxsUnVsZSI6ICJldmVub2RkIiwKICAibGluZUNhcCI6ICJyb3VuZCIsCiAgImxpbmVKb2luIjogInJvdW5kIiwKICAib3BhY2l0eSI6IDEuMCwKICAicmFkaXVzIjogNSwKICAic3Ryb2tlIjogdHJ1ZSwKICAid2VpZ2h0IjogMwp9CiAgICAgICAgICAgICAgICApLmFkZFRvKG1hcF9hY2MxYjU3YTk3ODg0MmUwODgxMGU3NDY2MTdiODA2ZCk7CiAgICAgICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBwb3B1cF9jMDU2ZWY0ODkzYmE0YTRmYmQ5ZDk1NzUxM2Q5ODMzYiA9IEwucG9wdXAoe21heFdpZHRoOiAnMzAwJ30pOwoKICAgICAgICAgICAgCiAgICAgICAgICAgICAgICB2YXIgaHRtbF9hMDkxMjdjYTg5Mzg0MDMyYTY4Y2I4NGFkMmM1MGViZSA9ICQoJzxkaXYgaWQ9Imh0bWxfYTA5MTI3Y2E4OTM4NDAzMmE2OGNiODRhZDJjNTBlYmUiIHN0eWxlPSJ3aWR0aDogMTAwLjAlOyBoZWlnaHQ6IDEwMC4wJTsiPkZyZW5jaCBSZXN0YXVyYW50PC9kaXY+JylbMF07CiAgICAgICAgICAgICAgICBwb3B1cF9jMDU2ZWY0ODkzYmE0YTRmYmQ5ZDk1NzUxM2Q5ODMzYi5zZXRDb250ZW50KGh0bWxfYTA5MTI3Y2E4OTM4NDAzMmE2OGNiODRhZDJjNTBlYmUpOwogICAgICAgICAgICAKCiAgICAgICAgICAgIGNpcmNsZV9tYXJrZXJfM2E5NGVjYWI0YTljNGY5OGE1Njg2YmY1N2Q4NjI1ZTcuYmluZFBvcHVwKHBvcHVwX2MwNTZlZjQ4OTNiYTRhNGZiZDlkOTU3NTEzZDk4MzNiKTsKCiAgICAgICAgICAgIAogICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBjaXJjbGVfbWFya2VyXzdlMTA5ZDY2NTRkNTRiYzFiN2JlMDUxYWU3MzFiM2Y0ID0gTC5jaXJjbGVNYXJrZXIoCiAgICAgICAgICAgICAgICBbNDAuNzE2Mzc5ODQzMTcwNzEsLTc0LjAwOTYyOTMzNDUzNDI4XSwKICAgICAgICAgICAgICAgIHsKICAiYnViYmxpbmdNb3VzZUV2ZW50cyI6IHRydWUsCiAgImNvbG9yIjogImJsdWUiLAogICJkYXNoQXJyYXkiOiBudWxsLAogICJkYXNoT2Zmc2V0IjogbnVsbCwKICAiZmlsbCI6IHRydWUsCiAgImZpbGxDb2xvciI6ICJibHVlIiwKICAiZmlsbE9wYWNpdHkiOiAwLjYsCiAgImZpbGxSdWxlIjogImV2ZW5vZGQiLAogICJsaW5lQ2FwIjogInJvdW5kIiwKICAibGluZUpvaW4iOiAicm91bmQiLAogICJvcGFjaXR5IjogMS4wLAogICJyYWRpdXMiOiA1LAogICJzdHJva2UiOiB0cnVlLAogICJ3ZWlnaHQiOiAzCn0KICAgICAgICAgICAgICAgICkuYWRkVG8obWFwX2FjYzFiNTdhOTc4ODQyZTA4ODEwZTc0NjYxN2I4MDZkKTsKICAgICAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIHBvcHVwXzE4NWY4ZTQ2MzhlNjRlOTZhZmM2NzgzMDQxZDQ1ZWMwID0gTC5wb3B1cCh7bWF4V2lkdGg6ICczMDAnfSk7CgogICAgICAgICAgICAKICAgICAgICAgICAgICAgIHZhciBodG1sX2M1MjJmYzQ2YTA3YTRhNjFiMmJkYzUyZTBiNThhYjZjID0gJCgnPGRpdiBpZD0iaHRtbF9jNTIyZmM0NmEwN2E0YTYxYjJiZGM1MmUwYjU4YWI2YyIgc3R5bGU9IndpZHRoOiAxMDAuMCU7IGhlaWdodDogMTAwLjAlOyI+TmV3IEFtZXJpY2FuIFJlc3RhdXJhbnQ8L2Rpdj4nKVswXTsKICAgICAgICAgICAgICAgIHBvcHVwXzE4NWY4ZTQ2MzhlNjRlOTZhZmM2NzgzMDQxZDQ1ZWMwLnNldENvbnRlbnQoaHRtbF9jNTIyZmM0NmEwN2E0YTYxYjJiZGM1MmUwYjU4YWI2Yyk7CiAgICAgICAgICAgIAoKICAgICAgICAgICAgY2lyY2xlX21hcmtlcl83ZTEwOWQ2NjU0ZDU0YmMxYjdiZTA1MWFlNzMxYjNmNC5iaW5kUG9wdXAocG9wdXBfMTg1ZjhlNDYzOGU2NGU5NmFmYzY3ODMwNDFkNDVlYzApOwoKICAgICAgICAgICAgCiAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIGNpcmNsZV9tYXJrZXJfNDY1NGRlMjY0NTk3NGI1MDk1ZWIyYmM3ZjY1ODg4ZDEgPSBMLmNpcmNsZU1hcmtlcigKICAgICAgICAgICAgICAgIFs0MC43MTU0ODY1ODUyNDk3MzUsLTc0LjAwOTEzMzEzNTEwODM2XSwKICAgICAgICAgICAgICAgIHsKICAiYnViYmxpbmdNb3VzZUV2ZW50cyI6IHRydWUsCiAgImNvbG9yIjogImJsdWUiLAogICJkYXNoQXJyYXkiOiBudWxsLAogICJkYXNoT2Zmc2V0IjogbnVsbCwKICAiZmlsbCI6IHRydWUsCiAgImZpbGxDb2xvciI6ICJibHVlIiwKICAiZmlsbE9wYWNpdHkiOiAwLjYsCiAgImZpbGxSdWxlIjogImV2ZW5vZGQiLAogICJsaW5lQ2FwIjogInJvdW5kIiwKICAibGluZUpvaW4iOiAicm91bmQiLAogICJvcGFjaXR5IjogMS4wLAogICJyYWRpdXMiOiA1LAogICJzdHJva2UiOiB0cnVlLAogICJ3ZWlnaHQiOiAzCn0KICAgICAgICAgICAgICAgICkuYWRkVG8obWFwX2FjYzFiNTdhOTc4ODQyZTA4ODEwZTc0NjYxN2I4MDZkKTsKICAgICAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIHBvcHVwX2Q0NzAzZGUxYTUxNDQ1NTViMjQ5ZGM0MmU0ZmI4N2EwID0gTC5wb3B1cCh7bWF4V2lkdGg6ICczMDAnfSk7CgogICAgICAgICAgICAKICAgICAgICAgICAgICAgIHZhciBodG1sXzk2N2YwN2U3MjYxMDRhZmE5Y2M2ZDc2NTAyYjJiYWIzID0gJCgnPGRpdiBpZD0iaHRtbF85NjdmMDdlNzI2MTA0YWZhOWNjNmQ3NjUwMmIyYmFiMyIgc3R5bGU9IndpZHRoOiAxMDAuMCU7IGhlaWdodDogMTAwLjAlOyI+QW1lcmljYW4gUmVzdGF1cmFudDwvZGl2PicpWzBdOwogICAgICAgICAgICAgICAgcG9wdXBfZDQ3MDNkZTFhNTE0NDU1NWIyNDlkYzQyZTRmYjg3YTAuc2V0Q29udGVudChodG1sXzk2N2YwN2U3MjYxMDRhZmE5Y2M2ZDc2NTAyYjJiYWIzKTsKICAgICAgICAgICAgCgogICAgICAgICAgICBjaXJjbGVfbWFya2VyXzQ2NTRkZTI2NDU5NzRiNTA5NWViMmJjN2Y2NTg4OGQxLmJpbmRQb3B1cChwb3B1cF9kNDcwM2RlMWE1MTQ0NTU1YjI0OWRjNDJlNGZiODdhMCk7CgogICAgICAgICAgICAKICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgY2lyY2xlX21hcmtlcl80MGNiMDA1NDg3ZmY0NDEyYmY1M2VkOTZhZGRkOWZiYyA9IEwuY2lyY2xlTWFya2VyKAogICAgICAgICAgICAgICAgWzQwLjcxNjc0MDg0MTYzMzY5LC03NC4wMDg2NjY0NDM4ODkzXSwKICAgICAgICAgICAgICAgIHsKICAiYnViYmxpbmdNb3VzZUV2ZW50cyI6IHRydWUsCiAgImNvbG9yIjogImJsdWUiLAogICJkYXNoQXJyYXkiOiBudWxsLAogICJkYXNoT2Zmc2V0IjogbnVsbCwKICAiZmlsbCI6IHRydWUsCiAgImZpbGxDb2xvciI6ICJibHVlIiwKICAiZmlsbE9wYWNpdHkiOiAwLjYsCiAgImZpbGxSdWxlIjogImV2ZW5vZGQiLAogICJsaW5lQ2FwIjogInJvdW5kIiwKICAibGluZUpvaW4iOiAicm91bmQiLAogICJvcGFjaXR5IjogMS4wLAogICJyYWRpdXMiOiA1LAogICJzdHJva2UiOiB0cnVlLAogICJ3ZWlnaHQiOiAzCn0KICAgICAgICAgICAgICAgICkuYWRkVG8obWFwX2FjYzFiNTdhOTc4ODQyZTA4ODEwZTc0NjYxN2I4MDZkKTsKICAgICAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIHBvcHVwX2Q3OTA1MDViYjY3YTRlOTJiNWFiNTcyZjg0M2U1NTEyID0gTC5wb3B1cCh7bWF4V2lkdGg6ICczMDAnfSk7CgogICAgICAgICAgICAKICAgICAgICAgICAgICAgIHZhciBodG1sX2NhODAxNzJhNGIyZDQxZTk4YTcxY2JmY2VkNGMzMGFiID0gJCgnPGRpdiBpZD0iaHRtbF9jYTgwMTcyYTRiMmQ0MWU5OGE3MWNiZmNlZDRjMzBhYiIgc3R5bGU9IndpZHRoOiAxMDAuMCU7IGhlaWdodDogMTAwLjAlOyI+Q29ja3RhaWwgQmFyPC9kaXY+JylbMF07CiAgICAgICAgICAgICAgICBwb3B1cF9kNzkwNTA1YmI2N2E0ZTkyYjVhYjU3MmY4NDNlNTUxMi5zZXRDb250ZW50KGh0bWxfY2E4MDE3MmE0YjJkNDFlOThhNzFjYmZjZWQ0YzMwYWIpOwogICAgICAgICAgICAKCiAgICAgICAgICAgIGNpcmNsZV9tYXJrZXJfNDBjYjAwNTQ4N2ZmNDQxMmJmNTNlZDk2YWRkZDlmYmMuYmluZFBvcHVwKHBvcHVwX2Q3OTA1MDViYjY3YTRlOTJiNWFiNTcyZjg0M2U1NTEyKTsKCiAgICAgICAgICAgIAogICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBjaXJjbGVfbWFya2VyX2NiZGNhOGNkMmI3MDQxNzI4M2NiMjkyNDI5YjEwMGU4ID0gTC5jaXJjbGVNYXJrZXIoCiAgICAgICAgICAgICAgICBbNDAuNzE3MDQ1OTg4NTM3MDQsLTc0LjAxMTA5NDU3MDE1OTkxXSwKICAgICAgICAgICAgICAgIHsKICAiYnViYmxpbmdNb3VzZUV2ZW50cyI6IHRydWUsCiAgImNvbG9yIjogImJsdWUiLAogICJkYXNoQXJyYXkiOiBudWxsLAogICJkYXNoT2Zmc2V0IjogbnVsbCwKICAiZmlsbCI6IHRydWUsCiAgImZpbGxDb2xvciI6ICJibHVlIiwKICAiZmlsbE9wYWNpdHkiOiAwLjYsCiAgImZpbGxSdWxlIjogImV2ZW5vZGQiLAogICJsaW5lQ2FwIjogInJvdW5kIiwKICAibGluZUpvaW4iOiAicm91bmQiLAogICJvcGFjaXR5IjogMS4wLAogICJyYWRpdXMiOiA1LAogICJzdHJva2UiOiB0cnVlLAogICJ3ZWlnaHQiOiAzCn0KICAgICAgICAgICAgICAgICkuYWRkVG8obWFwX2FjYzFiNTdhOTc4ODQyZTA4ODEwZTc0NjYxN2I4MDZkKTsKICAgICAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIHBvcHVwXzA3ZGQwOWUyOWE1NDQyYWNhMTA5OTkyNDhjYWExMmQ1ID0gTC5wb3B1cCh7bWF4V2lkdGg6ICczMDAnfSk7CgogICAgICAgICAgICAKICAgICAgICAgICAgICAgIHZhciBodG1sXzM2NGIwY2U1NWNmZjQ2MzQ5ZmE0MDQyMzRiNTU4NWFhID0gJCgnPGRpdiBpZD0iaHRtbF8zNjRiMGNlNTVjZmY0NjM0OWZhNDA0MjM0YjU1ODVhYSIgc3R5bGU9IndpZHRoOiAxMDAuMCU7IGhlaWdodDogMTAwLjAlOyI+UGxheWdyb3VuZDwvZGl2PicpWzBdOwogICAgICAgICAgICAgICAgcG9wdXBfMDdkZDA5ZTI5YTU0NDJhY2ExMDk5OTI0OGNhYTEyZDUuc2V0Q29udGVudChodG1sXzM2NGIwY2U1NWNmZjQ2MzQ5ZmE0MDQyMzRiNTU4NWFhKTsKICAgICAgICAgICAgCgogICAgICAgICAgICBjaXJjbGVfbWFya2VyX2NiZGNhOGNkMmI3MDQxNzI4M2NiMjkyNDI5YjEwMGU4LmJpbmRQb3B1cChwb3B1cF8wN2RkMDllMjlhNTQ0MmFjYTEwOTk5MjQ4Y2FhMTJkNSk7CgogICAgICAgICAgICAKICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgY2lyY2xlX21hcmtlcl8wMzEwMmNiMmZmYTM0ZjJlOWFiMzFiN2UxYzVkMWIzYyA9IEwuY2lyY2xlTWFya2VyKAogICAgICAgICAgICAgICAgWzQwLjcxNTU4LC03NC4wMDk4NV0sCiAgICAgICAgICAgICAgICB7CiAgImJ1YmJsaW5nTW91c2VFdmVudHMiOiB0cnVlLAogICJjb2xvciI6ICJibHVlIiwKICAiZGFzaEFycmF5IjogbnVsbCwKICAiZGFzaE9mZnNldCI6IG51bGwsCiAgImZpbGwiOiB0cnVlLAogICJmaWxsQ29sb3IiOiAiYmx1ZSIsCiAgImZpbGxPcGFjaXR5IjogMC42LAogICJmaWxsUnVsZSI6ICJldmVub2RkIiwKICAibGluZUNhcCI6ICJyb3VuZCIsCiAgImxpbmVKb2luIjogInJvdW5kIiwKICAib3BhY2l0eSI6IDEuMCwKICAicmFkaXVzIjogNSwKICAic3Ryb2tlIjogdHJ1ZSwKICAid2VpZ2h0IjogMwp9CiAgICAgICAgICAgICAgICApLmFkZFRvKG1hcF9hY2MxYjU3YTk3ODg0MmUwODgxMGU3NDY2MTdiODA2ZCk7CiAgICAgICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBwb3B1cF8wNmQxMjgxMTYyYjE0ZmNhYWYyNjUyNmY2OTRmNzcyNyA9IEwucG9wdXAoe21heFdpZHRoOiAnMzAwJ30pOwoKICAgICAgICAgICAgCiAgICAgICAgICAgICAgICB2YXIgaHRtbF80YjIzNzMxNGI0YjE0Zjc1OTk5Y2QwZDQzMTAyNjBiMSA9ICQoJzxkaXYgaWQ9Imh0bWxfNGIyMzczMTRiNGIxNGY3NTk5OWNkMGQ0MzEwMjYwYjEiIHN0eWxlPSJ3aWR0aDogMTAwLjAlOyBoZWlnaHQ6IDEwMC4wJTsiPkJhZ2VsIFNob3A8L2Rpdj4nKVswXTsKICAgICAgICAgICAgICAgIHBvcHVwXzA2ZDEyODExNjJiMTRmY2FhZjI2NTI2ZjY5NGY3NzI3LnNldENvbnRlbnQoaHRtbF80YjIzNzMxNGI0YjE0Zjc1OTk5Y2QwZDQzMTAyNjBiMSk7CiAgICAgICAgICAgIAoKICAgICAgICAgICAgY2lyY2xlX21hcmtlcl8wMzEwMmNiMmZmYTM0ZjJlOWFiMzFiN2UxYzVkMWIzYy5iaW5kUG9wdXAocG9wdXBfMDZkMTI4MTE2MmIxNGZjYWFmMjY1MjZmNjk0Zjc3MjcpOwoKICAgICAgICAgICAgCiAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIGNpcmNsZV9tYXJrZXJfNDE0MjJkNzc4MjdmNGFhNWI4OWQxNjA2YWM1NmU4NGIgPSBMLmNpcmNsZU1hcmtlcigKICAgICAgICAgICAgICAgIFs0MC43MTcyMjYxNzI2MzUwMSwtNzQuMDA5NDMyOTcwMTMxMDJdLAogICAgICAgICAgICAgICAgewogICJidWJibGluZ01vdXNlRXZlbnRzIjogdHJ1ZSwKICAiY29sb3IiOiAiYmx1ZSIsCiAgImRhc2hBcnJheSI6IG51bGwsCiAgImRhc2hPZmZzZXQiOiBudWxsLAogICJmaWxsIjogdHJ1ZSwKICAiZmlsbENvbG9yIjogImJsdWUiLAogICJmaWxsT3BhY2l0eSI6IDAuNiwKICAiZmlsbFJ1bGUiOiAiZXZlbm9kZCIsCiAgImxpbmVDYXAiOiAicm91bmQiLAogICJsaW5lSm9pbiI6ICJyb3VuZCIsCiAgIm9wYWNpdHkiOiAxLjAsCiAgInJhZGl1cyI6IDUsCiAgInN0cm9rZSI6IHRydWUsCiAgIndlaWdodCI6IDMKfQogICAgICAgICAgICAgICAgKS5hZGRUbyhtYXBfYWNjMWI1N2E5Nzg4NDJlMDg4MTBlNzQ2NjE3YjgwNmQpOwogICAgICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgcG9wdXBfOGNlN2Y5YTVmM2FiNGQ3MzhmNjAxMjY2NGQxMDBkZTkgPSBMLnBvcHVwKHttYXhXaWR0aDogJzMwMCd9KTsKCiAgICAgICAgICAgIAogICAgICAgICAgICAgICAgdmFyIGh0bWxfZjI3YjFiMzNhZWE4NDAzNmI4N2UzMmUwMjQ0MGJhN2YgPSAkKCc8ZGl2IGlkPSJodG1sX2YyN2IxYjMzYWVhODQwMzZiODdlMzJlMDI0NDBiYTdmIiBzdHlsZT0id2lkdGg6IDEwMC4wJTsgaGVpZ2h0OiAxMDAuMCU7Ij5JdGFsaWFuIFJlc3RhdXJhbnQ8L2Rpdj4nKVswXTsKICAgICAgICAgICAgICAgIHBvcHVwXzhjZTdmOWE1ZjNhYjRkNzM4ZjYwMTI2NjRkMTAwZGU5LnNldENvbnRlbnQoaHRtbF9mMjdiMWIzM2FlYTg0MDM2Yjg3ZTMyZTAyNDQwYmE3Zik7CiAgICAgICAgICAgIAoKICAgICAgICAgICAgY2lyY2xlX21hcmtlcl80MTQyMmQ3NzgyN2Y0YWE1Yjg5ZDE2MDZhYzU2ZTg0Yi5iaW5kUG9wdXAocG9wdXBfOGNlN2Y5YTVmM2FiNGQ3MzhmNjAxMjY2NGQxMDBkZTkpOwoKICAgICAgICAgICAgCiAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIGNpcmNsZV9tYXJrZXJfMjIzZWRhNzUwZjg5NDJjNGI4NGE1ZWFkOThiY2QxOGQgPSBMLmNpcmNsZU1hcmtlcigKICAgICAgICAgICAgICAgIFs0MC43MTcxNzI3NTgwMTE2OCwtNzQuMDA5MzI4NjkxMjUxMTddLAogICAgICAgICAgICAgICAgewogICJidWJibGluZ01vdXNlRXZlbnRzIjogdHJ1ZSwKICAiY29sb3IiOiAiYmx1ZSIsCiAgImRhc2hBcnJheSI6IG51bGwsCiAgImRhc2hPZmZzZXQiOiBudWxsLAogICJmaWxsIjogdHJ1ZSwKICAiZmlsbENvbG9yIjogImJsdWUiLAogICJmaWxsT3BhY2l0eSI6IDAuNiwKICAiZmlsbFJ1bGUiOiAiZXZlbm9kZCIsCiAgImxpbmVDYXAiOiAicm91bmQiLAogICJsaW5lSm9pbiI6ICJyb3VuZCIsCiAgIm9wYWNpdHkiOiAxLjAsCiAgInJhZGl1cyI6IDUsCiAgInN0cm9rZSI6IHRydWUsCiAgIndlaWdodCI6IDMKfQogICAgICAgICAgICAgICAgKS5hZGRUbyhtYXBfYWNjMWI1N2E5Nzg4NDJlMDg4MTBlNzQ2NjE3YjgwNmQpOwogICAgICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgcG9wdXBfNzM4YjE4OWIwMmFkNGFlNWFjOTQyYTExOTFlOGU5YjMgPSBMLnBvcHVwKHttYXhXaWR0aDogJzMwMCd9KTsKCiAgICAgICAgICAgIAogICAgICAgICAgICAgICAgdmFyIGh0bWxfZTk1ODAyZmY4ZDY5NDZhN2FkYzY3NDAzZWM4YTI5MDEgPSAkKCc8ZGl2IGlkPSJodG1sX2U5NTgwMmZmOGQ2OTQ2YTdhZGM2NzQwM2VjOGEyOTAxIiBzdHlsZT0id2lkdGg6IDEwMC4wJTsgaGVpZ2h0OiAxMDAuMCU7Ij5QYXJrPC9kaXY+JylbMF07CiAgICAgICAgICAgICAgICBwb3B1cF83MzhiMTg5YjAyYWQ0YWU1YWM5NDJhMTE5MWU4ZTliMy5zZXRDb250ZW50KGh0bWxfZTk1ODAyZmY4ZDY5NDZhN2FkYzY3NDAzZWM4YTI5MDEpOwogICAgICAgICAgICAKCiAgICAgICAgICAgIGNpcmNsZV9tYXJrZXJfMjIzZWRhNzUwZjg5NDJjNGI4NGE1ZWFkOThiY2QxOGQuYmluZFBvcHVwKHBvcHVwXzczOGIxODliMDJhZDRhZTVhYzk0MmExMTkxZThlOWIzKTsKCiAgICAgICAgICAgIAogICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBjaXJjbGVfbWFya2VyXzFmYWI5ODYyMzlmMjQ2ZjBiYzZiZGY1YjA1MmYzZTNlID0gTC5jaXJjbGVNYXJrZXIoCiAgICAgICAgICAgICAgICBbNDAuNzE1MDQ1MTI1NTg5OTYsLTc0LjAxMTUwODcxMDI4MjFdLAogICAgICAgICAgICAgICAgewogICJidWJibGluZ01vdXNlRXZlbnRzIjogdHJ1ZSwKICAiY29sb3IiOiAiYmx1ZSIsCiAgImRhc2hBcnJheSI6IG51bGwsCiAgImRhc2hPZmZzZXQiOiBudWxsLAogICJmaWxsIjogdHJ1ZSwKICAiZmlsbENvbG9yIjogImJsdWUiLAogICJmaWxsT3BhY2l0eSI6IDAuNiwKICAiZmlsbFJ1bGUiOiAiZXZlbm9kZCIsCiAgImxpbmVDYXAiOiAicm91bmQiLAogICJsaW5lSm9pbiI6ICJyb3VuZCIsCiAgIm9wYWNpdHkiOiAxLjAsCiAgInJhZGl1cyI6IDUsCiAgInN0cm9rZSI6IHRydWUsCiAgIndlaWdodCI6IDMKfQogICAgICAgICAgICAgICAgKS5hZGRUbyhtYXBfYWNjMWI1N2E5Nzg4NDJlMDg4MTBlNzQ2NjE3YjgwNmQpOwogICAgICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgcG9wdXBfODY5YTZiMDlhNTBjNGFkMmI0ZTcyOGNkNzhkOTgzMzIgPSBMLnBvcHVwKHttYXhXaWR0aDogJzMwMCd9KTsKCiAgICAgICAgICAgIAogICAgICAgICAgICAgICAgdmFyIGh0bWxfODJjY2FjYzY3NmQ5NGEwMTgwYmEyNjcwZWI5MDg5YjIgPSAkKCc8ZGl2IGlkPSJodG1sXzgyY2NhY2M2NzZkOTRhMDE4MGJhMjY3MGViOTA4OWIyIiBzdHlsZT0id2lkdGg6IDEwMC4wJTsgaGVpZ2h0OiAxMDAuMCU7Ij5Db2ZmZWUgU2hvcDwvZGl2PicpWzBdOwogICAgICAgICAgICAgICAgcG9wdXBfODY5YTZiMDlhNTBjNGFkMmI0ZTcyOGNkNzhkOTgzMzIuc2V0Q29udGVudChodG1sXzgyY2NhY2M2NzZkOTRhMDE4MGJhMjY3MGViOTA4OWIyKTsKICAgICAgICAgICAgCgogICAgICAgICAgICBjaXJjbGVfbWFya2VyXzFmYWI5ODYyMzlmMjQ2ZjBiYzZiZGY1YjA1MmYzZTNlLmJpbmRQb3B1cChwb3B1cF84NjlhNmIwOWE1MGM0YWQyYjRlNzI4Y2Q3OGQ5ODMzMik7CgogICAgICAgICAgICAKICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgY2lyY2xlX21hcmtlcl9jNjkyYWQ0OGRlNTk0MmNlOTBhNzA2OTIwODJjNWYwNiA9IEwuY2lyY2xlTWFya2VyKAogICAgICAgICAgICAgICAgWzQwLjcxNjgwMjAzMzU3NDEyNiwtNzQuMDEwODc5OTkzNDM4NzJdLAogICAgICAgICAgICAgICAgewogICJidWJibGluZ01vdXNlRXZlbnRzIjogdHJ1ZSwKICAiY29sb3IiOiAiYmx1ZSIsCiAgImRhc2hBcnJheSI6IG51bGwsCiAgImRhc2hPZmZzZXQiOiBudWxsLAogICJmaWxsIjogdHJ1ZSwKICAiZmlsbENvbG9yIjogImJsdWUiLAogICJmaWxsT3BhY2l0eSI6IDAuNiwKICAiZmlsbFJ1bGUiOiAiZXZlbm9kZCIsCiAgImxpbmVDYXAiOiAicm91bmQiLAogICJsaW5lSm9pbiI6ICJyb3VuZCIsCiAgIm9wYWNpdHkiOiAxLjAsCiAgInJhZGl1cyI6IDUsCiAgInN0cm9rZSI6IHRydWUsCiAgIndlaWdodCI6IDMKfQogICAgICAgICAgICAgICAgKS5hZGRUbyhtYXBfYWNjMWI1N2E5Nzg4NDJlMDg4MTBlNzQ2NjE3YjgwNmQpOwogICAgICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgcG9wdXBfY2VmMzhjZTQyMDQ3NDFhNGE2ZTUyNmY5MjlkNGU2MzAgPSBMLnBvcHVwKHttYXhXaWR0aDogJzMwMCd9KTsKCiAgICAgICAgICAgIAogICAgICAgICAgICAgICAgdmFyIGh0bWxfMzJkNzA1OTUxNzI5NGQ1MGE2NmJkOWFkODYxMjIwODMgPSAkKCc8ZGl2IGlkPSJodG1sXzMyZDcwNTk1MTcyOTRkNTBhNjZiZDlhZDg2MTIyMDgzIiBzdHlsZT0id2lkdGg6IDEwMC4wJTsgaGVpZ2h0OiAxMDAuMCU7Ij5GYXJtZXJzIE1hcmtldDwvZGl2PicpWzBdOwogICAgICAgICAgICAgICAgcG9wdXBfY2VmMzhjZTQyMDQ3NDFhNGE2ZTUyNmY5MjlkNGU2MzAuc2V0Q29udGVudChodG1sXzMyZDcwNTk1MTcyOTRkNTBhNjZiZDlhZDg2MTIyMDgzKTsKICAgICAgICAgICAgCgogICAgICAgICAgICBjaXJjbGVfbWFya2VyX2M2OTJhZDQ4ZGU1OTQyY2U5MGE3MDY5MjA4MmM1ZjA2LmJpbmRQb3B1cChwb3B1cF9jZWYzOGNlNDIwNDc0MWE0YTZlNTI2ZjkyOWQ0ZTYzMCk7CgogICAgICAgICAgICAKICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgY2lyY2xlX21hcmtlcl84NTU0MmEyYWJiZDk0ZGU1ODgxYzlkYjkyYTIxY2RjNCA9IEwuY2lyY2xlTWFya2VyKAogICAgICAgICAgICAgICAgWzQwLjcxNTk0MTI1NTY2OTMxLC03NC4wMDg3MjA1MzU2NDQ5NF0sCiAgICAgICAgICAgICAgICB7CiAgImJ1YmJsaW5nTW91c2VFdmVudHMiOiB0cnVlLAogICJjb2xvciI6ICJibHVlIiwKICAiZGFzaEFycmF5IjogbnVsbCwKICAiZGFzaE9mZnNldCI6IG51bGwsCiAgImZpbGwiOiB0cnVlLAogICJmaWxsQ29sb3IiOiAiYmx1ZSIsCiAgImZpbGxPcGFjaXR5IjogMC42LAogICJmaWxsUnVsZSI6ICJldmVub2RkIiwKICAibGluZUNhcCI6ICJyb3VuZCIsCiAgImxpbmVKb2luIjogInJvdW5kIiwKICAib3BhY2l0eSI6IDEuMCwKICAicmFkaXVzIjogNSwKICAic3Ryb2tlIjogdHJ1ZSwKICAid2VpZ2h0IjogMwp9CiAgICAgICAgICAgICAgICApLmFkZFRvKG1hcF9hY2MxYjU3YTk3ODg0MmUwODgxMGU3NDY2MTdiODA2ZCk7CiAgICAgICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBwb3B1cF9lYWI4NDQ0MmI0OTA0MDE0OGNlZDI1ZWE2MDgyOTgyZiA9IEwucG9wdXAoe21heFdpZHRoOiAnMzAwJ30pOwoKICAgICAgICAgICAgCiAgICAgICAgICAgICAgICB2YXIgaHRtbF8wMzk1NmVlZmFiYWE0MWQyYmNhNGJjMWU2MjgxMmZjMCA9ICQoJzxkaXYgaWQ9Imh0bWxfMDM5NTZlZWZhYmFhNDFkMmJjYTRiYzFlNjI4MTJmYzAiIHN0eWxlPSJ3aWR0aDogMTAwLjAlOyBoZWlnaHQ6IDEwMC4wJTsiPk5haWwgU2Fsb248L2Rpdj4nKVswXTsKICAgICAgICAgICAgICAgIHBvcHVwX2VhYjg0NDQyYjQ5MDQwMTQ4Y2VkMjVlYTYwODI5ODJmLnNldENvbnRlbnQoaHRtbF8wMzk1NmVlZmFiYWE0MWQyYmNhNGJjMWU2MjgxMmZjMCk7CiAgICAgICAgICAgIAoKICAgICAgICAgICAgY2lyY2xlX21hcmtlcl84NTU0MmEyYWJiZDk0ZGU1ODgxYzlkYjkyYTIxY2RjNC5iaW5kUG9wdXAocG9wdXBfZWFiODQ0NDJiNDkwNDAxNDhjZWQyNWVhNjA4Mjk4MmYpOwoKICAgICAgICAgICAgCiAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIGNpcmNsZV9tYXJrZXJfZmU1YzIyMGY0ZmVmNDg0YTg3NTlmOWJhNmIwZjY3YWMgPSBMLmNpcmNsZU1hcmtlcigKICAgICAgICAgICAgICAgIFs0MC43MTczOTQ0NTI5MTY1LC03NC4wMTAxMDMyNDYwNzEyNV0sCiAgICAgICAgICAgICAgICB7CiAgImJ1YmJsaW5nTW91c2VFdmVudHMiOiB0cnVlLAogICJjb2xvciI6ICJibHVlIiwKICAiZGFzaEFycmF5IjogbnVsbCwKICAiZGFzaE9mZnNldCI6IG51bGwsCiAgImZpbGwiOiB0cnVlLAogICJmaWxsQ29sb3IiOiAiYmx1ZSIsCiAgImZpbGxPcGFjaXR5IjogMC42LAogICJmaWxsUnVsZSI6ICJldmVub2RkIiwKICAibGluZUNhcCI6ICJyb3VuZCIsCiAgImxpbmVKb2luIjogInJvdW5kIiwKICAib3BhY2l0eSI6IDEuMCwKICAicmFkaXVzIjogNSwKICAic3Ryb2tlIjogdHJ1ZSwKICAid2VpZ2h0IjogMwp9CiAgICAgICAgICAgICAgICApLmFkZFRvKG1hcF9hY2MxYjU3YTk3ODg0MmUwODgxMGU3NDY2MTdiODA2ZCk7CiAgICAgICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBwb3B1cF85NTk4Mjg1YWQ4Zjk0OTBkOWZiNzZkNTA3YWFhMDIxNiA9IEwucG9wdXAoe21heFdpZHRoOiAnMzAwJ30pOwoKICAgICAgICAgICAgCiAgICAgICAgICAgICAgICB2YXIgaHRtbF85MTY3OGRhOTVkYzk0YzI2Yjg4ZTk3NDMyZjQ3NzEwYSA9ICQoJzxkaXYgaWQ9Imh0bWxfOTE2NzhkYTk1ZGM5NGMyNmI4OGU5NzQzMmY0NzcxMGEiIHN0eWxlPSJ3aWR0aDogMTAwLjAlOyBoZWlnaHQ6IDEwMC4wJTsiPkNvZmZlZSBTaG9wPC9kaXY+JylbMF07CiAgICAgICAgICAgICAgICBwb3B1cF85NTk4Mjg1YWQ4Zjk0OTBkOWZiNzZkNTA3YWFhMDIxNi5zZXRDb250ZW50KGh0bWxfOTE2NzhkYTk1ZGM5NGMyNmI4OGU5NzQzMmY0NzcxMGEpOwogICAgICAgICAgICAKCiAgICAgICAgICAgIGNpcmNsZV9tYXJrZXJfZmU1YzIyMGY0ZmVmNDg0YTg3NTlmOWJhNmIwZjY3YWMuYmluZFBvcHVwKHBvcHVwXzk1OTgyODVhZDhmOTQ5MGQ5ZmI3NmQ1MDdhYWEwMjE2KTsKCiAgICAgICAgICAgIAogICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBjaXJjbGVfbWFya2VyXzJhYTk3NTg1NDZhZDRhZjA4NzZlMGNiMTIyMDRjOWI2ID0gTC5jaXJjbGVNYXJrZXIoCiAgICAgICAgICAgICAgICBbNDAuNzE2NzUyODE2ODc2NjM1LC03NC4wMDg1ODM3NjI5NTIyMV0sCiAgICAgICAgICAgICAgICB7CiAgImJ1YmJsaW5nTW91c2VFdmVudHMiOiB0cnVlLAogICJjb2xvciI6ICJibHVlIiwKICAiZGFzaEFycmF5IjogbnVsbCwKICAiZGFzaE9mZnNldCI6IG51bGwsCiAgImZpbGwiOiB0cnVlLAogICJmaWxsQ29sb3IiOiAiYmx1ZSIsCiAgImZpbGxPcGFjaXR5IjogMC42LAogICJmaWxsUnVsZSI6ICJldmVub2RkIiwKICAibGluZUNhcCI6ICJyb3VuZCIsCiAgImxpbmVKb2luIjogInJvdW5kIiwKICAib3BhY2l0eSI6IDEuMCwKICAicmFkaXVzIjogNSwKICAic3Ryb2tlIjogdHJ1ZSwKICAid2VpZ2h0IjogMwp9CiAgICAgICAgICAgICAgICApLmFkZFRvKG1hcF9hY2MxYjU3YTk3ODg0MmUwODgxMGU3NDY2MTdiODA2ZCk7CiAgICAgICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBwb3B1cF9lMDI1ZDUzODkzMmY0ODgxOTc3MmVhZDUxZmRkNDA0OSA9IEwucG9wdXAoe21heFdpZHRoOiAnMzAwJ30pOwoKICAgICAgICAgICAgCiAgICAgICAgICAgICAgICB2YXIgaHRtbF82MzNkZGY1Y2UyZWU0MzczYTI4Y2RmNDhmMjkyNGEwZiA9ICQoJzxkaXYgaWQ9Imh0bWxfNjMzZGRmNWNlMmVlNDM3M2EyOGNkZjQ4ZjI5MjRhMGYiIHN0eWxlPSJ3aWR0aDogMTAwLjAlOyBoZWlnaHQ6IDEwMC4wJTsiPkFzaWFuIFJlc3RhdXJhbnQ8L2Rpdj4nKVswXTsKICAgICAgICAgICAgICAgIHBvcHVwX2UwMjVkNTM4OTMyZjQ4ODE5NzcyZWFkNTFmZGQ0MDQ5LnNldENvbnRlbnQoaHRtbF82MzNkZGY1Y2UyZWU0MzczYTI4Y2RmNDhmMjkyNGEwZik7CiAgICAgICAgICAgIAoKICAgICAgICAgICAgY2lyY2xlX21hcmtlcl8yYWE5NzU4NTQ2YWQ0YWYwODc2ZTBjYjEyMjA0YzliNi5iaW5kUG9wdXAocG9wdXBfZTAyNWQ1Mzg5MzJmNDg4MTk3NzJlYWQ1MWZkZDQwNDkpOwoKICAgICAgICAgICAgCiAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIGNpcmNsZV9tYXJrZXJfZWJjYWMwYzU3YjVkNDNjM2JmZjExYzMyYjVjZWE1MmMgPSBMLmNpcmNsZU1hcmtlcigKICAgICAgICAgICAgICAgIFs0MC43MTI3NTI1MTc3MTQ4NSwtNzQuMDA4NzMzNTU2MDE1NzFdLAogICAgICAgICAgICAgICAgewogICJidWJibGluZ01vdXNlRXZlbnRzIjogdHJ1ZSwKICAiY29sb3IiOiAiYmx1ZSIsCiAgImRhc2hBcnJheSI6IG51bGwsCiAgImRhc2hPZmZzZXQiOiBudWxsLAogICJmaWxsIjogdHJ1ZSwKICAiZmlsbENvbG9yIjogImJsdWUiLAogICJmaWxsT3BhY2l0eSI6IDAuNiwKICAiZmlsbFJ1bGUiOiAiZXZlbm9kZCIsCiAgImxpbmVDYXAiOiAicm91bmQiLAogICJsaW5lSm9pbiI6ICJyb3VuZCIsCiAgIm9wYWNpdHkiOiAxLjAsCiAgInJhZGl1cyI6IDUsCiAgInN0cm9rZSI6IHRydWUsCiAgIndlaWdodCI6IDMKfQogICAgICAgICAgICAgICAgKS5hZGRUbyhtYXBfYWNjMWI1N2E5Nzg4NDJlMDg4MTBlNzQ2NjE3YjgwNmQpOwogICAgICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgcG9wdXBfZjBmZDRjN2ZkMjliNDY4YWE5NWMzOGIxYmQ2NjQwZTAgPSBMLnBvcHVwKHttYXhXaWR0aDogJzMwMCd9KTsKCiAgICAgICAgICAgIAogICAgICAgICAgICAgICAgdmFyIGh0bWxfNjE1MGQxZDliZDY1NDAwMjllYjAzN2FhNTdkOGM3N2MgPSAkKCc8ZGl2IGlkPSJodG1sXzYxNTBkMWQ5YmQ2NTQwMDI5ZWIwMzdhYTU3ZDhjNzdjIiBzdHlsZT0id2lkdGg6IDEwMC4wJTsgaGVpZ2h0OiAxMDAuMCU7Ij5HeW0gLyBGaXRuZXNzIENlbnRlcjwvZGl2PicpWzBdOwogICAgICAgICAgICAgICAgcG9wdXBfZjBmZDRjN2ZkMjliNDY4YWE5NWMzOGIxYmQ2NjQwZTAuc2V0Q29udGVudChodG1sXzYxNTBkMWQ5YmQ2NTQwMDI5ZWIwMzdhYTU3ZDhjNzdjKTsKICAgICAgICAgICAgCgogICAgICAgICAgICBjaXJjbGVfbWFya2VyX2ViY2FjMGM1N2I1ZDQzYzNiZmYxMWMzMmI1Y2VhNTJjLmJpbmRQb3B1cChwb3B1cF9mMGZkNGM3ZmQyOWI0NjhhYTk1YzM4YjFiZDY2NDBlMCk7CgogICAgICAgICAgICAKICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgY2lyY2xlX21hcmtlcl80ZDg0YTA5YTA1Y2Y0YzlhYTkyZDQxMGU2Y2M1Yjg0YSA9IEwuY2lyY2xlTWFya2VyKAogICAgICAgICAgICAgICAgWzQwLjcxNTE0MzksLTc0LjAwOTE4MjZdLAogICAgICAgICAgICAgICAgewogICJidWJibGluZ01vdXNlRXZlbnRzIjogdHJ1ZSwKICAiY29sb3IiOiAiYmx1ZSIsCiAgImRhc2hBcnJheSI6IG51bGwsCiAgImRhc2hPZmZzZXQiOiBudWxsLAogICJmaWxsIjogdHJ1ZSwKICAiZmlsbENvbG9yIjogImJsdWUiLAogICJmaWxsT3BhY2l0eSI6IDAuNiwKICAiZmlsbFJ1bGUiOiAiZXZlbm9kZCIsCiAgImxpbmVDYXAiOiAicm91bmQiLAogICJsaW5lSm9pbiI6ICJyb3VuZCIsCiAgIm9wYWNpdHkiOiAxLjAsCiAgInJhZGl1cyI6IDUsCiAgInN0cm9rZSI6IHRydWUsCiAgIndlaWdodCI6IDMKfQogICAgICAgICAgICAgICAgKS5hZGRUbyhtYXBfYWNjMWI1N2E5Nzg4NDJlMDg4MTBlNzQ2NjE3YjgwNmQpOwogICAgICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgcG9wdXBfMDVjNGY1NjczYTk1NDhkODk3MDQ4ZWE4ZTAxMGYwMDYgPSBMLnBvcHVwKHttYXhXaWR0aDogJzMwMCd9KTsKCiAgICAgICAgICAgIAogICAgICAgICAgICAgICAgdmFyIGh0bWxfNzI5YzI5NDY5Zjk1NDNlMDhhODNiZDRkNTFlOWVhMTIgPSAkKCc8ZGl2IGlkPSJodG1sXzcyOWMyOTQ2OWY5NTQzZTA4YTgzYmQ0ZDUxZTllYTEyIiBzdHlsZT0id2lkdGg6IDEwMC4wJTsgaGVpZ2h0OiAxMDAuMCU7Ij5Ib3RlbDwvZGl2PicpWzBdOwogICAgICAgICAgICAgICAgcG9wdXBfMDVjNGY1NjczYTk1NDhkODk3MDQ4ZWE4ZTAxMGYwMDYuc2V0Q29udGVudChodG1sXzcyOWMyOTQ2OWY5NTQzZTA4YTgzYmQ0ZDUxZTllYTEyKTsKICAgICAgICAgICAgCgogICAgICAgICAgICBjaXJjbGVfbWFya2VyXzRkODRhMDlhMDVjZjRjOWFhOTJkNDEwZTZjYzViODRhLmJpbmRQb3B1cChwb3B1cF8wNWM0ZjU2NzNhOTU0OGQ4OTcwNDhlYThlMDEwZjAwNik7CgogICAgICAgICAgICAKICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgY2lyY2xlX21hcmtlcl9lZDQ2NWQyYTg0YTk0MWQ0YTdiZTlhMmM1YThiNTA2OCA9IEwuY2lyY2xlTWFya2VyKAogICAgICAgICAgICAgICAgWzQwLjcxNDk0MzkyNjQzNzA2LC03NC4wMDY0ODQ2MTk1MjcwNl0sCiAgICAgICAgICAgICAgICB7CiAgImJ1YmJsaW5nTW91c2VFdmVudHMiOiB0cnVlLAogICJjb2xvciI6ICJibHVlIiwKICAiZGFzaEFycmF5IjogbnVsbCwKICAiZGFzaE9mZnNldCI6IG51bGwsCiAgImZpbGwiOiB0cnVlLAogICJmaWxsQ29sb3IiOiAiYmx1ZSIsCiAgImZpbGxPcGFjaXR5IjogMC42LAogICJmaWxsUnVsZSI6ICJldmVub2RkIiwKICAibGluZUNhcCI6ICJyb3VuZCIsCiAgImxpbmVKb2luIjogInJvdW5kIiwKICAib3BhY2l0eSI6IDEuMCwKICAicmFkaXVzIjogNSwKICAic3Ryb2tlIjogdHJ1ZSwKICAid2VpZ2h0IjogMwp9CiAgICAgICAgICAgICAgICApLmFkZFRvKG1hcF9hY2MxYjU3YTk3ODg0MmUwODgxMGU3NDY2MTdiODA2ZCk7CiAgICAgICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBwb3B1cF9kNWM4NzBkZWJjMjY0MWQyODBhYjRlNmZlNDRjMjllNiA9IEwucG9wdXAoe21heFdpZHRoOiAnMzAwJ30pOwoKICAgICAgICAgICAgCiAgICAgICAgICAgICAgICB2YXIgaHRtbF9mNDM2Y2MxN2IwYjU0MDM1YjUyMTYxZGI1YWMwNzhlMCA9ICQoJzxkaXYgaWQ9Imh0bWxfZjQzNmNjMTdiMGI1NDAzNWI1MjE2MWRiNWFjMDc4ZTAiIHN0eWxlPSJ3aWR0aDogMTAwLjAlOyBoZWlnaHQ6IDEwMC4wJTsiPkJhcjwvZGl2PicpWzBdOwogICAgICAgICAgICAgICAgcG9wdXBfZDVjODcwZGViYzI2NDFkMjgwYWI0ZTZmZTQ0YzI5ZTYuc2V0Q29udGVudChodG1sX2Y0MzZjYzE3YjBiNTQwMzViNTIxNjFkYjVhYzA3OGUwKTsKICAgICAgICAgICAgCgogICAgICAgICAgICBjaXJjbGVfbWFya2VyX2VkNDY1ZDJhODRhOTQxZDRhN2JlOWEyYzVhOGI1MDY4LmJpbmRQb3B1cChwb3B1cF9kNWM4NzBkZWJjMjY0MWQyODBhYjRlNmZlNDRjMjllNik7CgogICAgICAgICAgICAKICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgY2lyY2xlX21hcmtlcl80ZmM3MjYyNTI0OGQ0MGY5OTU5OWQwMjZhNDRhYzAwZSA9IEwuY2lyY2xlTWFya2VyKAogICAgICAgICAgICAgICAgWzQwLjcxNjQwNDE5NTI2Mzc2LC03NC4wMDg1Njk3OTU1MDEyM10sCiAgICAgICAgICAgICAgICB7CiAgImJ1YmJsaW5nTW91c2VFdmVudHMiOiB0cnVlLAogICJjb2xvciI6ICJibHVlIiwKICAiZGFzaEFycmF5IjogbnVsbCwKICAiZGFzaE9mZnNldCI6IG51bGwsCiAgImZpbGwiOiB0cnVlLAogICJmaWxsQ29sb3IiOiAiYmx1ZSIsCiAgImZpbGxPcGFjaXR5IjogMC42LAogICJmaWxsUnVsZSI6ICJldmVub2RkIiwKICAibGluZUNhcCI6ICJyb3VuZCIsCiAgImxpbmVKb2luIjogInJvdW5kIiwKICAib3BhY2l0eSI6IDEuMCwKICAicmFkaXVzIjogNSwKICAic3Ryb2tlIjogdHJ1ZSwKICAid2VpZ2h0IjogMwp9CiAgICAgICAgICAgICAgICApLmFkZFRvKG1hcF9hY2MxYjU3YTk3ODg0MmUwODgxMGU3NDY2MTdiODA2ZCk7CiAgICAgICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBwb3B1cF9jYzg1ZTgwMWUxYTU0M2RhYTQzMWY1YzQ1NzYxNzJlZCA9IEwucG9wdXAoe21heFdpZHRoOiAnMzAwJ30pOwoKICAgICAgICAgICAgCiAgICAgICAgICAgICAgICB2YXIgaHRtbF9kNzk1MWI2MzE0OGU0YzhiYTBlZGM1YjkzNjQyYTYxNyA9ICQoJzxkaXYgaWQ9Imh0bWxfZDc5NTFiNjMxNDhlNGM4YmEwZWRjNWI5MzY0MmE2MTciIHN0eWxlPSJ3aWR0aDogMTAwLjAlOyBoZWlnaHQ6IDEwMC4wJTsiPlNwYTwvZGl2PicpWzBdOwogICAgICAgICAgICAgICAgcG9wdXBfY2M4NWU4MDFlMWE1NDNkYWE0MzFmNWM0NTc2MTcyZWQuc2V0Q29udGVudChodG1sX2Q3OTUxYjYzMTQ4ZTRjOGJhMGVkYzViOTM2NDJhNjE3KTsKICAgICAgICAgICAgCgogICAgICAgICAgICBjaXJjbGVfbWFya2VyXzRmYzcyNjI1MjQ4ZDQwZjk5NTk5ZDAyNmE0NGFjMDBlLmJpbmRQb3B1cChwb3B1cF9jYzg1ZTgwMWUxYTU0M2RhYTQzMWY1YzQ1NzYxNzJlZCk7CgogICAgICAgICAgICAKICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgY2lyY2xlX21hcmtlcl8xZjk3MTE4ZDBkMGE0NmFmYjcxM2M1OWNlM2ViZDM0OSA9IEwuY2lyY2xlTWFya2VyKAogICAgICAgICAgICAgICAgWzQwLjcxMjYxMjQ2NTAxOTgzLC03NC4wMDkzODAzMjA4NzYyOF0sCiAgICAgICAgICAgICAgICB7CiAgImJ1YmJsaW5nTW91c2VFdmVudHMiOiB0cnVlLAogICJjb2xvciI6ICJibHVlIiwKICAiZGFzaEFycmF5IjogbnVsbCwKICAiZGFzaE9mZnNldCI6IG51bGwsCiAgImZpbGwiOiB0cnVlLAogICJmaWxsQ29sb3IiOiAiYmx1ZSIsCiAgImZpbGxPcGFjaXR5IjogMC42LAogICJmaWxsUnVsZSI6ICJldmVub2RkIiwKICAibGluZUNhcCI6ICJyb3VuZCIsCiAgImxpbmVKb2luIjogInJvdW5kIiwKICAib3BhY2l0eSI6IDEuMCwKICAicmFkaXVzIjogNSwKICAic3Ryb2tlIjogdHJ1ZSwKICAid2VpZ2h0IjogMwp9CiAgICAgICAgICAgICAgICApLmFkZFRvKG1hcF9hY2MxYjU3YTk3ODg0MmUwODgxMGU3NDY2MTdiODA2ZCk7CiAgICAgICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBwb3B1cF8yNDI1YmQxZTk4YmI0YmNkYmViMzFiYTk5ZDI0M2E3YyA9IEwucG9wdXAoe21heFdpZHRoOiAnMzAwJ30pOwoKICAgICAgICAgICAgCiAgICAgICAgICAgICAgICB2YXIgaHRtbF9mZGIyNDI0YWQ5N2U0ODUxYjQ2YzY1MWQzNGIxMjY0YSA9ICQoJzxkaXYgaWQ9Imh0bWxfZmRiMjQyNGFkOTdlNDg1MWI0NmM2NTFkMzRiMTI2NGEiIHN0eWxlPSJ3aWR0aDogMTAwLjAlOyBoZWlnaHQ6IDEwMC4wJTsiPkhvdGVsPC9kaXY+JylbMF07CiAgICAgICAgICAgICAgICBwb3B1cF8yNDI1YmQxZTk4YmI0YmNkYmViMzFiYTk5ZDI0M2E3Yy5zZXRDb250ZW50KGh0bWxfZmRiMjQyNGFkOTdlNDg1MWI0NmM2NTFkMzRiMTI2NGEpOwogICAgICAgICAgICAKCiAgICAgICAgICAgIGNpcmNsZV9tYXJrZXJfMWY5NzExOGQwZDBhNDZhZmI3MTNjNTljZTNlYmQzNDkuYmluZFBvcHVwKHBvcHVwXzI0MjViZDFlOThiYjRiY2RiZWIzMWJhOTlkMjQzYTdjKTsKCiAgICAgICAgICAgIAogICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBjaXJjbGVfbWFya2VyXzUwZjhhNDc1YmY2ODQwYWNhYWIwNzE3ZmVjOWJmOTQ2ID0gTC5jaXJjbGVNYXJrZXIoCiAgICAgICAgICAgICAgICBbNDAuNzE0NTM3NDMxMTcwNDg0LC03NC4wMDU5OTg1MjYxMTU5Ml0sCiAgICAgICAgICAgICAgICB7CiAgImJ1YmJsaW5nTW91c2VFdmVudHMiOiB0cnVlLAogICJjb2xvciI6ICJibHVlIiwKICAiZGFzaEFycmF5IjogbnVsbCwKICAiZGFzaE9mZnNldCI6IG51bGwsCiAgImZpbGwiOiB0cnVlLAogICJmaWxsQ29sb3IiOiAiYmx1ZSIsCiAgImZpbGxPcGFjaXR5IjogMC42LAogICJmaWxsUnVsZSI6ICJldmVub2RkIiwKICAibGluZUNhcCI6ICJyb3VuZCIsCiAgImxpbmVKb2luIjogInJvdW5kIiwKICAib3BhY2l0eSI6IDEuMCwKICAicmFkaXVzIjogNSwKICAic3Ryb2tlIjogdHJ1ZSwKICAid2VpZ2h0IjogMwp9CiAgICAgICAgICAgICAgICApLmFkZFRvKG1hcF9hY2MxYjU3YTk3ODg0MmUwODgxMGU3NDY2MTdiODA2ZCk7CiAgICAgICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBwb3B1cF8zYmZiOGYyYThkMzE0ZTVlOTNiZDAxNWNiNDc1YjJiOCA9IEwucG9wdXAoe21heFdpZHRoOiAnMzAwJ30pOwoKICAgICAgICAgICAgCiAgICAgICAgICAgICAgICB2YXIgaHRtbF9jN2Y2ZDQ3M2JjMTY0NjNkODhkNDA3NmFiOTE2OGQ5NyA9ICQoJzxkaXYgaWQ9Imh0bWxfYzdmNmQ0NzNiYzE2NDYzZDg4ZDQwNzZhYjkxNjhkOTciIHN0eWxlPSJ3aWR0aDogMTAwLjAlOyBoZWlnaHQ6IDEwMC4wJTsiPkd5bTwvZGl2PicpWzBdOwogICAgICAgICAgICAgICAgcG9wdXBfM2JmYjhmMmE4ZDMxNGU1ZTkzYmQwMTVjYjQ3NWIyYjguc2V0Q29udGVudChodG1sX2M3ZjZkNDczYmMxNjQ2M2Q4OGQ0MDc2YWI5MTY4ZDk3KTsKICAgICAgICAgICAgCgogICAgICAgICAgICBjaXJjbGVfbWFya2VyXzUwZjhhNDc1YmY2ODQwYWNhYWIwNzE3ZmVjOWJmOTQ2LmJpbmRQb3B1cChwb3B1cF8zYmZiOGYyYThkMzE0ZTVlOTNiZDAxNWNiNDc1YjJiOCk7CgogICAgICAgICAgICAKICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgY2lyY2xlX21hcmtlcl8xNjUzNDc4ZWVmMDQ0ODJjOWVlNTdmYzhjNWM5NWIzMyA9IEwuY2lyY2xlTWFya2VyKAogICAgICAgICAgICAgICAgWzQwLjcxNTYwNTMzMzE0NzE0NSwtNzQuMDExNzg2MzMyMDg3NzFdLAogICAgICAgICAgICAgICAgewogICJidWJibGluZ01vdXNlRXZlbnRzIjogdHJ1ZSwKICAiY29sb3IiOiAiYmx1ZSIsCiAgImRhc2hBcnJheSI6IG51bGwsCiAgImRhc2hPZmZzZXQiOiBudWxsLAogICJmaWxsIjogdHJ1ZSwKICAiZmlsbENvbG9yIjogImJsdWUiLAogICJmaWxsT3BhY2l0eSI6IDAuNiwKICAiZmlsbFJ1bGUiOiAiZXZlbm9kZCIsCiAgImxpbmVDYXAiOiAicm91bmQiLAogICJsaW5lSm9pbiI6ICJyb3VuZCIsCiAgIm9wYWNpdHkiOiAxLjAsCiAgInJhZGl1cyI6IDUsCiAgInN0cm9rZSI6IHRydWUsCiAgIndlaWdodCI6IDMKfQogICAgICAgICAgICAgICAgKS5hZGRUbyhtYXBfYWNjMWI1N2E5Nzg4NDJlMDg4MTBlNzQ2NjE3YjgwNmQpOwogICAgICAgICAgICAKICAgIAogICAgICAgICAgICB2YXIgcG9wdXBfMTkzOWQ2YjgzOTU0NGRhYThkOWUwODllOTIwNTM3NjcgPSBMLnBvcHVwKHttYXhXaWR0aDogJzMwMCd9KTsKCiAgICAgICAgICAgIAogICAgICAgICAgICAgICAgdmFyIGh0bWxfNTFmYjRkZDg5YThiNGY4Y2E3M2E4MDYwM2UxNWMwYWQgPSAkKCc8ZGl2IGlkPSJodG1sXzUxZmI0ZGQ4OWE4YjRmOGNhNzNhODA2MDNlMTVjMGFkIiBzdHlsZT0id2lkdGg6IDEwMC4wJTsgaGVpZ2h0OiAxMDAuMCU7Ij5Cb29rc3RvcmU8L2Rpdj4nKVswXTsKICAgICAgICAgICAgICAgIHBvcHVwXzE5MzlkNmI4Mzk1NDRkYWE4ZDllMDg5ZTkyMDUzNzY3LnNldENvbnRlbnQoaHRtbF81MWZiNGRkODlhOGI0ZjhjYTczYTgwNjAzZTE1YzBhZCk7CiAgICAgICAgICAgIAoKICAgICAgICAgICAgY2lyY2xlX21hcmtlcl8xNjUzNDc4ZWVmMDQ0ODJjOWVlNTdmYzhjNWM5NWIzMy5iaW5kUG9wdXAocG9wdXBfMTkzOWQ2YjgzOTU0NGRhYThkOWUwODllOTIwNTM3NjcpOwoKICAgICAgICAgICAgCiAgICAgICAgCjwvc2NyaXB0Pg== onload=\"this.contentDocument.open();this.contentDocument.write(atob(this.getAttribute('data-html')));this.contentDocument.close();\" allowfullscreen webkitallowfullscreen mozallowfullscreen></iframe></div></div>" | |
], | |
"text/plain": [ | |
"<folium.folium.Map at 0x7f9e1ed96fd0>" | |
] | |
}, | |
"execution_count": 28, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"venues_map = folium.Map(location=[latitude, longitude], zoom_start=15) # generate map centred around Ecco\n", | |
"\n", | |
"\n", | |
"# add Ecco as a red circle mark\n", | |
"folium.features.CircleMarker(\n", | |
" [latitude, longitude],\n", | |
" radius=10,\n", | |
" popup='Ecco',\n", | |
" fill=True,\n", | |
" color='red',\n", | |
" fill_color='red',\n", | |
" fill_opacity=0.6\n", | |
" ).add_to(venues_map)\n", | |
"\n", | |
"\n", | |
"# add popular spots to the map as blue circle markers\n", | |
"for lat, lng, label in zip(dataframe_filtered.lat, dataframe_filtered.lng, dataframe_filtered.categories):\n", | |
" folium.features.CircleMarker(\n", | |
" [lat, lng],\n", | |
" radius=5,\n", | |
" popup=label,\n", | |
" fill=True,\n", | |
" color='blue',\n", | |
" fill_color='blue',\n", | |
" fill_opacity=0.6\n", | |
" ).add_to(venues_map)\n", | |
"\n", | |
"# display map\n", | |
"venues_map" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
" " | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"<a id=\"item5\"></a>" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"## 5. Explore Trending Venues\n", | |
"> `https://api.foursquare.com/v2/venues/`**trending**`?client_id=`**CLIENT_ID**`&client_secret=`**CLIENT_SECRET**`&ll=`**LATITUDE**`,`**LONGITUDE**`&v=`**VERSION**" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"#### Now, instead of simply exploring the area around Ecco, you are interested in knowing the venues that are trending at the time you are done with your lunch, meaning the places with the highest foot traffic. So let's do that and get the trending venues around Ecco." | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 29, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"data": { | |
"text/plain": [ | |
"{'meta': {'code': 200, 'requestId': '5ed6455e9fcb92001b6d8c10'},\n", | |
" 'response': {'venues': []}}" | |
] | |
}, | |
"execution_count": 29, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"# define URL\n", | |
"url = 'https://api.foursquare.com/v2/venues/trending?client_id={}&client_secret={}&ll={},{}&v={}'.format(CLIENT_ID, CLIENT_SECRET, latitude, longitude, VERSION)\n", | |
"\n", | |
"# send GET request and get trending venues\n", | |
"results = requests.get(url).json()\n", | |
"results" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"### Check if any venues are trending at this time" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 30, | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [], | |
"source": [ | |
"if len(results['response']['venues']) == 0:\n", | |
" trending_venues_df = 'No trending venues are available at the moment!'\n", | |
" \n", | |
"else:\n", | |
" trending_venues = results['response']['venues']\n", | |
" trending_venues_df = json_normalize(trending_venues)\n", | |
"\n", | |
" # filter columns\n", | |
" columns_filtered = ['name', 'categories'] + ['location.distance', 'location.city', 'location.postalCode', 'location.state', 'location.country', 'location.lat', 'location.lng']\n", | |
" trending_venues_df = trending_venues_df.loc[:, columns_filtered]\n", | |
"\n", | |
" # filter the category for each row\n", | |
" trending_venues_df['categories'] = trending_venues_df.apply(get_category_type, axis=1)" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 31, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"data": { | |
"text/plain": [ | |
"'No trending venues are available at the moment!'" | |
] | |
}, | |
"execution_count": 31, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"# display trending venues\n", | |
"trending_venues_df" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"Now, depending on when you run the above code, you might get different venues since the venues with the highest foot traffic are fetched live. " | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"### Visualize trending venues" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 32, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [], | |
"source": [ | |
"if len(results['response']['venues']) == 0:\n", | |
" venues_map = 'Cannot generate visual as no trending venues are available at the moment!'\n", | |
"\n", | |
"else:\n", | |
" venues_map = folium.Map(location=[latitude, longitude], zoom_start=15) # generate map centred around Ecco\n", | |
"\n", | |
"\n", | |
" # add Ecco as a red circle mark\n", | |
" folium.features.CircleMarker(\n", | |
" [latitude, longitude],\n", | |
" radius=10,\n", | |
" popup='Ecco',\n", | |
" fill=True,\n", | |
" color='red',\n", | |
" fill_color='red',\n", | |
" fill_opacity=0.6\n", | |
" ).add_to(venues_map)\n", | |
"\n", | |
"\n", | |
" # add the trending venues as blue circle markers\n", | |
" for lat, lng, label in zip(trending_venues_df['location.lat'], trending_venues_df['location.lng'], trending_venues_df['name']):\n", | |
" folium.features.CircleMarker(\n", | |
" [lat, lng],\n", | |
" radius=5,\n", | |
" poup=label,\n", | |
" fill=True,\n", | |
" color='blue',\n", | |
" fill_color='blue',\n", | |
" fill_opacity=0.6\n", | |
" ).add_to(venues_map)" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"execution_count": 33, | |
"metadata": { | |
"button": false, | |
"collapsed": false, | |
"deletable": true, | |
"jupyter": { | |
"outputs_hidden": false | |
}, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"outputs": [ | |
{ | |
"data": { | |
"text/plain": [ | |
"'Cannot generate visual as no trending venues are available at the moment!'" | |
] | |
}, | |
"execution_count": 33, | |
"metadata": {}, | |
"output_type": "execute_result" | |
} | |
], | |
"source": [ | |
"# display map\n", | |
"venues_map" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"<a id=\"item6\"></a>" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
" " | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"### Thank you for completing this lab!\n", | |
"\n", | |
"This notebook was created by [Alex Aklson](https://www.linkedin.com/in/aklson/). I hope you found this lab interesting and educational. Feel free to contact me if you have any questions!" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"This notebook is part of a course on **Coursera** called *Applied Data Science Capstone*. If you accessed this notebook outside the course, you can take this course online by clicking [here](http://cocl.us/DP0701EN_Coursera_Week2_LAB1)." | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"button": false, | |
"deletable": true, | |
"new_sheet": false, | |
"run_control": { | |
"read_only": false | |
} | |
}, | |
"source": [ | |
"<hr>\n", | |
"Copyright © 2018 [Cognitive Class](https://cognitiveclass.ai/?utm_source=bducopyrightlink&utm_medium=dswb&utm_campaign=bdu). This notebook and its source code are released under the terms of the [MIT License](https://bigdatauniversity.com/mit-license/)." | |
] | |
} | |
], | |
"metadata": { | |
"kernelspec": { | |
"display_name": "Python", | |
"language": "python", | |
"name": "conda-env-python-py" | |
}, | |
"language_info": { | |
"codemirror_mode": { | |
"name": "ipython", | |
"version": 3 | |
}, | |
"file_extension": ".py", | |
"mimetype": "text/x-python", | |
"name": "python", | |
"nbconvert_exporter": "python", | |
"pygments_lexer": "ipython3", | |
"version": "3.6.10" | |
}, | |
"widgets": { | |
"state": {}, | |
"version": "1.1.2" | |
} | |
}, | |
"nbformat": 4, | |
"nbformat_minor": 4 | |
} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment