Created
September 27, 2020 10:11
-
-
Save ProtichiChatterjee/ce8ef6fd2f69105198a6793a9c3bbb40 to your computer and use it in GitHub Desktop.
This is the peer-review project done by me for the course assessment of IBM Data science on Coursers
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "cells": [ | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "<a><img src=\"https://ibm.box.com/shared/static/ugcqz6ohbvff804xp84y4kqnvvk3bq1g.png\" width=\"200\" align=\"center\"></a>" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "<h1>Analyzing US Economic Data and Building a Dashboard </h1>\n<h2>Description</h2>\n" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "Extracting essential data from a dataset and displaying it is a necessary part of data science; therefore individuals can make correct decisions based on the data. In this assignment, you will extract some essential economic indicators from some data, you will then display these economic indicators in a Dashboard. You can then share the dashboard via an URL.\n<p>\n<a href=\"https://en.wikipedia.org/wiki/Gross_domestic_product\"> Gross domestic product (GDP)</a> is a measure of the market value of all the final goods and services produced in a period. GDP is an indicator of how well the economy is doing. A drop in GDP indicates the economy is producing less; similarly an increase in GDP suggests the economy is performing better. In this lab, you will examine how changes in GDP impact the unemployment rate. You will take screen shots of every step, you will share the notebook and the URL pointing to the dashboard.</p>" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "<h2>Table of Contents</h2>\n<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n <ul>\n <li><a href=\"#Section_1\"> Define a Function that Makes a Dashboard </a></li>\n <li><a href=\"#Section_2\">Question 1: Create a dataframe that contains the GDP data and display it</a> </li>\n <li><a href=\"#Section_3\">Question 2: Create a dataframe that contains the unemployment data and display it</a></li>\n <li><a href=\"#Section_4\">Question 3: Display a dataframe where unemployment was greater than 8.5%</a></li>\n <li><a href=\"#Section_5\">Question 4: Use the function make_dashboard to make a dashboard</a></li>\n <li><a href=\"#Section_6\"><b>(Optional not marked)</b> Save the dashboard on IBM cloud and display it</a></li>\n </ul>\n<p>\n Estimated Time Needed: <strong>180 min</strong></p>\n</div>\n\n<hr>" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "<h2 id=\"Section_1\"> Define Function that Makes a Dashboard </h2>" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "We will import the following libraries." | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "import pandas as pd\nfrom bokeh.plotting import figure, output_file, show,output_notebook\noutput_notebook()" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "In this section, we define the function <code>make_dashboard</code>. \nYou don't have to know how the function works, you should only care about the inputs. The function will produce a dashboard as well as an html file. You can then use this html file to share your dashboard. If you do not know what an html file is don't worry everything you need to know will be provided in the lab. " | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "def make_dashboard(x, gdp_change, unemployment, title, file_name):\n output_file(file_name)\n p = figure(title=title, x_axis_label='year', y_axis_label='%')\n p.line(x.squeeze(), gdp_change.squeeze(), color=\"firebrick\", line_width=4, legend=\"% GDP change\")\n p.line(x.squeeze(), unemployment.squeeze(), line_width=4, legend=\"% unemployed\")\n show(p)" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "The dictionary <code>links</code> contain the CSV files with all the data. The value for the key <code>GDP</code> is the file that contains the GDP data. The value for the key <code>unemployment</code> contains the unemployment data." | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "links={'GDP':'https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/PY0101EN/projects/coursera_project/clean_gdp.csv',\\\n 'unemployment':'https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/PY0101EN/projects/coursera_project/clean_unemployment.csv'}" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "<h3 id=\"Section_2\"> Question 1: Create a dataframe that contains the GDP data and display the first five rows of the dataframe.</h3>" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "Use the dictionary <code>links</code> and the function <code>pd.read_csv</code> to create a Pandas dataframes that contains the GDP data." | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "<b>Hint: <code>links[\"GDP\"]</code> contains the path or name of the file.</b>" | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": 1, | |
| "metadata": {}, | |
| "outputs": [ | |
| { | |
| "ename": "NameError", | |
| "evalue": "name 'pd' is not defined", | |
| "output_type": "error", | |
| "traceback": [ | |
| "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", | |
| "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", | |
| "\u001b[0;32m<ipython-input-1-dbfa6d55f89a>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0;31m# Type your code here\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mdf\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_csv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlinks\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", | |
| "\u001b[0;31mNameError\u001b[0m: name 'pd' is not defined" | |
| ] | |
| } | |
| ], | |
| "source": "# Type your code here\ndf=pd.read_csv(links)" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "Use the method <code>head()</code> to display the first five rows of the GDP data, then take a screen-shot." | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "# Type your code here\ndf.head()\nprint df" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "<h3 id=\"Section_2\"> Question 2: Create a dataframe that contains the unemployment data. Display the first five rows of the dataframe. </h3>" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "Use the dictionary <code>links</code> and the function <code>pd.read_csv</code> to create a Pandas dataframes that contains the unemployment data." | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "# Type your code here\nunemployement_frame=pd.DataFrame(unemployement)" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "Use the method <code>head()</code> to display the first five rows of the GDP data, then take a screen-shot." | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "# Type your code here" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "<h3 id=\"Section_3\">Question 3: Display a dataframe where unemployment was greater than 8.5%. Take a screen-shot.</h3>" | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "# Type your code here" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "<h3 id=\"Section_4\">Question 4: Use the function make_dashboard to make a dashboard</h3>" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "In this section, you will call the function <code>make_dashboard</code> , to produce a dashboard. We will use the convention of giving each variable the same name as the function parameter." | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "Create a new dataframe with the column <code>'date'</code> called <code>x</code> from the dataframe that contains the GDP data." | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "x = # Create your dataframe with column date" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "Create a new dataframe with the column <code>'change-current' </code> called <code>gdp_change</code> from the dataframe that contains the GDP data." | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "gdp_change = # Create your dataframe with column change-current" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "Create a new dataframe with the column <code>'unemployment' </code> called <code>unemployment</code> from the dataframe that contains the unemployment data." | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "unemployment = # Create your dataframe with column unemployment" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "Give your dashboard a string title, and assign it to the variable <code>title</code>" | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "title = # Give your dashboard a string title" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "Finally, the function <code>make_dashboard</code> will output an <code>.html</code> in your direictory, just like a <code>csv</code> file. The name of the file is <code>\"index.html\"</code> and it will be stored in the varable <code>file_name</code>." | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "file_name = \"index.html\"" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "Call the function <code>make_dashboard</code> , to produce a dashboard. Assign the parameter values accordingly take a the <b>, take a screen shot of the dashboard and submit it</b>." | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "# Fill up the parameters in the following function:\n# make_dashboard(x=, gdp_change=, unemployment=, title=, file_name=)" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "<h3 id=\"Section_5\"> <b>(Optional not marked)</b>Save the dashboard on IBM cloud and display it </h3>" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "From the tutorial <i>PROVISIONING AN OBJECT STORAGE INSTANCE ON IBM CLOUD</i> copy the JSON object containing the credentials you created. You\u2019ll want to store everything you see in a credentials variable like the one below (obviously, replace the placeholder values with your own). Take special note of your <code>access_key_id</code> and <code>secret_access_key</code>. <b>Do not delete <code># @hidden_cell </code> as this will not allow people to see your credentials when you share your notebook. </b>" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "<code>\ncredentials = {<br>\n \"apikey\": \"your-api-key\",<br>\n \"cos_hmac_keys\": {<br>\n \"access_key_id\": \"your-access-key-here\", <br>\n \"secret_access_key\": \"your-secret-access-key-here\"<br>\n },<br>\n</code>\n<code>\n \"endpoints\": \"your-endpoints\",<br>\n \"iam_apikey_description\": \"your-iam_apikey_description\",<br>\n \"iam_apikey_name\": \"your-iam_apikey_name\",<br>\n \"iam_role_crn\": \"your-iam_apikey_name\",<br>\n \"iam_serviceid_crn\": \"your-iam_serviceid_crn\",<br>\n \"resource_instance_id\": \"your-resource_instance_id\"<br>\n}\n</code>" | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "# The code was removed by Watson Studio for sharing." | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "You will need the endpoint make sure the setting are the same as <i> PROVISIONING AN OBJECT STORAGE INSTANCE ON IBM CLOUD </i> assign the name of your bucket to the variable <code>bucket_name </code> " | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "endpoint = 'https://s3-api.us-geo.objectstorage.softlayer.net'" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "From the tutorial <i> PROVISIONING AN OBJECT STORAGE INSTANCE ON IBM CLOUD </i> assign the name of your bucket to the variable <code>bucket_name </code> " | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "bucket_name = # Type your bucket name on IBM Cloud" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "We can access IBM Cloud Object Storage with Python useing the <code>boto3</code> library, which we\u2019ll import below:" | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "import boto3" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "We can interact with IBM Cloud Object Storage through a <code>boto3</code> resource object." | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "resource = boto3.resource(\n 's3',\n aws_access_key_id = credentials[\"cos_hmac_keys\"]['access_key_id'],\n aws_secret_access_key = credentials[\"cos_hmac_keys\"][\"secret_access_key\"],\n endpoint_url = endpoint,\n)" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "We are going to use <code>open</code> to create a file object. To get the path of the file, you are going to concatenate the name of the file stored in the variable <code>file_name</code>. The directory stored in the variable directory using the <code>+</code> operator and assign it to the variable \n<code>html_path</code>. We will use the function <code>getcwd()</code> to find current the working directory." | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "import os\n\ndirectory = os.getcwd()\nhtml_path = directory + \"/\" + file_name" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "Now you must read the html file, use the function <code>f = open(html_path, mode)</code> to create a file object and assign it to the variable <code>f</code>. The parameter <code>file</code> should be the variable <code>html_path</code>, the mode should be <code>\"r\"</code> for read. " | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "# Type your code here" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "To load your dataset into the bucket we will use the method <code>put_object</code>, you must set the parameter name to the name of the bucket, the parameter <code>Key</code> should be the name of the HTML file and the value for the parameter Body should be set to <code>f.read()</code>." | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "# Fill up the parameters in the following function:\n# resource.Bucket(name=).put_object(Key=, Body=)" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "In the dictionary <code>Params</code> provide the bucket name as the value for the key <i>'Bucket'</i>. Also for the value of the key <i>'Key'</i> add the name of the <code>html</code> file, both values should be strings." | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "# Fill in the value for each key\n# Params = {'Bucket': ,'Key': }" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "The following lines of code will generate a URL to share your dashboard. The URL only last seven days, but don't worry you will get full marks if the URL is visible in your notebook. " | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": {}, | |
| "outputs": [], | |
| "source": "import sys\ntime = 7*24*60**2\nclient = boto3.client(\n 's3',\n aws_access_key_id = credentials[\"cos_hmac_keys\"]['access_key_id'],\n aws_secret_access_key = credentials[\"cos_hmac_keys\"][\"secret_access_key\"],\n endpoint_url=endpoint,\n\n)\nurl = client.generate_presigned_url('get_object',Params=Params,ExpiresIn=time)\nprint(url)" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "<h2 id=\"Section_5\"> How to submit </h2>" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "<p>Once you complete your notebook you will have to share it to be marked. Select the icon on the top right a marked in red in the image below, a dialogue box should open, select the option all content excluding sensitive code cells.</p>\n\n<p><img height=\"440\" width=\"700\" src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/PY0101EN/projects/EdX/ReadMe%20files/share_noteook1.png\" alt=\"share notebook\" /></p>\n<p></p>\n\n<p>You can then share the notebook via a URL by scrolling down as shown in the following image:</p>\n<p style=\"text-align: center;\"> <img height=\"308\" width=\"350\" src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/PY0101EN/projects/EdX/ReadMe%20files/link2.png\" alt=\"share notebook\" /> </p>" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "<hr>\n<p>Copyright © 2019 IBM Developer Skills Network. This notebook and its source code are released under the terms of the <a href=\"https://cognitiveclass.ai/mit-license/\">MIT License</a>.</p>" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "<h2>About the Authors:</h2> \n\n<a href=\"https://www.linkedin.com/in/joseph-s-50398b136/\">Joseph Santarcangelo</a> has a PhD in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.\n<p>\nOther contributors: <a href=\"https://www.linkedin.com/in/yi-leng-yao-84451275/\">Yi leng Yao</a>, <a href=\"www.linkedin.com/in/jiahui-mavis-zhou-a4537814a\">Mavis Zhou</a> \n</p>" | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "<h2>References :</h2> " | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": {}, | |
| "source": "<ul>\n <il>\n 1) <a href=\"https://research.stlouisfed.org/\">Economic Research at the St. Louis Fed </a>:<a href=\"https://fred.stlouisfed.org/series/UNRATE/\"> Civilian Unemployment Rate</a>\n </il> \n <p>\n <il>\n 2) <a href=\"https://github.com/datasets\">Data Packaged Core Datasets\n </a>\n </il> \n </p>\n \n</ul>\n</div>" | |
| } | |
| ], | |
| "metadata": { | |
| "kernelspec": { | |
| "display_name": "Python 3.6", | |
| "language": "python", | |
| "name": "python3" | |
| }, | |
| "language_info": { | |
| "codemirror_mode": { | |
| "name": "ipython", | |
| "version": 3 | |
| }, | |
| "file_extension": ".py", | |
| "mimetype": "text/x-python", | |
| "name": "python", | |
| "nbconvert_exporter": "python", | |
| "pygments_lexer": "ipython3", | |
| "version": "3.6.9" | |
| } | |
| }, | |
| "nbformat": 4, | |
| "nbformat_minor": 2 | |
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment