Jupyter Lab Api SpaceX.ipynb

{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "<center>\n    <img src=\"https://gitlab.com/ibm/skills-network/courses/placeholder101/-/raw/master/labs/module%201/images/IDSNlogo.png\" width=\"300\" alt=\"cognitiveclass.ai logo\"  />\n</center>\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# **SpaceX  Falcon 9 first stage Landing Prediction**\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Lab 1: Collecting the data\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Estimated time needed: **45** minutes\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "In this capstone, we will predict if the Falcon 9 first stage will land successfully. SpaceX advertises Falcon 9 rocket launches on its website with a cost of 62 million dollars; other providers cost upward of 165 million dollars each, much of the savings is because SpaceX can reuse the first stage. Therefore if we can determine if the first stage will land, we can determine the cost of a launch. This information can be used if an alternate company wants to bid against SpaceX for a rocket launch. In this lab, you will collect and make sure the data is in the correct format from an API. The following is an example of a successful and launch.\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "![](https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DS0701EN-SkillsNetwork/lab_v2/images/landing\\_1.gif)\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Several examples of an unsuccessful landing are shown here:\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "![](https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DS0701EN-SkillsNetwork/lab_v2/images/crash.gif)\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Most unsuccessful landings are planned. Space X performs a controlled landing in the oceans.\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "## Objectives\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "In this lab, you will make a get request to the SpaceX API. You will also do some basic data wrangling and formating.\n\n*   Request to the SpaceX API\n*   Clean the requested data\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "***\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "## Import Libraries and Define Auxiliary Functions\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "We will import the following libraries into the lab\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Requests allows us to make HTTP requests which we will use to get data from an API\nimport requests\n# Pandas is a software library written for the Python programming language for data manipulation and analysis.\nimport pandas as pd\n# NumPy is a library for the Python programming language, adding support for large, multi-dimensional arrays and matrices, along with a large collection of high-level mathematical functions to operate on these arrays\nimport numpy as np\n# Datetime is a library that allows us to represent dates\nimport datetime\n\n# Setting this option will print all collumns of a dataframe\npd.set_option('display.max_columns', None)\n# Setting this option will print all of the data in a feature\npd.set_option('display.max_colwidth', None)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Below we will define a series of helper functions that will help us use the API to extract information using identification numbers in the launch data.\n\nFrom the <code>rocket</code> column we would like to learn the booster name.\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Takes the dataset and uses the rocket column to call the API and append the data to the list\ndef getBoosterVersion(data):\n    for x in data['rocket']:\n        response = requests.get(\"https://api.spacexdata.com/v4/rockets/\"+str(x)).json()\n        BoosterVersion.append(response['name'])"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "From the <code>launchpad</code> we would like to know the name of the launch site being used, the logitude, and the latitude.\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Takes the dataset and uses the launchpad column to call the API and append the data to the list\ndef getLaunchSite(data):\n    for x in data['launchpad']:\n        response = requests.get(\"https://api.spacexdata.com/v4/launchpads/\"+str(x)).json()\n        Longitude.append(response['longitude'])\n        Latitude.append(response['latitude'])\n        LaunchSite.append(response['name'])"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "From the <code>payload</code> we would like to learn the mass of the payload and the orbit that it is going to.\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Takes the dataset and uses the payloads column to call the API and append the data to the lists\ndef getPayloadData(data):\n    for load in data['payloads']:\n        response = requests.get(\"https://api.spacexdata.com/v4/payloads/\"+load).json()\n        PayloadMass.append(response['mass_kg'])\n        Orbit.append(response['orbit'])"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "From <code>cores</code> we would like to learn the outcome of the landing, the type of the landing, number of flights with that core, whether gridfins were used, wheter the core is reused, wheter legs were used, the landing pad used, the block of the core which is a number used to seperate version of cores, the number of times this specific core has been reused, and the serial of the core.\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Takes the dataset and uses the cores column to call the API and append the data to the lists\ndef getCoreData(data):\n    for core in data['cores']:\n            if core['core'] != None:\n                response = requests.get(\"https://api.spacexdata.com/v4/cores/\"+core['core']).json()\n                Block.append(response['block'])\n                ReusedCount.append(response['reuse_count'])\n                Serial.append(response['serial'])\n            else:\n                Block.append(None)\n                ReusedCount.append(None)\n                Serial.append(None)\n            Outcome.append(str(core['landing_success'])+' '+str(core['landing_type']))\n            Flights.append(core['flight'])\n            GridFins.append(core['gridfins'])\n            Reused.append(core['reused'])\n            Legs.append(core['legs'])\n            LandingPad.append(core['landpad'])"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Now let's start requesting rocket launch data from SpaceX API with the following URL:\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "spacex_url=\"https://api.spacexdata.com/v4/launches/past\""
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "response = requests.get(spacex_url)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Check the content of the response\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "print(response.content)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "You should see the response contains massive information about SpaceX launches. Next, let's try to discover some more relevant information for this project.\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "### Task 1: Request and parse the SpaceX launch data using the GET request\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "To make the requested JSON results more consistent, we will use the following static response object for this project:\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "static_json_url='https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBM-DS0321EN-SkillsNetwork/datasets/API_call_spacex_api.json'"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "We should see that the request was successfull with the 200 status response code\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "response.status_code"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Now we decode the response content as a Json using <code>.json()</code> and turn it into a Pandas dataframe using <code>.json_normalize()</code>\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Use json_normalize meethod to convert the json result into a dataframe\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Using the dataframe <code>data</code> print the first 5 rows\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Get the head of the dataframe\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "You will notice that a lot of the data are IDs. For example the rocket column has no information about the rocket just an identification number.\n\nWe will now use the API again to get information about the launches using the IDs given for each launch. Specifically we will be using columns <code>rocket</code>, <code>payloads</code>, <code>launchpad</code>, and <code>cores</code>.\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Lets take a subset of our dataframe keeping only the features we want and the flight number, and date_utc.\ndata = data[['rocket', 'payloads', 'launchpad', 'cores', 'flight_number', 'date_utc']]\n\n# We will remove rows with multiple cores because those are falcon rockets with 2 extra rocket boosters and rows that have multiple payloads in a single rocket.\ndata = data[data['cores'].map(len)==1]\ndata = data[data['payloads'].map(len)==1]\n\n# Since payloads and cores are lists of size 1 we will also extract the single value in the list and replace the feature.\ndata['cores'] = data['cores'].map(lambda x : x[0])\ndata['payloads'] = data['payloads'].map(lambda x : x[0])\n\n# We also want to convert the date_utc to a datetime datatype and then extracting the date leaving the time\ndata['date'] = pd.to_datetime(data['date_utc']).dt.date\n\n# Using the date we will restrict the dates of the launches\ndata = data[data['date'] <= datetime.date(2020, 11, 13)]"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "*   From the <code>rocket</code> we would like to learn the booster name\n\n*   From the <code>payload</code> we would like to learn the mass of the payload and the orbit that it is going to\n\n*   From the <code>launchpad</code> we would like to know the name of the launch site being used, the longitude, and the latitude.\n\n*   From <code>cores</code> we would like to learn the outcome of the landing, the type of the landing, number of flights with that core, whether gridfins were used, whether the core is reused, whether legs were used, the landing pad used, the block of the core which is a number used to seperate version of cores, the number of times this specific core has been reused, and the serial of the core.\n\nThe data from these requests will be stored in lists and will be used to create a new dataframe.\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "#Global variables \nBoosterVersion = []\nPayloadMass = []\nOrbit = []\nLaunchSite = []\nOutcome = []\nFlights = []\nGridFins = []\nReused = []\nLegs = []\nLandingPad = []\nBlock = []\nReusedCount = []\nSerial = []\nLongitude = []\nLatitude = []"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "These functions will apply the outputs globally to the above variables. Let's take a looks at <code>BoosterVersion</code> variable. Before we apply  <code>getBoosterVersion</code> the list is empty:\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "BoosterVersion"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Now, let's apply <code> getBoosterVersion</code> function method to get the booster version\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Call getBoosterVersion\ngetBoosterVersion(data)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "the list has now been update\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "BoosterVersion[0:5]"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "we can apply the rest of the  functions here:\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Call getLaunchSite\ngetLaunchSite(data)"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Call getPayloadData\ngetPayloadData(data)"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Call getCoreData\ngetCoreData(data)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Finally lets construct our dataset using the data we have obtained. We we combine the columns into a dictionary.\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "launch_dict = {'FlightNumber': list(data['flight_number']),\n'Date': list(data['date']),\n'BoosterVersion':BoosterVersion,\n'PayloadMass':PayloadMass,\n'Orbit':Orbit,\n'LaunchSite':LaunchSite,\n'Outcome':Outcome,\n'Flights':Flights,\n'GridFins':GridFins,\n'Reused':Reused,\n'Legs':Legs,\n'LandingPad':LandingPad,\n'Block':Block,\n'ReusedCount':ReusedCount,\n'Serial':Serial,\n'Longitude': Longitude,\n'Latitude': Latitude}\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Then, we need to create a Pandas data frame from the dictionary launch_dict.\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Create a data from launch_dict\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Show the summary of the dataframe\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Show the head of the dataframe"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "### Task 2: Filter the dataframe to only include `Falcon 9` launches\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Finally we will remove the Falcon 1 launches keeping only the Falcon 9 launches. Filter the data dataframe using the <code>BoosterVersion</code> column to only keep the Falcon 9 launches. Save the filtered data to a new dataframe called <code>data_falcon9</code>.\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Hint data['BoosterVersion']!='Falcon 1'\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Now that we have removed some values we should reset the FlgihtNumber column\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "data_falcon9.loc[:,'FlightNumber'] = list(range(1, data_falcon9.shape[0]+1))\ndata_falcon9"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "## Data Wrangling\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "We can see below that some of the rows are missing values in our dataset.\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "data_falcon9.isnull().sum()"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Before we can continue we must deal with these missing values. The <code>LandingPad</code> column will retain None values to represent when landing pads were not used.\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "### Task 3: Dealing with Missing Values\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Calculate below the mean for the <code>PayloadMass</code> using the <code>.mean()</code>. Then use the mean and the <code>.replace()</code> function to replace `np.nan` values in the data with the mean you calculated.\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Calculate the mean value of PayloadMass column\n\n# Replace the np.nan values with its mean value"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "You should see the number of missing values of the <code>PayLoadMass</code> change to zero.\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Now we should have no missing values in our dataset except for in <code>LandingPad</code>.\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "We can now export it to a <b>CSV</b> for the next section,but to make the answers consistent, in the next lab we will provide data in a pre-selected date range.\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "<code>data_falcon9.to_csv('dataset_part\\_1.csv', index=False)</code>\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "## Authors\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "<a href=\"https://www.linkedin.com/in/joseph-s-50398b136/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDS0321ENSkillsNetwork26802033-2021-01-01\">Joseph Santarcangelo</a> has a PhD in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "## Change Log\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "| Date (YYYY-MM-DD) | Version | Changed By | Change Description                  |\n| ----------------- | ------- | ---------- | ----------------------------------- |\n| 2020-09-20        | 1.1     | Joseph     | get result each time you run        |\n| 2020-09-20        | 1.1     | Azim       | Created Part 1 Lab using SpaceX API |\n| 2020-09-20        | 1.0     | Joseph     | Modified Multiple Areas             |\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Copyright \u00a9 2021 IBM Corporation. All rights reserved.\n"
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3.9",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.9.7"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 4
}