Skip to content

Instantly share code, notes, and snippets.

@natzir

natzir/crux-api-bulk.ipynb

Created July 4, 2023 09:10
Show Gist options
  • Save natzir/e032aaf0d29b09fdce7fcf13bba55249 to your computer and use it in GitHub Desktop.
Save natzir/e032aaf0d29b09fdce7fcf13bba55249 to your computer and use it in GitHub Desktop.
Download ZIP
CrUX-API-bulk.ipynb
Display the source blob
Display the rendered blob
Raw
crux-api-bulk.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"provenance": [],
"authorship_tag": "ABX9TyNCaVKy7NouV2tumQHsqrdD",
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/natzir/e032aaf0d29b09fdce7fcf13bba55249/crux-api-bulk.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"source": [
"# Get the CrUX Metrics in bulk\n",
"\n",
"\n",
"---\n",
"\n",
"**Author:** Natzir Turrado, Technical SEO / Data Scientist\n",
"<br>**Twitter:** [@natzir9](https://twitter.com/natzir9)\n",
"\n",
"---\n",
"\n",
"This Colab notebook allows you to get the Chrome User Experience Report (CrUX) metrics of a list of URLs. It's ideal for benchmarking against a list of competitors of your choice.\n",
"\n",
"If you're aiming to benchmark Google's SERP for a query, make [use of this code](https://colab.research.google.com/drive/1WXHgV9OsBbh9wciSXIj6H50B97OuJDJb).\n",
"\n",
"Before you can use this document, please make sure to **create a copy** of it in your Google Drive (File > Save a copy in Drive).\n",
"\n",
"---\n",
"\n"
],
"metadata": {
"id": "gsJCeKB3cSOG"
}
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "5tvkQpogXxu8",
"cellView": "form"
},
"outputs": [],
"source": [
"#@markdown # Step 1: 'Play' this cell to install the the required libraries\n",
"#@markdown ---\n",
"!apt install nodejs\n",
"!npm install crux-api\n",
"!npm install node-fetch@2.6.1"
]
},
{
"cell_type": "code",
"source": [
"#@markdown # Step 2: Creating a node.js script for retrieving CrUX data\n",
"#@markdown In this step, a Node.js script will be created to retrieve CrUX data. This data captures user experience metrics from actual browsers all around the world.\n",
"#@markdown\n",
"#@markdown The script will make requests to CrUX API for the URLs you will provide bellow.\n",
"#@markdown\n",
"#@markdown If CrUX data for a particular URL is unavailable, the script will fetch the origin data from that URL. Please note that due to criteria such as public discoverability and significant user traffic, not all urls are included in its dataset.\n",
"#@markdown\n",
"#@markdown This step must be performed independently because a single Colab cell cannot support both JavaScript and Python scripts concurrently.\n",
"#@markdown\n",
"#@markdown ### Continue by running this cell.\n",
"#@markdown ---\n",
"\n",
"\n",
"%%writefile script.mjs\n",
"const fs = require('fs');\n",
"const createQueryRecord = require('crux-api').createQueryRecord;\n",
"const nodeFetch = require('node-fetch');\n",
"const queryRecord = createQueryRecord({ key: JSON.parse(fs.readFileSync('data.json', 'utf8')).key, fetch: nodeFetch });\n",
"let device = JSON.parse(fs.readFileSync('data.json', 'utf8')).device;\n",
"const urls = JSON.parse(fs.readFileSync('data.json', 'utf8')).urls;\n",
"\n",
"async function getData(url) {\n",
" let res = await queryRecord({\n",
" url: url,\n",
" formFactor: device\n",
" });\n",
"\n",
" let dataType = 'URL';\n",
"\n",
" if (!res || Object.keys(res).length === 0) {\n",
" res = await queryRecord({\n",
" origin: url,\n",
" formFactor: device\n",
" });\n",
" dataType = 'Origin';\n",
" }\n",
"\n",
" return {\n",
" url: url,\n",
" dataType: dataType,\n",
" response: res\n",
" };\n",
"}\n",
"\n",
"async function main() {\n",
" const results = [];\n",
"\n",
" for (let i = 0; i < urls.length; i++) {\n",
" const data = await getData(urls[i]);\n",
" results.push(data);\n",
" }\n",
"\n",
" fs.writeFileSync('result.json', JSON.stringify(results, null, 2));\n",
"}\n",
"\n",
"main();"
],
"metadata": {
"id": "HZBFJLwUdp8u",
"cellView": "form"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"#@markdown # Step 3: Extract CrUX data in bulk\n",
"#@markdown Set up the CrUX API (get yours here https://console.cloud.google.com/marketplace/product/google/chromeuxreport.googleapis.com)\n",
"\n",
"CRUX_KEY = '' #@param {type: \"string\"}\n",
"\n",
"# @markdown Select the device\n",
"DEVICE = 'phone' #@param [\"phone\", \"desktop\", \"tablet\"]\n",
"\n",
"# @markdown Enter your URLs here, separated by line breaks or spaces. You can directly paste a list from a spreadsheet or text file.\n",
"URLS = 'https://www.adidas.es https://www.adidas.es/zapatilla-stan-smith-luxe/IG3389.html https://www.adidas.es/zapatillas-mujer https://www.nike.com https://www.nike.com/t/air-jordan-1-low-mens-shoes-0LXhbn/553558-215 https://www.nike.com/w/mens-jordan-shoes-37eefznik1zy7ok https://www.nordstrom.com https://www.nordstrom.com/browse/women/clothing https://www.nordstrom.com/s/kut-from-the-kloth-jasmine-top/4856576 https://www.patagonia.com/home/ https://www.patagonia.com/product/mens-baggies-shorts-5-inch/57022.html https://www.patagonia.com/shop/mens https://www.desigual.com/de_DE/ https://www.desigual.com/de_DE/23SWVW772000.html https://www.desigual.com/de_DE/damenmode/kleidung/kleider/' #@param {type: \"string\"}\n",
"urls_list = URLS.split()\n",
"\n",
"\n",
"data = {\n",
" \"key\": CRUX_KEY,\n",
" \"device\": DEVICE,\n",
" \"urls\":urls_list\n",
"}\n",
"\n",
"import json\n",
"\n",
"with open('data.json', 'w') as file:\n",
" json.dump(data, file)\n",
"\n",
"!node script.mjs\n",
"\n",
"import pandas as pd\n",
"import numpy as np\n",
"\n",
"with open('result.json') as file:\n",
" data = json.load(file)\n",
"\n",
"table = []\n",
"\n",
"key_metrics = [\n",
" 'cumulative_layout_shift',\n",
" 'first_contentful_paint',\n",
" 'first_input_delay',\n",
" 'interaction_to_next_paint',\n",
" 'largest_contentful_paint',\n",
" 'experimental_time_to_first_byte'\n",
"]\n",
"\n",
"for element in data:\n",
" url = element['url']\n",
" dataType = element['dataType'] if 'dataType' in element else None\n",
" register = {'url': url, 'dataType': dataType}\n",
"\n",
"\n",
" if 'response' in element and element['response'] is not None:\n",
" record = element['response']['record']\n",
" key = record['key']\n",
" metrics = record['metrics']\n",
"\n",
" for metric in key_metrics:\n",
" if metric in metrics:\n",
" metric_data = metrics[metric]\n",
" histogram = metric_data['histogram']\n",
" percentiles = metric_data['percentiles']\n",
" p75 = percentiles['p75']\n",
"\n",
" for i, data in enumerate(histogram):\n",
" register[f'{metric}_{i}_start'] = data.get('start')\n",
" register[f'{metric}_{i}_end'] = data.get('end')\n",
" register[f'{metric}_{i}_density'] = data.get('density')\n",
"\n",
" register[f'{metric}'] = p75\n",
"\n",
" end_1 = histogram[0]['end']\n",
" end_2 = histogram[1]['end']\n",
" if end_1 is not None and float(p75) < float(end_1):\n",
" register[f'{metric}_status'] = 'Good'\n",
" elif end_2 is not None and float(p75) > float(end_2):\n",
" register[f'{metric}_status'] = 'Bad'\n",
" else:\n",
" register[f'{metric}_status'] = 'Need improvement'\n",
" else:\n",
" for i in range(2):\n",
" register[f'{metric}_{i}_start'] = None\n",
" register[f'{metric}_{i}_end'] = None\n",
" register[f'{metric}_{i}_density'] = None\n",
" register[f'{metric}'] = None\n",
" register[f'{metric}_status'] = None\n",
" else:\n",
" for metric in key_metrics:\n",
" for i in range(2):\n",
" register[f'{metric}_{i}_start'] = None\n",
" register[f'{metric}_{i}_end'] = None\n",
" register[f'{metric}_{i}_density'] = None\n",
" register[f'{metric}'] = None\n",
" register[f'{metric}_status'] = None\n",
"\n",
" table.append(register)\n",
"\n",
"df_crux = pd.DataFrame(table)\n",
"\n",
"numeric_cols = df_crux.select_dtypes(include=[np.number]).columns\n",
"df_crux[numeric_cols] = df_crux[numeric_cols].where(pd.notnull(df_crux[numeric_cols]), None)\n",
"\n",
"df2 = df_crux.copy()\n",
"columns_to_remove = [col for col in df2.columns if 'start' in col or 'end' in col or 'density' in col]\n",
"df2.drop(columns=columns_to_remove, inplace=True)\n",
"df2 = df2.fillna(\"-\")\n",
"\n",
"for metric in key_metrics:\n",
" df2[metric] = df2[metric].astype(str) + \" (\" + df2[metric + '_status'] + \")\"\n",
"\n",
"status_columns = [metric + '_status' for metric in key_metrics]\n",
"df2.drop(columns=status_columns, inplace=True)\n",
"\n",
"df2 = df2.rename(columns={\n",
" \"cumulative_layout_shift\": \"CLS\",\n",
" \"first_contentful_paint\": \"FCP\",\n",
" \"first_input_delay\": \"FID\",\n",
" \"interaction_to_next_paint\": \"INP\",\n",
" \"largest_contentful_paint\": \"LCP\",\n",
" \"experimental_time_to_first_byte\": \"TTFB\",\n",
"})\n",
"\n",
"df2.sort_values(by='url', inplace=True)\n",
"\n",
"def color_based_on_value(value):\n",
" if 'Good' in value:\n",
" color = 'green'\n",
" elif 'Need improvement' in value:\n",
" color = 'orange'\n",
" elif 'Bad' in value:\n",
" color = 'red'\n",
" else:\n",
" color = 'black'\n",
" return 'color: %s' % color\n",
"\n",
"styled_df = df2.style.applymap(color_based_on_value, subset=['CLS', 'FCP', 'FID', 'INP', 'LCP', 'TTFB'])\n",
"styled_df\n",
"\n",
"#@markdown ## After setting the parameters, click the 'Play' button\n",
"#@markdown ---"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 519
},
"id": "EtXozScu3Uve",
"outputId": "b3888b0f-975d-4521-ee26-e197e7b8efb6",
"cellView": "form"
},
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<pandas.io.formats.style.Styler at 0x7f373a019c60>"
],
"text/html": [
"<style type=\"text/css\">\n",
"#T_3744a_row0_col2, #T_3744a_row0_col4, #T_3744a_row0_col5, #T_3744a_row0_col6, #T_3744a_row1_col2, #T_3744a_row1_col3, #T_3744a_row1_col4, #T_3744a_row1_col5, #T_3744a_row1_col6, #T_3744a_row1_col7, #T_3744a_row2_col3, #T_3744a_row2_col4, #T_3744a_row2_col5, #T_3744a_row2_col6, #T_3744a_row2_col7, #T_3744a_row3_col2, #T_3744a_row3_col5, #T_3744a_row4_col5, #T_3744a_row5_col5, #T_3744a_row5_col7, #T_3744a_row6_col2, #T_3744a_row6_col6, #T_3744a_row7_col2, #T_3744a_row7_col3, #T_3744a_row7_col5, #T_3744a_row7_col6, #T_3744a_row7_col7, #T_3744a_row9_col2, #T_3744a_row9_col5, #T_3744a_row10_col2, #T_3744a_row10_col3, #T_3744a_row10_col5, #T_3744a_row11_col3, #T_3744a_row11_col5, #T_3744a_row14_col5 {\n",
" color: red;\n",
"}\n",
"#T_3744a_row0_col3, #T_3744a_row0_col7, #T_3744a_row2_col2, #T_3744a_row3_col3, #T_3744a_row3_col7, #T_3744a_row4_col4, #T_3744a_row4_col6, #T_3744a_row5_col3, #T_3744a_row5_col4, #T_3744a_row5_col6, #T_3744a_row6_col3, #T_3744a_row6_col5, #T_3744a_row6_col7, #T_3744a_row8_col3, #T_3744a_row8_col5, #T_3744a_row8_col6, #T_3744a_row9_col3, #T_3744a_row9_col6, #T_3744a_row9_col7, #T_3744a_row10_col6, #T_3744a_row10_col7, #T_3744a_row11_col4, #T_3744a_row11_col6, #T_3744a_row11_col7, #T_3744a_row12_col3, #T_3744a_row12_col5, #T_3744a_row12_col6, #T_3744a_row12_col7, #T_3744a_row13_col5, #T_3744a_row13_col7, #T_3744a_row14_col3, #T_3744a_row14_col4, #T_3744a_row14_col7 {\n",
" color: orange;\n",
"}\n",
"#T_3744a_row3_col4, #T_3744a_row3_col6, #T_3744a_row4_col2, #T_3744a_row4_col3, #T_3744a_row4_col7, #T_3744a_row5_col2, #T_3744a_row6_col4, #T_3744a_row7_col4, #T_3744a_row8_col2, #T_3744a_row8_col4, #T_3744a_row8_col7, #T_3744a_row9_col4, #T_3744a_row10_col4, #T_3744a_row11_col2, #T_3744a_row12_col2, #T_3744a_row12_col4, #T_3744a_row13_col2, #T_3744a_row13_col3, #T_3744a_row13_col4, #T_3744a_row13_col6, #T_3744a_row14_col2, #T_3744a_row14_col6 {\n",
" color: green;\n",
"}\n",
"</style>\n",
"<table id=\"T_3744a\" class=\"dataframe\">\n",
" <thead>\n",
" <tr>\n",
" <th class=\"blank level0\" >&nbsp;</th>\n",
" <th id=\"T_3744a_level0_col0\" class=\"col_heading level0 col0\" >url</th>\n",
" <th id=\"T_3744a_level0_col1\" class=\"col_heading level0 col1\" >dataType</th>\n",
" <th id=\"T_3744a_level0_col2\" class=\"col_heading level0 col2\" >CLS</th>\n",
" <th id=\"T_3744a_level0_col3\" class=\"col_heading level0 col3\" >FCP</th>\n",
" <th id=\"T_3744a_level0_col4\" class=\"col_heading level0 col4\" >FID</th>\n",
" <th id=\"T_3744a_level0_col5\" class=\"col_heading level0 col5\" >INP</th>\n",
" <th id=\"T_3744a_level0_col6\" class=\"col_heading level0 col6\" >LCP</th>\n",
" <th id=\"T_3744a_level0_col7\" class=\"col_heading level0 col7\" >TTFB</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th id=\"T_3744a_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
" <td id=\"T_3744a_row0_col0\" class=\"data row0 col0\" >https://www.adidas.es</td>\n",
" <td id=\"T_3744a_row0_col1\" class=\"data row0 col1\" >URL</td>\n",
" <td id=\"T_3744a_row0_col2\" class=\"data row0 col2\" >0.48 (Bad)</td>\n",
" <td id=\"T_3744a_row0_col3\" class=\"data row0 col3\" >2287 (Need improvement)</td>\n",
" <td id=\"T_3744a_row0_col4\" class=\"data row0 col4\" >840 (Bad)</td>\n",
" <td id=\"T_3744a_row0_col5\" class=\"data row0 col5\" >3670 (Bad)</td>\n",
" <td id=\"T_3744a_row0_col6\" class=\"data row0 col6\" >6524 (Bad)</td>\n",
" <td id=\"T_3744a_row0_col7\" class=\"data row0 col7\" >1287 (Need improvement)</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_3744a_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
" <td id=\"T_3744a_row1_col0\" class=\"data row1 col0\" >https://www.adidas.es/zapatilla-stan-smith-luxe/IG3389.html</td>\n",
" <td id=\"T_3744a_row1_col1\" class=\"data row1 col1\" >Origin</td>\n",
" <td id=\"T_3744a_row1_col2\" class=\"data row1 col2\" >0.27 (Bad)</td>\n",
" <td id=\"T_3744a_row1_col3\" class=\"data row1 col3\" >3495 (Bad)</td>\n",
" <td id=\"T_3744a_row1_col4\" class=\"data row1 col4\" >1155 (Bad)</td>\n",
" <td id=\"T_3744a_row1_col5\" class=\"data row1 col5\" >3493 (Bad)</td>\n",
" <td id=\"T_3744a_row1_col6\" class=\"data row1 col6\" >4933 (Bad)</td>\n",
" <td id=\"T_3744a_row1_col7\" class=\"data row1 col7\" >2430 (Bad)</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_3744a_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
" <td id=\"T_3744a_row2_col0\" class=\"data row2 col0\" >https://www.adidas.es/zapatillas-mujer</td>\n",
" <td id=\"T_3744a_row2_col1\" class=\"data row2 col1\" >URL</td>\n",
" <td id=\"T_3744a_row2_col2\" class=\"data row2 col2\" >0.22 (Need improvement)</td>\n",
" <td id=\"T_3744a_row2_col3\" class=\"data row2 col3\" >3561 (Bad)</td>\n",
" <td id=\"T_3744a_row2_col4\" class=\"data row2 col4\" >2445 (Bad)</td>\n",
" <td id=\"T_3744a_row2_col5\" class=\"data row2 col5\" >5762 (Bad)</td>\n",
" <td id=\"T_3744a_row2_col6\" class=\"data row2 col6\" >4127 (Bad)</td>\n",
" <td id=\"T_3744a_row2_col7\" class=\"data row2 col7\" >2452 (Bad)</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_3744a_level0_row3\" class=\"row_heading level0 row3\" >12</th>\n",
" <td id=\"T_3744a_row3_col0\" class=\"data row3 col0\" >https://www.desigual.com/de_DE/</td>\n",
" <td id=\"T_3744a_row3_col1\" class=\"data row3 col1\" >URL</td>\n",
" <td id=\"T_3744a_row3_col2\" class=\"data row3 col2\" >0.30 (Bad)</td>\n",
" <td id=\"T_3744a_row3_col3\" class=\"data row3 col3\" >1939 (Need improvement)</td>\n",
" <td id=\"T_3744a_row3_col4\" class=\"data row3 col4\" >85 (Good)</td>\n",
" <td id=\"T_3744a_row3_col5\" class=\"data row3 col5\" >553 (Bad)</td>\n",
" <td id=\"T_3744a_row3_col6\" class=\"data row3 col6\" >2345 (Good)</td>\n",
" <td id=\"T_3744a_row3_col7\" class=\"data row3 col7\" >1214 (Need improvement)</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_3744a_level0_row4\" class=\"row_heading level0 row4\" >13</th>\n",
" <td id=\"T_3744a_row4_col0\" class=\"data row4 col0\" >https://www.desigual.com/de_DE/23SWVW772000.html</td>\n",
" <td id=\"T_3744a_row4_col1\" class=\"data row4 col1\" >URL</td>\n",
" <td id=\"T_3744a_row4_col2\" class=\"data row4 col2\" >0.00 (Good)</td>\n",
" <td id=\"T_3744a_row4_col3\" class=\"data row4 col3\" >1351 (Good)</td>\n",
" <td id=\"T_3744a_row4_col4\" class=\"data row4 col4\" >187 (Need improvement)</td>\n",
" <td id=\"T_3744a_row4_col5\" class=\"data row4 col5\" >534 (Bad)</td>\n",
" <td id=\"T_3744a_row4_col6\" class=\"data row4 col6\" >3551 (Need improvement)</td>\n",
" <td id=\"T_3744a_row4_col7\" class=\"data row4 col7\" >678 (Good)</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_3744a_level0_row5\" class=\"row_heading level0 row5\" >14</th>\n",
" <td id=\"T_3744a_row5_col0\" class=\"data row5 col0\" >https://www.desigual.com/de_DE/damenmode/kleidung/kleider/</td>\n",
" <td id=\"T_3744a_row5_col1\" class=\"data row5 col1\" >URL</td>\n",
" <td id=\"T_3744a_row5_col2\" class=\"data row5 col2\" >0.01 (Good)</td>\n",
" <td id=\"T_3744a_row5_col3\" class=\"data row5 col3\" >2629 (Need improvement)</td>\n",
" <td id=\"T_3744a_row5_col4\" class=\"data row5 col4\" >199 (Need improvement)</td>\n",
" <td id=\"T_3744a_row5_col5\" class=\"data row5 col5\" >835 (Bad)</td>\n",
" <td id=\"T_3744a_row5_col6\" class=\"data row5 col6\" >3406 (Need improvement)</td>\n",
" <td id=\"T_3744a_row5_col7\" class=\"data row5 col7\" >1937 (Bad)</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_3744a_level0_row6\" class=\"row_heading level0 row6\" >3</th>\n",
" <td id=\"T_3744a_row6_col0\" class=\"data row6 col0\" >https://www.nike.com</td>\n",
" <td id=\"T_3744a_row6_col1\" class=\"data row6 col1\" >URL</td>\n",
" <td id=\"T_3744a_row6_col2\" class=\"data row6 col2\" >0.96 (Bad)</td>\n",
" <td id=\"T_3744a_row6_col3\" class=\"data row6 col3\" >2983 (Need improvement)</td>\n",
" <td id=\"T_3744a_row6_col4\" class=\"data row6 col4\" >71 (Good)</td>\n",
" <td id=\"T_3744a_row6_col5\" class=\"data row6 col5\" >412 (Need improvement)</td>\n",
" <td id=\"T_3744a_row6_col6\" class=\"data row6 col6\" >4616 (Bad)</td>\n",
" <td id=\"T_3744a_row6_col7\" class=\"data row6 col7\" >1052 (Need improvement)</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_3744a_level0_row7\" class=\"row_heading level0 row7\" >4</th>\n",
" <td id=\"T_3744a_row7_col0\" class=\"data row7 col0\" >https://www.nike.com/t/air-jordan-1-low-mens-shoes-0LXhbn/553558-215</td>\n",
" <td id=\"T_3744a_row7_col1\" class=\"data row7 col1\" >URL</td>\n",
" <td id=\"T_3744a_row7_col2\" class=\"data row7 col2\" >0.66 (Bad)</td>\n",
" <td id=\"T_3744a_row7_col3\" class=\"data row7 col3\" >3168 (Bad)</td>\n",
" <td id=\"T_3744a_row7_col4\" class=\"data row7 col4\" >43 (Good)</td>\n",
" <td id=\"T_3744a_row7_col5\" class=\"data row7 col5\" >778 (Bad)</td>\n",
" <td id=\"T_3744a_row7_col6\" class=\"data row7 col6\" >4526 (Bad)</td>\n",
" <td id=\"T_3744a_row7_col7\" class=\"data row7 col7\" >1846 (Bad)</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_3744a_level0_row8\" class=\"row_heading level0 row8\" >5</th>\n",
" <td id=\"T_3744a_row8_col0\" class=\"data row8 col0\" >https://www.nike.com/w/mens-jordan-shoes-37eefznik1zy7ok</td>\n",
" <td id=\"T_3744a_row8_col1\" class=\"data row8 col1\" >URL</td>\n",
" <td id=\"T_3744a_row8_col2\" class=\"data row8 col2\" >0.01 (Good)</td>\n",
" <td id=\"T_3744a_row8_col3\" class=\"data row8 col3\" >1836 (Need improvement)</td>\n",
" <td id=\"T_3744a_row8_col4\" class=\"data row8 col4\" >17 (Good)</td>\n",
" <td id=\"T_3744a_row8_col5\" class=\"data row8 col5\" >339 (Need improvement)</td>\n",
" <td id=\"T_3744a_row8_col6\" class=\"data row8 col6\" >3603 (Need improvement)</td>\n",
" <td id=\"T_3744a_row8_col7\" class=\"data row8 col7\" >786 (Good)</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_3744a_level0_row9\" class=\"row_heading level0 row9\" >6</th>\n",
" <td id=\"T_3744a_row9_col0\" class=\"data row9 col0\" >https://www.nordstrom.com</td>\n",
" <td id=\"T_3744a_row9_col1\" class=\"data row9 col1\" >URL</td>\n",
" <td id=\"T_3744a_row9_col2\" class=\"data row9 col2\" >1.13 (Bad)</td>\n",
" <td id=\"T_3744a_row9_col3\" class=\"data row9 col3\" >2584 (Need improvement)</td>\n",
" <td id=\"T_3744a_row9_col4\" class=\"data row9 col4\" >44 (Good)</td>\n",
" <td id=\"T_3744a_row9_col5\" class=\"data row9 col5\" >764 (Bad)</td>\n",
" <td id=\"T_3744a_row9_col6\" class=\"data row9 col6\" >3874 (Need improvement)</td>\n",
" <td id=\"T_3744a_row9_col7\" class=\"data row9 col7\" >1142 (Need improvement)</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_3744a_level0_row10\" class=\"row_heading level0 row10\" >7</th>\n",
" <td id=\"T_3744a_row10_col0\" class=\"data row10 col0\" >https://www.nordstrom.com/browse/women/clothing</td>\n",
" <td id=\"T_3744a_row10_col1\" class=\"data row10 col1\" >URL</td>\n",
" <td id=\"T_3744a_row10_col2\" class=\"data row10 col2\" >0.59 (Bad)</td>\n",
" <td id=\"T_3744a_row10_col3\" class=\"data row10 col3\" >3169 (Bad)</td>\n",
" <td id=\"T_3744a_row10_col4\" class=\"data row10 col4\" >27 (Good)</td>\n",
" <td id=\"T_3744a_row10_col5\" class=\"data row10 col5\" >804 (Bad)</td>\n",
" <td id=\"T_3744a_row10_col6\" class=\"data row10 col6\" >3652 (Need improvement)</td>\n",
" <td id=\"T_3744a_row10_col7\" class=\"data row10 col7\" >1112 (Need improvement)</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_3744a_level0_row11\" class=\"row_heading level0 row11\" >8</th>\n",
" <td id=\"T_3744a_row11_col0\" class=\"data row11 col0\" >https://www.nordstrom.com/s/kut-from-the-kloth-jasmine-top/4856576</td>\n",
" <td id=\"T_3744a_row11_col1\" class=\"data row11 col1\" >URL</td>\n",
" <td id=\"T_3744a_row11_col2\" class=\"data row11 col2\" >0.02 (Good)</td>\n",
" <td id=\"T_3744a_row11_col3\" class=\"data row11 col3\" >3061 (Bad)</td>\n",
" <td id=\"T_3744a_row11_col4\" class=\"data row11 col4\" >198 (Need improvement)</td>\n",
" <td id=\"T_3744a_row11_col5\" class=\"data row11 col5\" >778 (Bad)</td>\n",
" <td id=\"T_3744a_row11_col6\" class=\"data row11 col6\" >3532 (Need improvement)</td>\n",
" <td id=\"T_3744a_row11_col7\" class=\"data row11 col7\" >1762 (Need improvement)</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_3744a_level0_row12\" class=\"row_heading level0 row12\" >9</th>\n",
" <td id=\"T_3744a_row12_col0\" class=\"data row12 col0\" >https://www.patagonia.com/home/</td>\n",
" <td id=\"T_3744a_row12_col1\" class=\"data row12 col1\" >URL</td>\n",
" <td id=\"T_3744a_row12_col2\" class=\"data row12 col2\" >0.01 (Good)</td>\n",
" <td id=\"T_3744a_row12_col3\" class=\"data row12 col3\" >2406 (Need improvement)</td>\n",
" <td id=\"T_3744a_row12_col4\" class=\"data row12 col4\" >34 (Good)</td>\n",
" <td id=\"T_3744a_row12_col5\" class=\"data row12 col5\" >305 (Need improvement)</td>\n",
" <td id=\"T_3744a_row12_col6\" class=\"data row12 col6\" >2695 (Need improvement)</td>\n",
" <td id=\"T_3744a_row12_col7\" class=\"data row12 col7\" >1111 (Need improvement)</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_3744a_level0_row13\" class=\"row_heading level0 row13\" >10</th>\n",
" <td id=\"T_3744a_row13_col0\" class=\"data row13 col0\" >https://www.patagonia.com/product/mens-baggies-shorts-5-inch/57022.html</td>\n",
" <td id=\"T_3744a_row13_col1\" class=\"data row13 col1\" >URL</td>\n",
" <td id=\"T_3744a_row13_col2\" class=\"data row13 col2\" >0.00 (Good)</td>\n",
" <td id=\"T_3744a_row13_col3\" class=\"data row13 col3\" >1774 (Good)</td>\n",
" <td id=\"T_3744a_row13_col4\" class=\"data row13 col4\" >53 (Good)</td>\n",
" <td id=\"T_3744a_row13_col5\" class=\"data row13 col5\" >434 (Need improvement)</td>\n",
" <td id=\"T_3744a_row13_col6\" class=\"data row13 col6\" >2015 (Good)</td>\n",
" <td id=\"T_3744a_row13_col7\" class=\"data row13 col7\" >933 (Need improvement)</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_3744a_level0_row14\" class=\"row_heading level0 row14\" >11</th>\n",
" <td id=\"T_3744a_row14_col0\" class=\"data row14 col0\" >https://www.patagonia.com/shop/mens</td>\n",
" <td id=\"T_3744a_row14_col1\" class=\"data row14 col1\" >URL</td>\n",
" <td id=\"T_3744a_row14_col2\" class=\"data row14 col2\" >0.01 (Good)</td>\n",
" <td id=\"T_3744a_row14_col3\" class=\"data row14 col3\" >2084 (Need improvement)</td>\n",
" <td id=\"T_3744a_row14_col4\" class=\"data row14 col4\" >241 (Need improvement)</td>\n",
" <td id=\"T_3744a_row14_col5\" class=\"data row14 col5\" >1510 (Bad)</td>\n",
" <td id=\"T_3744a_row14_col6\" class=\"data row14 col6\" >2462 (Good)</td>\n",
" <td id=\"T_3744a_row14_col7\" class=\"data row14 col7\" >961 (Need improvement)</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n"
]
},
"metadata": {},
"execution_count": 38
}
]
},
{
"cell_type": "code",
"source": [
"#@markdown # Step 4: Press 'Play' to download the data (optional)\n",
"\n",
"from google.colab import files\n",
"\n",
"filename = f'CrUX-API-bulk-{DEVICE}.csv'\n",
"merged_df.to_csv(filename)\n",
"files.download(filename)\n",
"\n",
"#@markdown ---"
],
"metadata": {
"cellView": "form",
"id": "Cdn72xjCFjUc"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"If you want to modify the code, here are the documentation of the CrUX API: https://developer.chrome.com/docs/crux/api/\n",
"\n",
"---\n",
"\n",
"**Author:** Natzir Turrado, Technical SEO / Data Scientist\n",
"<br>**Twitter:** [@natzir9](https://twitter.com/natzir9)\n",
"\n",
"---"
],
"metadata": {
"id": "JC6LY6ztGjeE"
}
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment