Skip to content

Instantly share code, notes, and snippets.

@statmike

statmike/bq-information_schema-for-high-slot-repeating-jobs.ipynb Secret

Last active October 15, 2021 10:41
Show Gist options
  • Save statmike/8f1fc48700bd57026c68cd0f3fcc4b64 to your computer and use it in GitHub Desktop.
Save statmike/8f1fc48700bd57026c68cd0f3fcc4b64 to your computer and use it in GitHub Desktop.
Download ZIP
BQ INFORMATION_SCHEMA For High Slot Repeating Jobs
Display the source blob
Display the rendered blob
Raw
bq-information_schema-for-high-slot-repeating-jobs.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "BQ INFORMATION_SCHEMA For High Slot Repeating Jobs",
"provenance": [],
"collapsed_sections": [],
"authorship_tag": "ABX9TyPMKm+1jqTExdvpjH69DVTz",
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/statmike/8f1fc48700bd57026c68cd0f3fcc4b64/bq-information_schema-for-high-slot-repeating-jobs.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qWQgzcWqHlhx"
},
"source": [
"# Goals\n",
"- For a specify project...\n",
"- Identify queries that are associated with high slot utilization moments\n",
"- Identy when these jobs are associated with >2000 slot needs\n",
"- Identify the specify jobs in [INFORMATION_SCHEMA.JOBS_BY_PROJECT](https://cloud.google.com/bigquery/docs/information-schema-jobs)\n",
"- Use [INFORMATION_SCHEMA.JOBS_TIMELINE_BY_PROJECT](https://cloud.google.com/bigquery/docs/information-schema-jobs-timeline)\n",
" - has a row for each job_id for each period_start (TIMESTAMP to second)\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "d3p8-sVBRHwP"
},
"source": [
"## Setup"
]
},
{
"cell_type": "code",
"metadata": {
"id": "qa4xCvOnHiUN"
},
"source": [
"from google.colab import auth\n",
"auth.authenticate_user()"
],
"execution_count": 1,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "rH5-Pyb2HNaH"
},
"source": [
"project_id = 'statmike-project-1'"
],
"execution_count": 2,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "srivUFXWKwXg",
"outputId": "d3190031-227b-4fe5-faaa-3f2770c1d8f7"
},
"source": [
"!gcloud config set project {project_id}"
],
"execution_count": 3,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Updated property [core/project].\n"
]
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "yBzyXnNpKyNP"
},
"source": [
"from google.cloud import bigquery\n",
"bq = bigquery.Client(project = project_id)"
],
"execution_count": 4,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "g5ak1cu-QZaq"
},
"source": [
"# IDEA 1: Find A Specific Query\n",
"- Identify the moments when slot usage >= 2000 while a specific query was running"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "7VxJYP6-bRID"
},
"source": [
"## A Specific Query\n",
"- Note the commented line with a name\n",
"- This is a way to identify a specific query\n",
"- You could also search for an exact query or a unique part of a query\n",
"- Run This test a few times\n",
"- Notice it is set to not use caching"
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "S-G-or8_L4i8",
"outputId": "b926ca83-587b-4eef-813b-b39536aa2ad8"
},
"source": [
"query = f\"\"\"\n",
"SELECT COUNT(*)\n",
"# <My Named Query>\n",
"FROM bigquery-public-data.google_trends.top_rising_terms\n",
"WHERE dma_name = 'Atlanta GA'\n",
"\"\"\"\n",
"job = bq.query(query = query, job_config = bigquery.QueryJobConfig(use_query_cache = False))\n",
"job.result()"
],
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<google.cloud.bigquery.table.RowIterator at 0x7fbe0cf08790>"
]
},
"metadata": {},
"execution_count": 17
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PhWjNXwxc2LS"
},
"source": [
"## Slot Usage during the TIMESTAMP (seconds) that the specific query above was running\n",
"- First, subquery\n",
" - find the job_id's associated with the query that has the unique name we gave it in a comment\n",
" - Also, Give this query a name and exclude it from the data returned in the subquery\n",
"- return all the timeslice periods (seconds) where any of the specifc queries associated job_id's were running along with helpful metrics\n",
"- Note that period_avg_slot_usage is the average slots used in the project for the current period_start second\n",
"- If this query frequenty has job_id where the period_avg_slot_usage > 2000 then it could be the query that is pressuring the on-demand load for the project"
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 465
},
"id": "Kp-Ggw_TLHNf",
"outputId": "673d806f-1da4-4939-842c-0c4c9ea1be6c"
},
"source": [
"query = f\"\"\"\n",
"# Looking For <My Named Query>\n",
"WITH\n",
"jobs as (\n",
" SELECT job_id, project_id, query as job_query, \n",
" TIMESTAMP_DIFF(end_time, start_time, MILLISECOND)/1000 AS job_duration,\n",
" total_slot_ms / TIMESTAMP_DIFF(end_time, start_time, MILLISECOND) as job_avg_slot_usage\n",
" FROM {project_id}.region-us.INFORMATION_SCHEMA.JOBS_BY_PROJECT\n",
" WHERE CONTAINS_SUBSTR(query, '# <My Named Query>') AND NOT CONTAINS_SUBSTR(query, '# Looking For <My Named Query>')\n",
")\n",
"SELECT a.*, b.period_start,\n",
" b.period_slot_ms/1000 AS period_avg_slots_usage,\n",
" b.job_creation_time, b.job_start_time, b.job_end_time, b.state\n",
"FROM jobs AS a\n",
"INNER JOIN {project_id}.region-us.INFORMATION_SCHEMA.JOBS_TIMELINE_BY_PROJECT AS b\n",
"USING(job_id)\n",
"ORDER BY b.period_start, b.job_id\n",
"\"\"\"\n",
"result = bq.query(query = query).to_dataframe()\n",
"result"
],
"execution_count": 5,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>job_id</th>\n",
" <th>project_id</th>\n",
" <th>job_query</th>\n",
" <th>job_duration</th>\n",
" <th>job_avg_slot_usage</th>\n",
" <th>period_start</th>\n",
" <th>period_avg_slots_usage</th>\n",
" <th>job_creation_time</th>\n",
" <th>job_start_time</th>\n",
" <th>job_end_time</th>\n",
" <th>state</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>22163268-9ebb-46e1-a9ad-0d569a14791b</td>\n",
" <td>statmike-project-1</td>\n",
" <td>\\nSELECT COUNT(*)\\n# &lt;My Named Query&gt;\\nFROM bi...</td>\n",
" <td>0.258</td>\n",
" <td>2.290698</td>\n",
" <td>2021-10-15 00:46:08+00:00</td>\n",
" <td>0.591</td>\n",
" <td>2021-10-15 00:46:08.239000+00:00</td>\n",
" <td>2021-10-15 00:46:08.323000+00:00</td>\n",
" <td>2021-10-15 00:46:08.581000+00:00</td>\n",
" <td>DONE</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>5766b7cd-ac86-483a-9a67-b8a8c4b503b7</td>\n",
" <td>statmike-project-1</td>\n",
" <td>\\nSELECT COUNT(*)\\n# &lt;My Named Query&gt;\\nFROM bi...</td>\n",
" <td>0.264</td>\n",
" <td>2.000000</td>\n",
" <td>2021-10-15 00:46:12+00:00</td>\n",
" <td>0.528</td>\n",
" <td>2021-10-15 00:46:12.358000+00:00</td>\n",
" <td>2021-10-15 00:46:12.422000+00:00</td>\n",
" <td>2021-10-15 00:46:12.686000+00:00</td>\n",
" <td>DONE</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>b1aae587-d26b-4661-8c6c-3072c97855c0</td>\n",
" <td>statmike-project-1</td>\n",
" <td>\\nSELECT COUNT(*)\\n# &lt;My Named Query&gt;\\nFROM bi...</td>\n",
" <td>0.307</td>\n",
" <td>2.061889</td>\n",
" <td>2021-10-15 00:46:14+00:00</td>\n",
" <td>0.633</td>\n",
" <td>2021-10-15 00:46:14.503000+00:00</td>\n",
" <td>2021-10-15 00:46:14.566000+00:00</td>\n",
" <td>2021-10-15 00:46:14.873000+00:00</td>\n",
" <td>DONE</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>d5b3d313-7281-45d8-bad5-a82da5b126ae</td>\n",
" <td>statmike-project-1</td>\n",
" <td>\\nSELECT COUNT(*)\\n# &lt;My Named Query&gt;\\nFROM bi...</td>\n",
" <td>0.246</td>\n",
" <td>2.174797</td>\n",
" <td>2021-10-15 00:46:17+00:00</td>\n",
" <td>0.535</td>\n",
" <td>2021-10-15 00:46:17.648000+00:00</td>\n",
" <td>2021-10-15 00:46:17.706000+00:00</td>\n",
" <td>2021-10-15 00:46:17.952000+00:00</td>\n",
" <td>DONE</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" job_id ... state\n",
"0 22163268-9ebb-46e1-a9ad-0d569a14791b ... DONE\n",
"1 5766b7cd-ac86-483a-9a67-b8a8c4b503b7 ... DONE\n",
"2 b1aae587-d26b-4661-8c6c-3072c97855c0 ... DONE\n",
"3 d5b3d313-7281-45d8-bad5-a82da5b126ae ... DONE\n",
"\n",
"[4 rows x 11 columns]"
]
},
"metadata": {},
"execution_count": 5
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Fzk0kx9VSAtm"
},
"source": [
"# IDEA 2: Identify Any Query Associated with moments of high slot utlization\n",
"\n",
"- First, Subquery:\n",
" - find timeslice periods where period_slot_ms/1000 >= 1900\n",
" - make a table of job_id, count_high_seconds\n",
"- return job_id along with\n",
" - job_duration_seconds\n",
" - count_high_seconds\n",
" - the query "
]
},
{
"cell_type": "code",
"metadata": {
"id": "uQ07Md1iPKoQ",
"outputId": "3b082507-8c05-4c9e-8097-218c5be6e69b",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 419
}
},
"source": [
"query = f\"\"\"\n",
"WITH\n",
"spikes as (\n",
" SELECT job_id, count(*) as count_high_seconds \n",
" FROM statmike-project-1.region-us.INFORMATION_SCHEMA.JOBS_TIMELINE_BY_PROJECT\n",
" WHERE period_slot_ms/1000 >= 1900\n",
" GROUP BY job_id\n",
")\n",
"SELECT a.*, TIMESTAMP_DIFF(b.end_time, b.start_time, MILLISECOND)/1000 AS job_duration_seconds, b.query\n",
"FROM spikes AS a\n",
"INNER JOIN statmike-project-1.region-us.INFORMATION_SCHEMA.JOBS_BY_PROJECT AS b\n",
"USING(job_id)\n",
"ORDER BY b.query\n",
"\"\"\"\n",
"result = bq.query(query = query).to_dataframe()\n",
"result"
],
"execution_count": 6,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>job_id</th>\n",
" <th>count_high_seconds</th>\n",
" <th>job_duration_seconds</th>\n",
" <th>query</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>25290aaf-7322-47d2-bd8a-606768e0da8c</td>\n",
" <td>6</td>\n",
" <td>6.245</td>\n",
" <td>None</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>bquxjob_5a906ccd_17a0bd6058c</td>\n",
" <td>2</td>\n",
" <td>36.496</td>\n",
" <td>\\n# DERIVED TABLE - expand to all seconds, nor...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>bquxjob_ea31a51_17a196d9c11</td>\n",
" <td>2</td>\n",
" <td>39.714</td>\n",
" <td>\\n# DERIVED TABLE - expand to all seconds, nor...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>c4f33d44-3b9a-4328-9a9f-b71e07216a7e</td>\n",
" <td>2</td>\n",
" <td>36.630</td>\n",
" <td>\\n# DERIVED TABLE - expand to all seconds, nor...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>2e7f4351-ab12-4eed-95b3-2efbdc77fa3d</td>\n",
" <td>20</td>\n",
" <td>96.078</td>\n",
" <td>\\n# Looking For &lt;My Named Query&gt;\\nWITH jobs as...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>...</th>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>67</th>\n",
" <td>7e60c8cd-7776-47b5-a2cb-3ff4e31cd7c8</td>\n",
" <td>17</td>\n",
" <td>84.396</td>\n",
" <td>SELECT job_id, \\n TIMESTAMP_DIFF(job_end_time...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>68</th>\n",
" <td>426b8317-2903-4fa0-b0d7-e668ff62e039</td>\n",
" <td>1</td>\n",
" <td>19.581</td>\n",
" <td>SELECT job_id, job_type, query, total_slot_ms,...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>69</th>\n",
" <td>6d099a47-0e74-4caa-85fd-bea2810a98df</td>\n",
" <td>24</td>\n",
" <td>88.675</td>\n",
" <td>SELECT period_start, period_slot_ms, state, to...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>70</th>\n",
" <td>5f0b8d80-11c7-49af-82c2-5ac2427042e4</td>\n",
" <td>19</td>\n",
" <td>76.566</td>\n",
" <td>SELECT period_start, period_slot_ms, state, to...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>71</th>\n",
" <td>bquxjob_716e096c_17a0ab8b660</td>\n",
" <td>2</td>\n",
" <td>41.863</td>\n",
" <td>WITH\\nminutes AS (SELECT minutes\\n ...</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>72 rows × 4 columns</p>\n",
"</div>"
],
"text/plain": [
" job_id ... query\n",
"0 25290aaf-7322-47d2-bd8a-606768e0da8c ... None\n",
"1 bquxjob_5a906ccd_17a0bd6058c ... \\n# DERIVED TABLE - expand to all seconds, nor...\n",
"2 bquxjob_ea31a51_17a196d9c11 ... \\n# DERIVED TABLE - expand to all seconds, nor...\n",
"3 c4f33d44-3b9a-4328-9a9f-b71e07216a7e ... \\n# DERIVED TABLE - expand to all seconds, nor...\n",
"4 2e7f4351-ab12-4eed-95b3-2efbdc77fa3d ... \\n# Looking For <My Named Query>\\nWITH jobs as...\n",
".. ... ... ...\n",
"67 7e60c8cd-7776-47b5-a2cb-3ff4e31cd7c8 ... SELECT job_id, \\n TIMESTAMP_DIFF(job_end_time...\n",
"68 426b8317-2903-4fa0-b0d7-e668ff62e039 ... SELECT job_id, job_type, query, total_slot_ms,...\n",
"69 6d099a47-0e74-4caa-85fd-bea2810a98df ... SELECT period_start, period_slot_ms, state, to...\n",
"70 5f0b8d80-11c7-49af-82c2-5ac2427042e4 ... SELECT period_start, period_slot_ms, state, to...\n",
"71 bquxjob_716e096c_17a0ab8b660 ... WITH\\nminutes AS (SELECT minutes\\n ...\n",
"\n",
"[72 rows x 4 columns]"
]
},
"metadata": {},
"execution_count": 6
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "cbRr6PdmbSXJ"
},
"source": [
"Extend the query above to group the results by unique queries and display the queries average duration and the average amount of the duration that was associated with a spike"
]
},
{
"cell_type": "code",
"metadata": {
"id": "2HHUAS3_bRVw",
"outputId": "6e223ff3-5c74-41c6-d511-d5c1ed3422c4",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 419
}
},
"source": [
"query = f\"\"\"\n",
"WITH\n",
"spikes as (\n",
" SELECT job_id, count(*) as count_high_seconds \n",
" FROM statmike-project-1.region-us.INFORMATION_SCHEMA.JOBS_TIMELINE_BY_PROJECT\n",
" WHERE period_slot_ms/1000 >= 1900\n",
" GROUP BY job_id\n",
"),\n",
"jobspikes as (\n",
"SELECT a.*, TIMESTAMP_DIFF(b.end_time, b.start_time, MILLISECOND)/1000 AS job_duration_seconds, b.query\n",
"FROM spikes AS a\n",
"INNER JOIN statmike-project-1.region-us.INFORMATION_SCHEMA.JOBS_BY_PROJECT AS b\n",
"USING(job_id)\n",
")\n",
"SELECT query,\n",
" count(*) as runs_with_spike,\n",
" avg(count_high_seconds) as avg_spike_duration,\n",
" avg(job_duration_seconds) as avg_query_duration\n",
"FROM jobspikes\n",
"GROUP BY query\n",
"ORDER BY avg_spike_duration DESC\n",
"\"\"\"\n",
"result = bq.query(query = query).to_dataframe()\n",
"result"
],
"execution_count": 11,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>query</th>\n",
" <th>runs_with_spike</th>\n",
" <th>avg_spike_duration</th>\n",
" <th>avg_query_duration</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>\\nWITH jobs as (\\n SELECT job_id, project_id,...</td>\n",
" <td>1</td>\n",
" <td>26.0</td>\n",
" <td>131.420</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>SELECT period_start, period_slot_ms, state, to...</td>\n",
" <td>1</td>\n",
" <td>24.0</td>\n",
" <td>88.675</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>SELECT job_id, \\n TIMESTAMP_DIFF(job_end_time...</td>\n",
" <td>1</td>\n",
" <td>22.0</td>\n",
" <td>107.563</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>\\n# Looking For &lt;My Named Query&gt;\\nWITH jobs as...</td>\n",
" <td>1</td>\n",
" <td>20.0</td>\n",
" <td>96.078</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>\\n# Looking For &lt;My Named Query&gt;\\nWITH jobs as...</td>\n",
" <td>2</td>\n",
" <td>19.5</td>\n",
" <td>106.790</td>\n",
" </tr>\n",
" <tr>\n",
" <th>...</th>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>62</th>\n",
" <td>\\n# DERIVED TABLE - expand to all seconds, nor...</td>\n",
" <td>1</td>\n",
" <td>2.0</td>\n",
" <td>36.630</td>\n",
" </tr>\n",
" <tr>\n",
" <th>63</th>\n",
" <td>CREATE OR REPLACE TABLE statmike-project-1.byt...</td>\n",
" <td>1</td>\n",
" <td>2.0</td>\n",
" <td>38.734</td>\n",
" </tr>\n",
" <tr>\n",
" <th>64</th>\n",
" <td>SELECT job_id, job_type, query, total_slot_ms,...</td>\n",
" <td>1</td>\n",
" <td>1.0</td>\n",
" <td>19.581</td>\n",
" </tr>\n",
" <tr>\n",
" <th>65</th>\n",
" <td>SELECT *\\nFROM `statmike-project-1.bytes.F_wik...</td>\n",
" <td>1</td>\n",
" <td>1.0</td>\n",
" <td>30.566</td>\n",
" </tr>\n",
" <tr>\n",
" <th>66</th>\n",
" <td>\\nCREATE OR REPLACE TABLE statmike-project-1.b...</td>\n",
" <td>1</td>\n",
" <td>1.0</td>\n",
" <td>63.046</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>67 rows × 4 columns</p>\n",
"</div>"
],
"text/plain": [
" query ... avg_query_duration\n",
"0 \\nWITH jobs as (\\n SELECT job_id, project_id,... ... 131.420\n",
"1 SELECT period_start, period_slot_ms, state, to... ... 88.675\n",
"2 SELECT job_id, \\n TIMESTAMP_DIFF(job_end_time... ... 107.563\n",
"3 \\n# Looking For <My Named Query>\\nWITH jobs as... ... 96.078\n",
"4 \\n# Looking For <My Named Query>\\nWITH jobs as... ... 106.790\n",
".. ... ... ...\n",
"62 \\n# DERIVED TABLE - expand to all seconds, nor... ... 36.630\n",
"63 CREATE OR REPLACE TABLE statmike-project-1.byt... ... 38.734\n",
"64 SELECT job_id, job_type, query, total_slot_ms,... ... 19.581\n",
"65 SELECT *\\nFROM `statmike-project-1.bytes.F_wik... ... 30.566\n",
"66 \\nCREATE OR REPLACE TABLE statmike-project-1.b... ... 63.046\n",
"\n",
"[67 rows x 4 columns]"
]
},
"metadata": {},
"execution_count": 11
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "_UEa4ESVfCbd"
},
"source": [
""
],
"execution_count": null,
"outputs": []
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment