Skip to content

Instantly share code, notes, and snippets.

View emtwo's full-sized avatar

Marina Samuel emtwo

  • Wealthsimple
  • Toronto
View GitHub Profile
~/Projects/mozilla/redash>:docker-compose build --pull
redis uses an image, skipping
postgres uses an image, skipping
flower uses an image, skipping
Building server
Step 1/18 : FROM node:10 as frontend-builder
10: Pulling from library/node
Digest: sha256:bdc6d102e926b70690ce0cc0b077d450b1b231524a69b874912a9b337c719e6e
Status: Image is up to date for node:10
---> d9b29c7fd8c0
SELECT
client_id,
os,
app_version,
app_build_id,
channel,
metric,
metric_type,
key,
agg_type,
--------------------------------------------------------------------------------
Starting attempt 7 of 8
--------------------------------------------------------------------------------
[2019-09-17 19:53:47,831] {models.py:1593} INFO - Executing <Task(GKEPodOperator): clients_daily_histogram_aggregates> on 2019-09-16T00:00:00+00:00
[2019-09-17 19:53:47,832] {base_task_runner.py:118} INFO - Running: ['bash', '-c', u'airflow run mdv2 clients_daily_histogram_aggregates 2019-09-16T00:00:00+00:00 --job_id 17 --raw -sd DAGS_FOLDER/mdv2.py --cfg_path /tmp/tmpvNVFon']
[2019-09-17 19:53:48,599] {base_task_runner.py:101} INFO - Job 17: Subtask clients_daily_histogram_aggregates [2019-09-17 19:53:48,599] {settings.py:174} INFO - settings.configure_orm(): Using pool settings. pool_size=5, pool_recycle=3600, pid=661
[2019-09-17 19:53:49,155] {base_task_runner.py:101} INFO - Job 17: Subtask clients_daily_histogram_aggregates [2019-09-17 19:53:49,155] {default_celery.py:90} WARNING - You have configured a result_backend of
2019-09-16 18:37:33,194 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f34f589ae90>: Failed to establish a new connection: [Errno 111] Connection refused',)': /api/v1/namespaces/default/pods
[2019-09-16 18:37:33,194] {connectionpool.py:665} WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f34f589ae90>: Failed to establish a new connection: [Errno 111] Connection refused',)': /api/v1/namespaces/default/pods
ERROR: (gcloud.container.clusters.get-credentials) ResponseError: code=403, message=Required "container.clusters.get" permission(s) for "projects/emtwo-252813/zones/us-central1-a/clusters/bq-load-gke-1". See https://cloud.google.com/kubernetes-engine/docs/troubleshooting#gke_service_account_deleted for more info.
[2019-09-13 14:15:30,352] {models.py:1788} ERROR - Command '['gcloud', 'container', 'clusters', 'get-credentials', u'bq-load-gke-1', '--zone', u'us-central1-a', '--project', u'emtwo-252813']' returned non-zero exit status 1
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/airflow/models.py", line 1657, in _run_raw_task
result = task_copy.execute(context=context)
File "/app/dags/operators/gcp_container_operator.py", line 52, in execute
"--project", self.project_id])
File "/usr/local/lib/python2.7/subprocess.py", line 190, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['gcloud', 'container', 'clusters', 'get-credential
CREATE OR REPLACE VIEW
`moz-fx-data-shared-prod.telemetry.client_probe_counts`
AS SELECT
*
FROM
`moz-fx-data-shared-prod.telemetry_derived.client_probe_counts_v1`
WHERE metric != "search_counts"
AND metric NOT LIKE "%browser_search%"
-- Query generated by: sql/clients_daily_scalar_aggregates.sql.py
CREATE TEMP FUNCTION
udf_aggregate_map_sum(maps ANY TYPE) AS (STRUCT(ARRAY(
SELECT
AS STRUCT key,
SUM(value) AS value
FROM
UNNEST(maps),
UNNEST(key_value)
GROUP BY
-- Query generated by: sql/clients_daily_scalar_aggregates.sql.py
CREATE TEMP FUNCTION
udf_aggregate_map_sum(maps ANY TYPE) AS (STRUCT(ARRAY(
SELECT
AS STRUCT key,
SUM(value) AS value
FROM
UNNEST(maps),
UNNEST(key_value)
GROUP BY
* default.clients_daily_v6
* default.cliqz_testpilot
* default.containers_shield_v1
* default.containers_testpilottest
* default.containers_testpilottest_v2
* default.core_client_count_v2018061520181215
* default.core_client_count_v2018080920190209
* default.core_client_count_v2018090220190302
* default.core_client_count_v2018092320190323
* default.core_client_count_v2018093020190330
* default.android_events_v1
* default.churn_v2
* default.churn_v3
* default.containers_testpilottest
* default.core_client_count_v2018012120180721
* default.core_client_count_v2018012820180728
* default.core_client_count_v2018060820181208
* default.core_client_count_v2018092320190323
* default.core_client_count_v2018093020190330
* default.cross_sectional_v20180623