Skip to content

Instantly share code, notes, and snippets.

@WouterNieuwerth
WouterNieuwerth / google_workflow_with_retry.jaml
Last active January 11, 2024 09:08
Google Workflow with Retry
main:
params: [event]
steps:
- init:
assign:
- parent: projects/[your-project]/locations/europe-west4/repositories/[your-repo]
- extract_tableId_step1: '${json.decode(base64.decode(event.data.message.data))}'
- extract_tableId_step2: '${extract_tableId_step1.protoPayload.metadata.tableCreation.table.tableName}'
- extract_tableId_step3: '${text.split(extract_tableId_step2, "/")}'
- extract_tableId_step4: '${extract_tableId_step3[len(extract_tableId_step3) -1]}'
@WouterNieuwerth
WouterNieuwerth / ga4_events.sqlx
Last active August 26, 2023 09:33
Datafrom config declaration
config {
type: "declaration",
database: "your-database",
schema: "analytics_123456789",
name: dataform.projectConfig.vars.GA4_TABLE,
}
@WouterNieuwerth
WouterNieuwerth / example.sql
Last active August 24, 2023 18:59
Example SQL with incremental tables
config {
type: "incremental",
description: "Counts the amount of specific events per session per day.",
schema: "monitoring",
tags: ["ga4"],
bigquery: {
partitionBy: "event_date",
}
}
@WouterNieuwerth
WouterNieuwerth / dataform.json
Created August 24, 2023 18:39
GA4_TABLE variable for dataform.json
{
"defaultSchema": "dataform",
"assertionSchema": "dataform_assertions",
"warehouse": "bigquery",
"defaultDatabase": "your-database",
"defaultLocation": "EU",
"vars": {
"GA4_TABLE": "events_*"
}
}
main:
params: [event]
steps:
- init:
assign:
- parent: projects/[your-project]/locations/europe-west4/repositories/[your-repo]
- extract_tableId_step1: '${json.decode(base64.decode(event.data.message.data))}'
- extract_tableId_step2: '${extract_tableId_step1.protoPayload.metadata.tableCreation.table.tableName}'
- extract_tableId_step3: '${text.split(extract_tableId_step2, "/")}'
- extract_tableId_step4: '${extract_tableId_step3[len(extract_tableId_step3) -1]}'
@WouterNieuwerth
WouterNieuwerth / GPX_analysis_step_complete.py
Last active June 28, 2023 20:55
GPX analysis - complete script
import gpxpy
import matplotlib.pyplot as plt
import datetime
from geopy import distance
from math import sqrt, floor
import numpy as np
import pandas as pd
import haversine
/**
* Loads the content of a Google Drive Spreadsheet into BigQuery
* Based on:
* - https://www.lunametrics.com/blog/2017/07/26/connect-google-analytics-data-tools-via-bigquery/
* - https://developers.google.com/apps-script/advanced/bigquery
* Edit by Wouter Nieuwerth, w.nieuwerth@adwise.nl
* https://www.adwise.nl
*/
function loadSpreadsheet() {
// An example tag template that makes a request to OpenWeatherMap.org
// and sends data about the weather to both GA4 and Universal Analytics.
const getAllEventData = require('getAllEventData');
const getEventData = require('getEventData');
const JSON = require('JSON');
const sendEventToGoogleAnalytics = require('sendEventToGoogleAnalytics');
const sendHttpRequest = require('sendHttpRequest');
const client_id = getEventData('client_id');
// Send data to Universal Analytics
const getEventData = require('getEventData');
const sendEventToGoogleAnalytics = require('sendEventToGoogleAnalytics');
const client_id = getEventData('client_id');
const events = {
'x-ga-measurement_id': 'UA-XXXXXX-1', // Replace with your own GA UA ID
'v': 1,
't': 'event',
// Send data to GA4
const getAllEventData = require('getAllEventData');
const sendEventToGoogleAnalytics = require('sendEventToGoogleAnalytics');
const events = getAllEventData();
// Add extra fields to the events object:
events.firstname = 'Wouter';
sendEventToGoogleAnalytics(events, (response) => {