Skip to content

Instantly share code, notes, and snippets.

@christippett
christippett / deploy_endpoints.sh
Created March 20, 2017 08:55
Automate deployment of Google Cloud Endpoints, including updating App Engine's app.yaml with latest config_id
# This script is designed to be run in a post-commit build script.
# It checks if openapi.yaml has been modified and deploys the latest
# API specification to Google Cloud Endpoints and updates app.yaml
# with the latest config_id
SERVICE_NAME=sample.endpoints.[PROJECT_ID].appspot.com
if $(git diff-tree --no-commit-id --name-only -r HEAD | grep -q openapi.yaml); then
echo "Detected changes to openapi.yaml in last commit, deploying new version to Cloud Endpoints..."
gcloud service-management deploy openapi.yaml
@christippett
christippett / pwpolicy.plist
Last active February 21, 2018 06:13
OSX Password Policy (passes CIS-CAT scan)
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>policyCategoryAuthentication</key>
<array>
<dict>
<key>policyContent</key>
<string>(policyAttributeFailedAuthentications &lt; policyAttributeMaximumFailedAuthentications) OR (policyAttributeCurrentTime &gt; (policyAttributeLastFailedAuthenticationTime + autoEnableInSeconds))</string>
<key>policyIdentifier</key>
@christippett
christippett / slack_file_downloader.py
Created July 11, 2018 06:11
Download files referenced in Slack JSON export
import os
import argparse
import json
import requests
class SlackFileDownloader:
def __init__(self, output_dir, slack_token):
self.output_dir = output_dir
@christippett
christippett / populate_geometry_columns_table.sql
Last active August 26, 2022 19:38
MS SQL stored procedure to populate `geometry_columns` table. This table is used by QGIS to identify tables with spatial data.
-- =============================================
-- Author: Chris Tippett
-- Create date: 2014-08-12
-- Description: Detect columns with geometry datatypes and add them to [dbo].[geometry_columns]
-- =============================================
CREATE PROCEDURE [dbo].[Populate_Geometry_Columns] @schema VARCHAR(MAX) = '', @table VARCHAR(MAX) = ''
AS
BEGIN
SET NOCOUNT ON;
@christippett
christippett / svg2wkt.py
Created July 19, 2018 12:15
Convert online shopping centre maps (in SVG) to well-known text (WKT)
"""
Created on Sun Sep 07 13:11:14 2014
@author: Chris
"""
try:
import xml.etree.cElementTree as ET
except ImportError:
import xml.etree.ElementTree as ET
import csv
@christippett
christippett / auth.py
Created September 13, 2018 05:16
Generate IAP token using Google service account
import google.auth
from google.auth.compute_engine.credentials import \
Credentials as ComputeEngineCredentials
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials as OAuth2Credentials
from google.oauth2.service_account import Credentials, IDTokenCredentials
IAM_SCOPE = 'https://www.googleapis.com/auth/iam'
OAUTH_TOKEN_URI = 'https://www.googleapis.com/oauth2/v4/token'
@christippett
christippett / gcp_utils.py
Created February 5, 2019 07:45
Useful Python functions for Google Cloud Platform
import re
GCS_URL_PATTERN = re.compile(
r'^(?:gs://)'
r'(?P<bucket_name>.+?)'
r'(?:/(?P<object_name>.+?)$|/?$)')
BIGQUERY_TABLE_PATTERN = re.compile(
r'(?:(?P<project_id>.+)\.)?'
r'(?P<dataset_id>.+)\.'
@christippett
christippett / gcs_mover.py
Created February 11, 2019 04:27
Migrate GCS bucket(s) from one project/location to another
import uuid
from google.cloud import storage
from google.cloud.storage import blob, bucket
client = storage.Client()
BUCKETS = [
"bucket-a",
@christippett
christippett / poll_dataflow_status.sh
Last active May 8, 2019 14:31
Shell script to poll successful completion of Dataflow job
#!/bin/sh
JOB_ID=$1
SLEEP_TIME=10
echo "Polling status for Dataflow job: ${JOB_ID}"
check_job_status() {
echo "${JOB_STATUS}" | grep "JOB_STATE_DONE\|JOB_STATE_CANCELLED\|JOB_STATE_FAILED\|JOB_STATE_DRAINED\|^$" > /dev/null
GREP_RETURN_CODE=$?
if [ ${GREP_RETURN_CODE} -eq 0 ]
then
@christippett
christippett / build.sh
Last active September 11, 2023 05:53
Trigger Cloud Builds in a mono-repo
#!/bin/sh -e
IGNORE_FILES=$(ls -p | grep -v /)
TRACKING_BUCKET="$COMMIT_STATE_BUCKET/$REPO_NAME/$BRANCH_NAME"
detect_changed_folders() {
gsutil cp gs://$TRACKING_BUCKET/LAST_COMMIT . &> /dev/null || true
last_commit_sha=`cat LAST_COMMIT 2> /dev/null || git rev-list --max-parents=0 HEAD`
echo "Detecting changes from last build: $last_commit_sha"
folders=`git diff --name-only "$last_commit_sha" | sort -u | awk 'BEGIN {FS="/"} {print $1}' | uniq`
export changed_components=$folders