Skip to content

Instantly share code, notes, and snippets.

View tomncooper's full-sized avatar
🏠
Working from home

Thomas Cooper tomncooper

🏠
Working from home
View GitHub Profile
@tomncooper
tomncooper / ksql-cli.sh
Created April 2, 2020 16:25
Basic deployment file and CLI script for running ksqlDB in a Kubernetes cluster
#!/usr/bin/env bash
kubectl -n ksqldb run ksqldb-cli-$1 -ti \
--image=confluentinc/ksqldb-cli:0.8.1 \
--rm=true --restart=Never \
-- /usr/bin/ksql http://ksqldb-server:8088

Keybase proof

I hereby claim:

  • I am tomncooper on github.
  • I am tomcooper (https://keybase.io/tomcooper) on keybase.
  • I have a public key ASAeVdDQ9R_c4VRJ-ygCRTMtxvy19Dri7Ulqz-Iu0J2glwo

To claim this, I am signing this object:

@tomncooper
tomncooper / petitition_signatures.py
Last active March 25, 2019 10:25
Script to extract signature data from the UK Government Petitions website (Python 3.4+)
import json
import os
import datetime as dt
from urllib.request import urlopen
from csv import DictWriter
from argparse import ArgumentParser
@tomncooper
tomncooper / tesco.py
Last active January 20, 2019 12:15
Methods for extracting nutritional information from the Tesco website
""" Methods for scraping product information from the tesco website.
You will need to install requests, beautifulsoup4, lxml and pandas libraries
$ pip install requests beautifulsoup4 lxml pandas
"""
from typing import List, Optional
import requests
import pandas as pd
@tomncooper
tomncooper / path_analysis.py
Created July 21, 2018 13:21
script for comparing graph traversal path discovery to just inferring the paths by combining all instances of components into paths.
import csv
from gremlin_python.process.graph_traversal import out, has
def find_all_paths(gremlin_client, source_comp, sink_comp):
source_vertices = (gremlin_client.graph_traversal.V()
.hasLabel("instance")
.has("component", source_comp).toList())
sink_task_ids = (gremlin_client.graph_traversal.V()
@tomncooper
tomncooper / toy_graph.py
Created July 21, 2018 13:19
Creates a toy topology graph using gremlin. Set the number of instances for each component by supplying an integer as the 1st argument when you call the script.
import sys
from tests.local.graph_connect import graph_client
g = graph_client.graph_traversal
parallelism = int(sys.argv[1])
components = {"Sp" : parallelism, "A" : parallelism, "B" : parallelism,
"C" : parallelism}
@tomncooper
tomncooper / cofid_groups_toy_example.R
Created March 8, 2018 14:16
Toy examle of using agrep to match food groups for participant food choices
library(tidyr)
# User data dataframe
user.foods = rbind("milk", "apples", "bread", "ice cream", "boxed fruits")
ID = rbind(1, 2, 3, 4, 5)
user.data = cbind(ID, user.foods)
colnames(user.data) = c("ID", "User.Food")
user.data = as.data.frame(user.data)
# Food Groups dataframe
@tomncooper
tomncooper / decs.py
Created October 2, 2017 15:33
Decorator examples
class after5(object):
def __init__(self, func):
self.func = func
self.counter = 0
def __call__(self):
if self.counter > 4:
self.func()