Skip to content

Instantly share code, notes, and snippets.

View Limess's full-sized avatar
🍋

Charlie Briggs Limess

🍋
View GitHub Profile
@gwhitelaw
gwhitelaw / aws-glue-zepplin.md
Last active January 26, 2022 01:39
Easily connect to an AWS Glue Dev endpoint

This is how I quickly got an Apache Zepplin notebook running against the AWS Glue Dev endpoint. None of the guides out there seemed concise, and I found some custom Docker containers doing what you can do easily. This gives you the power - it sets up port forwarding & runs the official Docker image.

  1. Create your Glue Dev endpoint (this involves creating a keypair, I just used ssh-keygen)
  2. Once READY, select it and copy the "SSH tunnel to remote interpreter"
  • eg: ssh -i <private-key.pem> -vnNT -L :9007:169.254.76.1:9007 glue@..compute.amazonaws.com
  1. Connect to the endpoint in a terminal session, modifying the above to match: ssh -i ~/.ssh/glue-dev -vnNT -L :9007:*127.0.0.1*:9007 glue@<ec2-endpoint>.<region>.compute.amazonaws.com
  2. Run the Apache Zepplin Docker container `docker run -p 8080:8080 --rm -v $PWD/logs:/logs -v $PWD/notebook:/notebook -e ZEPPELIN_LOG_DIR='/l
@HarlanH
HarlanH / run_dbt.sh
Created September 6, 2018 20:08
dbt full-refresh flag
#!/usr/bin/env bash
# When run, first tries to figure out if the full-refresh flag is set.
# Then, runs dbt with or without the flag.
# Do not run me in dev mode!
full_refresh_needed="$(
psql $REDSHIFT_URI -t <<EOF
SELECT full_refresh
@zelark
zelark / pg_test.types.clj
Last active February 2, 2024 19:47
Support json and jsonb Postgres types in Clojure.
;; For supporting more PG types, see https://github.com/remodoy/clj-postgresql
(ns pg-test.types
(:require [cheshire.core :as json]
[clojure.java.jdbc :as jdbc])
(:import [org.postgresql.util PGobject]
[java.sql PreparedStatement]))
;; Writing
(defn- to-pg-json [data json-type]
@noelbundick
noelbundick / LICENSE
Last active April 11, 2024 16:12
Exclude WSL installations from Windows Defender realtime protection
MIT License
Copyright (c) 2018 Noel Bundick
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
@gpeal
gpeal / build.gradle
Created May 22, 2018 02:01
Airbnb Gradle Flavors
...
apply from: './flavors.gradle'
...
android {
buildTypes {
productFlavors {
project.flavors.each { flavor, config ->
"$flavor" {
dimension 'scope'
if (flavor != 'full') {
@HyperBrain
HyperBrain / lifecycle-cheat-sheet.md
Last active March 20, 2024 00:17
Serverless Lifecycle Cheat Sheet

Serverless plugin author's cheat sheet

This cheat sheet provides a detailed overview of the exposed lifecycle events and available commands (and entrypoints) of the Serverless framework, that can be hooked by plugins (internal and external ones). The document is structured by the commands invoked by the user.

Lifecycle events are shown as the globally available outer events (all providers) and sub lifecycle events that are provider specific in the called order. Currently only the AWS provider is shown. If you have information about the other provider,

@jonico
jonico / Jenkinsfile
Last active January 31, 2024 09:43
Example for a full blown Jenkins pipeline script with CodeQL analysis steps, multiple stages, Kubernetes templates, shared volumes, input steps, injected credentials, heroku deploy, sonarqube and artifactory integration, Docker containers, multiple Git commit statuses, PR merge vs branch build detection, REST API calls to GitHub deployment API, …
#!groovy
import groovy.json.JsonOutput
import groovy.json.JsonSlurper
def label = "mypod-${UUID.randomUUID().toString()}"
podTemplate(label: label, yaml: """
spec:
containers:
- name: mvn
image: maven:3.3.9-jdk-8
cd /Library/Preferences
sudo rm com.sophos.sav.plist

cd /Library/Application\ Support/Sophos/cloud/Installer.app/Contents/MacOS/tools/
sudo ./InstallationDeployer —force_remove
(ns background
(:require [clojure.tools.logging :as log])
(:import [java.util.concurrent]))
(defonce !executor (delay (java.util.concurrent.Executors/newCachedThreadPool)))
(defn background
"Calls the fn passed in a thread from a thread pool, returning immediately.
Unlike future, background does not swallow exceptions."
[f]
@mholt
mholt / transcript
Created February 26, 2016 18:42
Is it necessary to consume response body before closing it (net/http client code)?
mholt [9:10 AM]
When using http.Get(), is it really necessary to read the full response body just to close it later?
[9:10]
The docs keep saying `Caller should close resp.Body when done reading from it.` and I keep seeing code like this:
```
io.Copy(ioutil.Discard, resp.Body)
resp.Body.Close()
```