Skip to content

Instantly share code, notes, and snippets.

package main
import (
"github.com/Jeffail/benthos/lib/stream"
"github.com/Jeffail/benthos/lib/input"
"github.com/Jeffail/benthos/lib/output"
"github.com/Jeffail/benthos/lib/types"
"time"
"os"
"os/signal"
# taken from http://www.piware.de/2011/01/creating-an-https-server-in-python/
# generate server.pem with the following command:
# openssl req -new -x509 -keyout server.pem -out server.pem -days 365 -nodes
# run as follows:
# python simple-https-server.py
# then in your browser, visit:
# https://localhost:8443
import BaseHTTPServer, SimpleHTTPServer
import ssl
@wheresalice
wheresalice / stations.json
Created August 13, 2018 17:03
All UK train stations in geojson format, via https://data.gov.uk/dataset/naptan
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
We can make this file beautiful and searchable if this error is corrected: It looks like row 6 should actually have 15 columns, instead of 13. in line 5.
"AtcoCode","TiplocCode","CrsCode","StationName","StationNameLang","GridType","Easting","Northing","CreationDateTime","ModificationDateTime","RevisionNumber","Modification",SV0000000000,"Latitude","Longitude"
"9100ABGLELE","ABGLELE","AGL","Abergele & Pensarn Rail Station","","U",294612,378681,"2003-11-04T00:00:00","2006-09-18T18:24:34",1,"rev",SH9461278681,53.294571,-3.5826219
"9100ANSDELL","ANSDELL","AFV","Ansdell & Fairhaven Rail Station","","U",334601,427689,"2003-11-04T00:00:00","2009-10-16T14:35:59",3,"rev",SD3460127689,53.741461,-2.9930339
"9100ARCHRAT","ARCHRAT","ART","Arrochar & Tarbet Rail Station","","U",231200,704600,"2003-11-04T00:00:00","2006-09-18T18:24:34",2,"rev",NN3120004600,56.203969,-4.7227699
"9100ASHWELC","ASHWELC","AWM","Ashwell & Morden Rail Station","","U",529770,238615,"2005-04-21T00:00:00","2006-09-18T18:24:34",1,"rev",TL2977038615,52.030770,-0.10978891
"9100BATABAL","BATABAL","BBL","Bat & Ball Rail Station","","U",553100,156800,"2003-11-04T00:00:00","2006-09-18T18:24:34",1,"rev",TQ53
#!/bin/bash -xe
# Parse server.properties to get Zookeeper hosts
ZK_HOST=$(grep 'zookeeper.connect=' /etc/kafka/server.properties | cut -d'=' -f 2)
# Find the most newly created directory of Kafka in the /opt directory
KAFKA_PATH=$(find /opt/ -maxdepth 1 -name kafka-2* -print0 | xargs -0 ls -d -1 -t | head -n1)
# Manually configured list of brokers to rebalance across (zero-indexed)
BROKER_LIST="0,1,2"

Kafka authentication options

SSL certs - fairly standard and easy to implement assuming a root ca is available. Can have issues with certificates being stolen - no checks for revoked certificates. Need to restart clients to update certs. Certs always run out at the wrong time - it always surprises people.

Sasl plain - username and password. Stored in file by default. Sent over plain text by default

Sasl Scramm - avoids passwords being sent as plain text. Passwords are stored in zookeeper.

Sasl gssapi - Kerberos. Can integrate with Active Directory. Passwords or keytabs supported. Lots of tickets for large-scale, can put a lot of pressure on Active Directory servers.

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2" xmlns:gx="http://www.google.com/kml/ext/2.2">
<Document id="feat_1">
<name>Unlabeled</name>
<Placemark id="feat_2">
<name>JW Marriott Hotel Shanghai at Tomorrow Square</name>
<Point id="geom_0">
<coordinates>121.469886,31.230711,0.0</coordinates>
</Point>
</Placemark>
@wheresalice
wheresalice / Facebook.txt
Last active March 27, 2018 20:46
Adblock text file for all known Facebook properties on the internet
||cdninstagram.com
||facebook-web-clients.appspot.com
||facebook.com
||facebook.com.edgekey.net
||facebook.com.edgesuite.net
||facebook.net
||facebook.net.edgekey.net
||fb.com
||fb.me
||fbcdn-profile-a.akamaihd.net
.DEFAULT_GOAL: build
build:
Rscript -e 'library(rmarkdown); rmarkdown::render("dashboard.Rmd", output_file="dashboard.html")'
dep:
Rscript -e 'install.packages("devtools", repos = "https://cloud.r-project.org/")'
Rscript -e 'install.packages(c("ggplot2","prophet","dplyr"), repos = "https://cloud.r-project.org/")'
Rscript -e 'devtools::install_github("rstudio/rmarkdown")'
Rscript -e 'install.packages("rstan", repos = "https://cloud.r-project.org/", dependencies = TRUE)'
Rscript -e 'install.packages(c("Rcpp", "rstan"), type = "source", repos = "https://cloud.r-project.org/")'
diff --git a/cmd/rake.go b/cmd/rake.go
index fcaca52..6b7d80f 100644
--- a/cmd/rake.go
+++ b/cmd/rake.go
@@ -1,6 +1,8 @@
package cmd
import "github.com/skybet/cali"
+import "github.com/docker/go-connections/nat"
+import _ "github.com/pkg/errors"