Skip to content

Instantly share code, notes, and snippets.

@alces
alces / UrlConnectTest.java
Last active October 9, 2019 10:13
Java "one-liner" for testing SSL connections (truststores, client auth, etc)
import java.io.*;
import java.net.*;
/**
* Open URL passed as the first command-line argument and read the first line from it
*
* Example usage for testing client auth:
* java -Djavax.net.ssl.trustStore=/path/to/truststore.jks \
* -Djavax.net.ssl.trustStorePassword=myword \
* -Djavax.net.ssl.keyStore=/path/to/keystore.jks \
@alces
alces / kafka-jdbc-src-reset.sh
Created January 29, 2019 07:31
Reset value of incrementing field for Kafka JDBC source connector
#!/bin/bash
echo '["my-connector-name",{"protocol":"1","table":"my_db_name.my_table_name"}]@{"incrementing":5}' | kafka-console-producer --topic _connect-offsets --broker-list 127.0.0.1:9092 --property parse.key=true --property key.separator=@
# And after issuing this command, restart the connector's task (pausing/resuming the connector isn't enoght)
@alces
alces / words_count.pig
Created November 30, 2018 06:08
Count words' occurrences in a text file
data = load '$file' using TextLoader();
tokens = foreach data generate FLATTEN(TOKENIZE($0));
words = filter tokens by $0 MATCHES '[A-Za-z]+';
lowers = foreach words generate LOWER($0);
groups = group lowers by $0;
counts = foreach groups generate group, COUNT(lowers.$0);
by_count = order counts by $1 DESC;
store by_count into '$file-by-count';
by_word = order counts by $0 ASC;
store by_word into '$file-by-word';
@alces
alces / jenkinsfile_pr_check.md
Created September 26, 2018 06:35
Using Jenkins pipelines as Github pull request cheks
  1. in Jenkins set up a multibranch pipeline with Branch source of type Github (under it, set up endpoint, credentials, repo name, etc.);
  2. in Github go to the repository Settings and add the user chosen on the previous step to the repository's colaborators;
  3. go to the Hooks menu and add a webhook pointing to <your-jenkins-host>/github-webhook/ and select pull request event under Let me select individual events option;
  4. create a pull request - after that Jenkins should automatically start a build;
  5. go to Branches menu under Settings and add the target branch to Protected branches;
  6. choose Require status checks to pass before merging and continuous-integration/jenkins/pr-merge under it
  7. commit a change into the pull request and see the Jenkins build result on the page.
@alces
alces / wrest_dict.py
Created April 4, 2018 11:47
Wrest a dictionary of lists (e.g., turn a dict of nodes as keys and lists of groups as values into a dict of groups as keys and lists of nodes as values)
def wrest_dict(orig):
pairs = [(g, n) for n in orig.keys() for g in orig[n]]
nodes4group = lamdba g: sorted(n[1] for n in pairs if n[0] == g)
return {g[0]: nodes4group(g[0]) for g in pairs}
@alces
alces / list_reduce.go
Created November 29, 2017 11:26
Adding Reduce method to List class
package main
import "container/list"
type MyList struct {
*list.List
}
func (l *MyList) Reduce(f func(interface{}, interface{}) interface{}) interface{} {
s := l.Front()
@alces
alces / list_map.go
Last active November 27, 2017 14:12
Adding Map method to List class
package main
import "container/list"
type MyList struct {
*list.List
}
func (l *MyList) Map(f func(interface{}) interface{}) *MyList {
nl := &MyList {list.New()}
@alces
alces / list_filter.go
Last active November 27, 2017 14:12
Adding Filter method to List
package main
import "container/list"
type MyList struct {
*list.List
}
func (l *MyList) Filter(f func(interface{}) bool) *MyList {
nl := &MyList{list.New()}
@alces
alces / terraform_existing_instances.py
Created November 24, 2017 11:55
return a list of google compute instances existing before a current run of terraform
import json
data = json.load(open('terraform.tfstate'))
all_res = [mod['resources'].values() for mod in data['modules']]
res_list = reduce(lambda x, y: x + y, all_res)
disk_attrs = [r['primary']['attributes'] for r in res_list if r['type'] == 'google_compute_disk']
print [a['users.0'].split('/')[-1] for a in disk_attrs if 'users.0' in a]
@alces
alces / build.gradle
Created July 17, 2017 12:19
Download dependencies into current directory and remove versions from filenames
apply plugin: 'java'
defaultTasks 'download'
dependencies {
runtime 'org.yaml:snakeyaml:1.18', 'org.mongodb:mongo-java-driver:3.4.2'
}
repositories {
mavenCentral()
}