Skip to content

Instantly share code, notes, and snippets.

@alces
alces / Jenkinsfile
Created Oct 7, 2019
Ask for password in Jenkins pipeline
View Jenkinsfile
pipeline {
agent {
label 'Linux'
}
stages {
stage('ask') {
steps {
script {
def askpass = input(
message: 'Please enter the password',
@alces
alces / delete_old_schemas.py
Created Sep 19, 2019
Remove old schemes from Confluent Schema Registry
View delete_old_schemas.py
#!/usr/bin/env python
# Cleanup old schemas from schema registry
import httplib
import json
import re
DELETE_REGEX = 'IT6[0-6][0-9]-'
con = httplib.HTTPConnection('127.0.0.1', 8081)
@alces
alces / get_compressed_url.py
Created Sep 10, 2019
Fetch compressed content from an URL (e.g., usable with Lenses REST API)
View get_compressed_url.py
#!/usr/bin/env python
# Get compressed URL content
import gzip
import StringIO
import urllib2
req = urllib2.Request('http://my.lenses.url/api/alerts',
headers = {
'Accept-encoding': 'gzip',
@alces
alces / check_srv.py
Created Sep 9, 2019
Check service availability from Python script
View check_srv.py
!/usr/bin/env python
# Check service availability
# Usage: $0 hostname port
import socket
import sys
if len(sys.argv) != 3:
sys.stderr.write("Usage: %s hostname port\n" % sys.argv[0])
sys.exit(2)
@alces
alces / pause_connectors.py
Created Aug 14, 2019
Pause or resume all connectors on a Kafka Connect cluster
View pause_connectors.py
#!/usr/bin/env python
CONNECT_ENDPOINT = 'http://10.0.0.24:8083'
import httplib
import json
import os
import sys
action = 'pause' if os.path.basename(sys.argv[0]).startswith('pause') else 'resume'
@alces
alces / curl_with_client_auth.sh
Created Apr 1, 2019
Provide SSL client authentication via curl command line
View curl_with_client_auth.sh
curl --cacert /path/to/ca-root.crt \
--cert /path/to/client.crt \
--key /path/to/client.key \
https://service.with.client.auth
@alces
alces / UrlConnectTest.java
Last active Oct 9, 2019
Java "one-liner" for testing SSL connections (truststores, client auth, etc)
View UrlConnectTest.java
import java.io.*;
import java.net.*;
/**
* Open URL passed as the first command-line argument and read the first line from it
*
* Example usage for testing client auth:
* java -Djavax.net.ssl.trustStore=/path/to/truststore.jks \
* -Djavax.net.ssl.trustStorePassword=myword \
* -Djavax.net.ssl.keyStore=/path/to/keystore.jks \
@alces
alces / kafka-jdbc-src-reset.sh
Created Jan 29, 2019
Reset value of incrementing field for Kafka JDBC source connector
View kafka-jdbc-src-reset.sh
#!/bin/bash
echo '["my-connector-name",{"protocol":"1","table":"my_db_name.my_table_name"}]@{"incrementing":5}' | kafka-console-producer --topic _connect-offsets --broker-list 127.0.0.1:9092 --property parse.key=true --property key.separator=@
# And after issuing this command, restart the connector's task (pausing/resuming the connector isn't enoght)
@alces
alces / words_count.pig
Created Nov 30, 2018
Count words' occurrences in a text file
View words_count.pig
data = load '$file' using TextLoader();
tokens = foreach data generate FLATTEN(TOKENIZE($0));
words = filter tokens by $0 MATCHES '[A-Za-z]+';
lowers = foreach words generate LOWER($0);
groups = group lowers by $0;
counts = foreach groups generate group, COUNT(lowers.$0);
by_count = order counts by $1 DESC;
store by_count into '$file-by-count';
by_word = order counts by $0 ASC;
store by_word into '$file-by-word';
@alces
alces / jenkinsfile_pr_check.md
Created Sep 26, 2018
Using Jenkins pipelines as Github pull request cheks
View jenkinsfile_pr_check.md
  1. in Jenkins set up a multibranch pipeline with Branch source of type Github (under it, set up endpoint, credentials, repo name, etc.);
  2. in Github go to the repository Settings and add the user chosen on the previous step to the repository's colaborators;
  3. go to the Hooks menu and add a webhook pointing to <your-jenkins-host>/github-webhook/ and select pull request event under Let me select individual events option;
  4. create a pull request - after that Jenkins should automatically start a build;
  5. go to Branches menu under Settings and add the target branch to Protected branches;
  6. choose Require status checks to pass before merging and continuous-integration/jenkins/pr-merge under it
  7. commit a change into the pull request and see the Jenkins build result on the page.