Skip to content

Instantly share code, notes, and snippets.

Avatar

Thomas Decaux ebuildy

View GitHub Profile
@ebuildy
ebuildy / docker_compose_config_service.bash
Created Jul 29, 2020
docker-compose config, filter service. Show config only for desired service.
View docker_compose_config_service.bash
docker-compose config | docker run -i karlkfi/yq -r ".services.my_service_name"
@ebuildy
ebuildy / instlal_gitlab_runner_playbook.yaml
Created May 11, 2020
Install gitlab-runner with Ansible
View instlal_gitlab_runner_playbook.yaml
---
- hosts: recette
become: yes
become_user: root
vars:
gitlab_url: https://XXXXXX
gitlab_runner_registration_token: XXXXX
tasks:
- name: setup user
user:
@ebuildy
ebuildy / playbook_install_docker_compose.yaml
Created May 11, 2020
Install docker-compose via Ansible
View playbook_install_docker_compose.yaml
---
- hosts: recette
become: yes
become_user: root
tasks:
- name: "Get Linux uname's"
command: "uname -{{ item }}"
register: unames
with_items: ["s", "m"]
- set_fact:
View kibana_create_index_patterns.py
#!/usr/bin/python3
import argparse, json, requests
parser = argparse.ArgumentParser(description='Create automatically index patterns, from elasticsearch indices.')
parser.add_argument('elasticsearch_url',type=str, help='elasticsearch full URL')
parser.add_argument('kibana_url', type=str, help='kibana full URL')
args = parser.parse_args()
View gist:ca963679b3d240120daa1ef1735a1df2
//create properties object
val prop = new java.util.Properties
prop.setProperty("driver", "com.mysql.jdbc.Driver")
prop.setProperty("user", "XXX")
prop.setProperty("password", "XXXX")
//jdbc mysql url - destination database is named "data"
val url = "jdbc:mysql://XXXXX:3306/datahub"
//write data from spark dataframe to database
View gist:a0d244adaf39d5be3f8e1dbacdc5bd30
val df = sqlContext.read.format("es")
.option("es.nodes.wan.only", "true")
.option("es.mapping.date.rich", "false")
.option("es.read.field.exclude", "dimensions.geo_point")
.option("es.nodes", "conso-es:9200")
.load("clicks")
@ebuildy
ebuildy / gist:736ddb7160b587f6405c88c556287a17
Last active Feb 20, 2020
Flatten Apache Spark Data Frame
View gist:736ddb7160b587f6405c88c556287a17
import org.apache.spark.sql.Column
import org.apache.spark.sql.types.StructType
import org.apache.spark.sql.functions.col
def flattenSchema(schema: StructType, prefix: String = null) : Array[Column] = {
schema.fields.flatMap(f => {
val colName = if (prefix == null) f.name else (prefix + "." + f.name)
f.dataType match {
case st: StructType => flattenSchema(st, colName)
View Setup elasticsearch built-in users via Kubernetes job
apiVersion: batch/v1
kind: Job
metadata:
name: {{ stack_name }}-security-setup-users
spec:
template:
spec:
containers:
- name: update-credentials
image: radial/busyboxplus
@ebuildy
ebuildy / gist:3380e4a8e9201dc78301fa311c8f9a02
Created Nov 15, 2019
Elastic packetbeat, log elasticsearch search query took time.
View gist:3380e4a8e9201dc78301fa311c8f9a02
setup:
# elasticsearch index template
template:
enabled: true
overwrite: true
# kibana
dashboards.enabled: true
kibana.host: '${KIBANA_HOST}'
http:
View snakeCaseFormat.java
private static String snakeCaseFormat(String name) {
final StringBuilder result = new StringBuilder();
boolean lastUppercase = false;
for (int i = 0; i < name.length(); i++) {
char ch = name.charAt(i);
char lastEntry = i == 0 ? 'X' : result.charAt(result.length() - 1);
if (ch == ' ' || ch == '_' || ch == '-' || ch == '.') {
lastUppercase = false;
You can’t perform that action at this time.