Skip to content

Instantly share code, notes, and snippets.

View minyk's full-sized avatar
😀
Containerize! all the things!

Drake Youngkun Min minyk

😀
Containerize! all the things!
View GitHub Profile
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@slenky
slenky / stagepage.js
Created January 13, 2021 11:24
Fixing the SparkUI + Jupyter Server Proxy
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
@slenky
slenky / utils.json
Created January 12, 2021 13:09
Fix jupyter-server-proxy for Spark UI + Kubeflow
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*

The following assumes you are using Minikube and Trow for container registry, installed using the quick install method documented here

Pull the following containers locally (docker pull):

For Kudo:

  1. kudobuilder/controller:v0.9.0

From params.yaml:

  1. mesosphere/cassandra-prometheus-exporter:2.2.1-0.1.1
  2. mesosphere/cassandra:3.11.5-0.1.1
@sticilface
sticilface / PROGMEM.md
Last active July 23, 2023 11:38
PROGMEM

Guide to PROGMEM on ESP8266 and Arduino IDE

Intro

On low memory devices like the arduino and esp8266 you do not want strings to be stored in RAM. This occurs by default on these systems. Declare a string const char * xyz = "this is a string" and it will use up RAM.

The solution on these devices is to allow strings to be stored in read only memory, in Arduino this is the PROGMEM macro. Most of my experience is with the ESP8266 which is a 32bit micros controller. This device stores PROGMEM data in flash. The macro PROGMEM on ESP8266 is simply

#define PROGMEM   ICACHE_RODATA_ATTR
@nlsun
nlsun / Repoxy.md
Last active April 17, 2019 04:00

Repoxy

How to Configure

Copy this snippet and write it to a file:

{
  "id": "/repoxy",
  "cpus": 1,
  "acceptedResourceRoles": [
# systemd unit file
# place in /etc/systemd/system
# systemctl enable gotty.service
# systemctl start gotty.service
[Unit]
Description=Gotty Web Terminal
After=network.target
[Service]
@tam7t
tam7t / vault-statsd-mapping.conf
Last active November 5, 2020 13:45
Example vault statsd prometheus exporter mapping
vault.barrier.*
name="vault_barrier"
method="$1"
vault.consul.*
name="vault_consul"
method="$1"
vault.route.*.*
name="vault_route"
@tobilg
tobilg / custom_s3_endpoint_in_spark.md
Last active May 22, 2023 14:57
Description on how to use a custom S3 endpoint (like Rados Gateway for Ceph)

Custom S3 endpoints with Spark

To be able to use custom endpoints with the latest Spark distribution, one needs to add an external package (hadoop-aws). Then, custum endpoints can be configured according to docs.

Use the hadoop-aws package

bin/spark-shell --packages org.apache.hadoop:hadoop-aws:2.7.2

SparkContext configuration

@shreyaskarnik
shreyaskarnik / Instructions.md
Last active March 24, 2023 15:35
Route Docker Logs to ELK Stack
  • With Docker 1.8.0 shipped new log-driver for GELF via UDP, this means that the logs from Docker Container(s) can be shipped directly to the ELK stack for further analysis.
  • This tutorial will illustrate how to use the GELF log-driver with Docker engine.
  • Step 1: Setup ELK Stack:
    • docker run -d --name es elasticsearch
    • docker run -d --name logstash --link es:elasticsearch logstash -v /tmp/logstash.conf:/config-dir/logstash.conf logstash logstash -f /config-dir/logstash.conf
    • Note the config for Logstash can be found at this link
    • docker run --link es:elasticsearch -d kibana
  • Once the ELK stack is up now let's fire up our nginx container which ships its logs to ELK stack.
  • LOGSTASH_ADDRESS=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' logstash)
  • `docker run -d --net=host --log-driver=gelf --log-opt gelf-address=u