Skip to content

Instantly share code, notes, and snippets.

View danielschulz's full-sized avatar

Daniel Schulz danielschulz

View GitHub Profile
@tigerhawkvok
tigerhawkvok / poetry-convert.py
Last active April 19, 2024 23:31
Convert a requirements.txt file to a Poetry project
#!python3
"""
Convert a requirements.txt file to a Poetry project.
Just place in the root of your working directory and run!
"""
sourceFile = "./requirements.txt"
import re
import os
@tallclair
tallclair / kubectl-cp.sh
Last active January 18, 2024 22:19
kubectl cp equivalents
# Copy /tmp/foo_dir local directory to /tmp/bar_dir in a remote pod in the default namespace
kubectl cp /tmp/foo_dir <some-pod>:/tmp/foo_dir
tar cf - /tmp/foo_dir | kubectl exec -i <some-pod> -- tar xf -
# Copy /tmp/foo local file to /tmp/bar in a remote pod in a specific container
kubectl cp /tmp/foo <some-pod>:/tmp/foo -c <specific-container>
tar cf - /tmp/foo | kubectl exec -i <some-pod> -c <specific-container> -- tar xf -
# Copy /tmp/foo local file to /tmp/bar in a remote pod in namespace <some-namespace>
kubectl cp /tmp/foo <some-namespace>/<some-pod>:/tmp/foo
@syaffers
syaffers / tes-names-dataset-2.py
Created May 15, 2019 07:19
The second iteration of the TES names dataset with major additions and updates
import os
from sklearn.preprocessing import LabelEncoder
from torch.utils.data import Dataset
import torch
class TESNamesDataset(Dataset):
def __init__(self, data_root, charset):
self.data_root = data_root
self.charset = charset
self.samples = []
@mcastelino
mcastelino / kcgroups.md
Last active April 12, 2024 08:58
Kubernetes and cgroups Resource Management/Static cpuManagerPolicy/Memory and Resource Isolation & Scheduling

Overview

The goal of this document to cover all aspects of Kubernetes management, including how resources are expressed, constrained and accounted for. This started a way to ensure that alternate container runtime implementation like Kata containers will behave from a resource accounting and consumption point of view in the same manner as runc.

Location of the latest version of this document: https://gist.github.com/mcastelino/b8ce9a70b00ee56036dadd70ded53e9f

If you do not understand cgroups please refer to a quick primer at the bottom of this document. This will help you understand how the resource enforcement actually works.

Kubernetes Resource Management

@fonylew
fonylew / install-local-server.sh
Last active August 31, 2023 12:50
A bunch of script to install Nvidia with CUDA10, CUDNN 7.4 and so on.
#!/bin/bash
# First sudo command
sudo whoami
# Update and upgrade
sudo apt update
sudo apt upgrade -y
# Utility
@colanconnon
colanconnon / Dockerfile
Created December 26, 2018 11:38
Apache Livy docker
FROM continuumio/miniconda3
ENV APACHE_SPARK_VERSION 2.3.1
ENV HADOOP_VERSION 2.7
RUN apt-get -y update && \
apt-get install --no-install-recommends -y openjdk-8-jre-headless ca-certificates-java && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
@hslatman
hslatman / fluent-filebeat-comparison.md
Created November 7, 2018 09:27 — forked from StevenACoffman/fluent-filebeat-comparison.md
Fluentd Fluent-bit FileBeat memory and cpu resources

Fluent-bit rocks

A short survey of log collection options and why you picked the wrong one. 😜

Who am I? Where am I from?

I'm Steve Coffman and I work at Ithaka. We do JStor (academic journals) and other stuff. How big is it?

Number what it means
101,332,633 unique visitors in 2017
@denji
denji / qcow2vdi.md
Last active March 8, 2024 04:15 — forked from mamonu/qcow2vdi.sh
convert a qcow2 vm to a VirtualBox vm format

VirtualBox command-line interface (VBoxManage) provides an easy way to convert raw disk image to the VDI/VMDK format and otherwise.

Let's assume that we have raw image of the sdb device:

$ sudo dd if=/dev/sdb of=./sdb.raw

To use it with VirtualBox we need to convert it to the VDI format:

$ VBoxManage convertdd sdb.raw sdb.vdi --format VDI
@anttu
anttu / secretsmanager.tf
Created May 4, 2018 16:03
Terraform AWS Secrets Manager example with key and value
resource "aws_secretsmanager_secret" "IRCSecrets" {
name = "irc/client/credentials"
description = "My IRC client credentials"
}
resource "aws_secretsmanager_secret_version" "IRCCredentials" {
secret_id = "${aws_secretsmanager_secret.IRCSecrets.id}"
secret_string = "{\"username\":\"AzureDiamond\",\"password\":\"hunter2\"}"
}