Skip to content

Instantly share code, notes, and snippets.

View Dentrax's full-sized avatar
🎶
GNW's Not Wololooo!

Furkan Türkal Dentrax

🎶
GNW's Not Wololooo!
View GitHub Profile
@Dentrax
Dentrax / instagram-collection-migrate.py
Created July 24, 2023 06:29
Move Instagram Collections to Another
from instagrapi import Client
from instagrapi.exceptions import ClientLoginRequired, ClientError
import os
# Instagram accounts credentials
acc1_username = os.environ.get('IM_FROM_USERNAME')
acc1_password = os.environ.get('IM_FROM_PASSWORD')
acc2_username = os.environ.get('IM_TO_USERNAME')
acc2_password = os.environ.get('IM_TO_PASSWORD')
@Dentrax
Dentrax / go-benchstat.sh
Created July 24, 2023 06:26
Go Benchstat Diff
#!/bin/bash
set -e
set -u
set -o pipefail
command -v benchstat >/dev/null 2>&1 || { echo >&2 "I need Benchstat!"; exit 1; }
command -v git >/dev/null 2>&1 || { echo >&2 "I need Git!"; exit 1; }
BRANCH_TARGET="master"
@Dentrax
Dentrax / config
Created May 30, 2023 06:32
~/.ssh/config for GitHub (macOS)
Host github.com
User git
Hostname github.com
AddKeysToAgent yes
IgnoreUnknown UseKeychain
UseKeychain yes
PreferredAuthentications publickey
IdentityFile /Users/USERNAME/.ssh/id_rsa
@Dentrax
Dentrax / etcd-check.md
Last active June 2, 2023 12:06
ETCD - Interact in Kubernetes VM
  1. Export all the required environment variables:
export $(grep -v '^#' /etc/etcd.env | xargs -d '\n')
  1. Start interacting with etcdctl
etcdctl endpoint health status --cluster -w table && etcdctl endpoint status --cluster -w table
@Dentrax
Dentrax / containerd-gc-analysis.md
Last active November 15, 2023 11:49
Containerd GC Analysis

In containerd, there is actually a garbage collector which can be found here: https://github.com/containerd/containerd/blob/master/docs/garbage-collection.md. In the cleanup phase, only objects that are not associated (i.e. have no image reference) are removed - those marked as "dirty" are kept. To clean up unused images and running/stopped containers, this can be used.

While not yet production-ready, the tool at https://github.com/Azure/eraser could be used to achieve this. However, it may be difficult and complex to run this on all nodes. Descheduler cannot solve this problem as it does not run as a daemonset, but kubelet garbage collection can be used instead (checking if it is enabled in the current configs): https://kubernetes.io/docs/concepts/architecture/garbage-collection/#containers-images.

It seems that containerd does not support log rotation. I found a solution that involves using kubelet (as described in containerd/containerd#3351 (comment), also pr: https

@Dentrax
Dentrax / etcd-defragmentation.md
Last active December 13, 2022 07:55
How etcd defragmentation works?

Abstract

Bolt operations are copy-on-write. When a page is updated, it is copied to a completely new page. The old page is added to a "freelist", which Bolt refers to when it needs a new page. This means that deleting large amounts of data will not actually free up space on disk, as the pages are instead kept on Bolt's freelist for future use. In order to free up this space to disk, you will need to perform a defrag.

The process of defragmentation releases this storage space back to the file system. Defragmentation is issued on a per-member so that cluster-wide latency spikes may be avoided.

Algorithm

  1. lock batchTx to ensure nobody is using previous tx, and then close previous ongoing tx.
  2. lock database after lock tx to avoid deadlock.
@Dentrax
Dentrax / oidc-flow-study-notes.md
Last active November 29, 2022 11:50
Kubernetes OIDC - IDToken
@Dentrax
Dentrax / stargz.index.json.md
Last active June 5, 2023 12:32
Stargz Study Notes

Stargz Study Notes

idea

  • proposed by bradfitz for acceralating the CI of the Go lang project
  • x/build: speed up large container start-up times without pre-pulling containers into VMs (CRFS): golang/go#30829
    • motivation: Our current situation (building a container, pushing to gcr.io, then automating the creation of a COS-like VM images that has the image pre-pulled) is pretty gross and tedious.
    • initial: https://github.com/google/crfs (read-only FUSE filesystem that lets you mount a container image, served directly from a container registry)

crfs

@Dentrax
Dentrax / copy-helm-images.sh
Last active December 13, 2021 14:56
Get all images in helm chart and copy to internal registry
#!/bin/bash
set -e
function usage(){
echo "$(basename $0) --registry registry.gitlab.com/images --platform linux/amd64 --chart fluent/fluent-bit --version 0.19.10" >&2
}
function teardown {
rm -rf "./tmp"
@Dentrax
Dentrax / main.go
Last active August 14, 2022 20:41
demo: delve-entrypoint-mutating
package main
import (
"archive/tar"
"bytes"
"fmt"
"github.com/google/go-containerregistry/pkg/crane"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/mutate"