Skip to content

Instantly share code, notes, and snippets.

@AoJ
AoJ / CMakeLists.txt
Created December 7, 2023 20:02 — forked from hacst/CMakeLists.txt
Basic Systemd sd_notify + watchdog usage
cmake_minimum_required(VERSION 2.8)
project(sdnotify)
add_executable(sdnotify sdnotify.cpp)
target_link_libraries(sdnotify PUBLIC systemd)
@AoJ
AoJ / pyspark_hdfs_utils.py
Created September 9, 2023 12:23 — forked from idiomer/pyspark_hdfs_utils.py
Using PySpark to handle HDFS, such as list (ls), rename (mv), delete (rm)
'''
The path is a directory by default
'''
def hdfs_list(path, subtract_one=True):
fs = spark._jvm.org.apache.hadoop.fs.FileSystem.get(spark._jsc.hadoopConfiguration())
list_status = fs.listStatus(spark._jvm.org.apache.hadoop.fs.Path(path))
# file.getPath().getName(), file.getBlockSize(), file.getLen()
files_size = [file.getLen() for file in list_status]
totol_size_in_MB = sum(files_size) / 1024.0 / 1024.0
@AoJ
AoJ / zfs-intro.md
Created January 22, 2023 13:27
Introduction to Z File System (ZFS)

Introduction to Z File System (ZFS)

What is Z File System (ZFS)?

Z File System or also known as ZFS was created by Matthew Ahrens and Jeff Bonwick in 2001 to be the next generation file system for Sun Microsystem's Solaris. In 2008 ZFS was ported to FreeBSD which at the same year a ZFS ported project for Linux is also started the project name was ZFS on Linux.

However since ZFS is under the Common Development and Distribution License it can't be included directly to Linux Kernel. To solve this many Linux distributions make an offer to install ZFS from their package manager.

@AoJ
AoJ / shell_tips.sh
Created January 11, 2023 11:01 — forked from komljen/shell_tips.sh
Linux shell tips and tricks
# Suspend process:
Ctrl + z
# Move process to foreground:
fg
# Generate random hex number where n is number of characters:
openssl rand -hex n
# Execute commands from a file in the current shell:
@AoJ
AoJ / build-qcow2.nix
Created September 28, 2022 08:52 — forked from tarnacious/build-qcow2.nix
Build a bare bones bootable nixos qcow2 image suitable for running with libvirt/qemu/kvm.
{ config, lib, pkgs, ... }:
with lib;
{
imports =
[
<nixpkgs/nixos/modules/installer/cd-dvd/channel.nix>
./machine-config.nix
];
@AoJ
AoJ / README.org
Created September 2, 2022 06:46 — forked from Ladicle/README.org
Emacs Configuration ver. 2022

Ladicle’s Emacs Configuration ver.2022

https://user-images.githubusercontent.com/1159133/156761456-4be104ae-4cb3-420e-81c7-8abd00497d51.png

Outline

Package Management (link)

  • leaf-keywords
    • el-get
    • hydra
  • package-utils
@AoJ
AoJ / otel-cli-ideas.sh
Created August 18, 2022 11:58 — forked from tobert/otel-cli-ideas.sh
span & event ideas for otel-cli
#!/bin/bash
# otel-cli-ideas.sh - some ideas I'm playing with for otel-cli to cover some
# use cases where manual spans or exec don't quite do the trick
#
# another cool outcome is you could run this without the collector in some
# cases without spamming your OTLP endpoint...
#
# I hope to OSS otel-cli under Apache 2 in early April 2021.
# maybe it could background itself and wait for events on a unix socket?
@AoJ
AoJ / victoria_metrics_push_gw.py
Created August 5, 2022 10:36 — forked from f41gh7/victoria_metrics_push_gw.py
VictoriaMetrics as push gateway.
from prometheus_client import Counter, start_http_server
from threading import Thread
import requests as re
import time
JOB_NAME = 'test'
INSTANCE = 'localhost'
def scrape_and_send(local_url: str, vm_url: str, scrape_interval: int):
@AoJ
AoJ / systemd_service_hardening.md
Created March 30, 2022 11:01 — forked from ageis/systemd_service_hardening.md
Options for hardening systemd service units

security and hardening options for systemd service units

A common and reliable pattern in service unit files is thus:

NoNewPrivileges=yes
PrivateTmp=yes
PrivateDevices=yes
DevicePolicy=closed
ProtectSystem=strict
@AoJ
AoJ / initial-setup.md
Created February 23, 2022 14:15 — forked from ace-subido/initial-setup.md
Benchmarking a Hadoop Cluster

Requirements

  1. HDP-2.2 installed by Ambari
  2. Install HDFS Client
  3. Patience

Instructions

Setup