Skip to content

Instantly share code, notes, and snippets.

@niyue
niyue / develop_pyarrow_remote_container.md
Last active September 13, 2022 09:48
develop pyarrow with visual studio code remote containers
  1. clone arrow git repo locally (under a macOS in my case)
  2. open arrow/python folder with vscode
  3. create a Dockerfile for pyarrow development. pyarrow provides such Dockerfile, so you only need to do some linking ln -s examples/minimal_build/Dockerfile.ubuntu Dockerfile
  4. Follow the steps here to open a remote container as dev env in vscode
    1. Run "Remote-Containers: Open Folder in Container..." in vscode
    2. Choose Dockerfile you just linked
    3. vscode will build the container and open it later, just wait
  5. Enter the container for building pyarrow
  6. docker exec -it bash
@redmcg
redmcg / kubedf
Last active April 15, 2024 15:30
Bash script to show k8s PVC usage
#!/usr/bin/env bash
NODESAPI=/api/v1/nodes
function getNodes() {
kubectl get --raw $NODESAPI | jq -r '.items[].metadata.name'
}
function getPVCs() {
jq -s '[flatten | .[].pods[].volume[]? | select(has("pvcRef")) | '\
@mhart
mhart / steps.sh
Last active September 11, 2023 11:01
Get SCL (and GCC 7) working on Amazon Linux 2017.03
yum install -y iso-codes # Needed for scl-utils-build
curl -O http://vault.centos.org/6.5/SCL/x86_64/scl-utils/scl-utils-20120927-11.el6.centos.alt.x86_64.rpm
curl -O http://vault.centos.org/6.5/SCL/x86_64/scl-utils/scl-utils-build-20120927-11.el6.centos.alt.x86_64.rpm
curl -O http://mirror.centos.org/centos/6/extras/x86_64/Packages/centos-release-scl-rh-2-3.el6.centos.noarch.rpm
curl -O http://mirror.centos.org/centos/6/extras/x86_64/Packages/centos-release-scl-7-3.el6.centos.noarch.rpm
rpm -Uvh *.rpm # Had to run this twice? Get an error first time, maybe Docker related
rm *.rpm
@htfy96
htfy96 / static_inline_example.md
Last active April 11, 2024 14:13
static inline vs inline vs static in C++

In this article we compared different behavior of static, inline and static inline free functions in compiled binary. All the following test was done under g++ 7.1.1 on Linux amd64, ELF64.

Test sources

header.hpp

#pragma once

inline int only_inline() { return 42; }
static int only_static() { return 42; }
  • What do Etcd, Consul, and Zookeeper do?
    • Service Registration:
      • Host, port number, and sometimes authentication credentials, protocols, versions numbers, and/or environment details.
    • Service Discovery:
      • Ability for client application to query the central registry to learn of service location.
    • Consistent and durable general-purpose K/V store across distributed system.
      • Some solutions support this better than others.
      • Based on Paxos or some derivative (i.e. Raft) algorithm to quickly converge to a consistent state.
  • Centralized locking can be based on this K/V store.
@irbull
irbull / OpenSSLExample.cpp
Created August 11, 2016 18:32
Code signing and verification with OpenSSL
#include <iostream>
#include <openssl/aes.h>
#include <openssl/evp.h>
#include <openssl/rsa.h>
#include <openssl/pem.h>
#include <openssl/ssl.h>
#include <openssl/bio.h>
#include <openssl/err.h>
#include <assert.h>
@andyleejordan
andyleejordan / main.cpp
Last active March 20, 2022 15:32
Algorithm for Efficient Chunked File Reading in C++
/* Algorithm for Efficient Chunked File Reading in C++
*
* The MIT License (MIT)
*
* Copyright 2014 Andrew Schwartzmeyer
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
* files (the "Software"), to deal in the Software without
* restriction, including without limitation the rights to use, copy,
@tnolet
tnolet / gist:7361441
Last active December 5, 2018 02:48
Install collectd 5.4 on Centos 6.x and make it spit out cool metrics. Copied from http://linuxdrops.com/install-collectd-statistics-collecter-on-centos-rhel-ubuntu-debian/ and tweaked for your and my pleasure. For all other cool options, check the provided link.
#!/bin/bash
# Perform installation as root
# Install prereqs
yum -y install libcurl libcurl-devel rrdtool rrdtool-devel rrdtool-prel libgcrypt-devel gcc make gcc-c++
# Get Collectd, untar it, make it and install
wget http://collectd.org/files/collectd-5.4.0.tar.gz
tar zxvf collectd-5.4.0.tar.gz
@LukeMurphey
LukeMurphey / splunk_rest_handler.py
Last active February 1, 2023 22:56
This is basic Splunk REST handler that can be used to make custom endpoints in Splunk. Tags: #splunk
import splunk.admin as admin
import splunk.entity as entity
import splunk
import logging
import logging.handlers
import os
import re
import copy
class StandardFieldValidator(object):
@wrunk
wrunk / jinja2_file_less.py
Last active September 19, 2023 15:05
python jinja2 examples
#!/usr/bin/env/python
#
# More of a reference of using jinaj2 without actual template files.
# This is great for a simple output transformation to standard out.
#
# Of course you will need to "sudo pip install jinja2" first!
#
# I like to refer to the following to remember how to use jinja2 :)
# http://jinja.pocoo.org/docs/templates/
#