Skip to content

Instantly share code, notes, and snippets.

@auxten
auxten / bench_chdb_dataframe.py
Last active May 29, 2023 02:53
Bench different impl of running chdb directly on dataframe
import os
import time
import pandas as pd
import pyarrow as pa
import chdb
import subprocess
# file size 117MB
data_path = '/home/Clickhouse/bench/hits_0.parquet'
@NickCrews
NickCrews / coalesce_parquet.py
Last active January 10, 2024 03:48
Coalesce parquet files
"""coalesce_parquets.py
gist of how to coalesce small row groups into larger row groups.
Solves the problem described in https://issues.apache.org/jira/browse/PARQUET-1115
"""
from __future__ import annotations
from pathlib import Path
from typing import Callable, Iterable, TypeVar
@maxymania
maxymania / hyperswitch.js
Last active December 30, 2020 17:02
NodeJS Hyperswarm utility.
/*
Copyright (C) 2019 Simon Schmidt
Usage of the works is permitted provided that this instrument is retained with
the works, so that any entity that uses the works is notified of this instrument.
DISCLAIMER: THE WORKS ARE WITHOUT WARRANTY.
*/
const hyperswarm = require('hyperswarm');
@ismail0352
ismail0352 / Dockerfile.deb_rpm_nginx
Last active May 26, 2024 10:49
Creating your own .deb file repo and host it using Nginx container with "autoindex on;"
FROM ubuntu as ubuntu
RUN apt-get update
RUN apt-get install -y dpkg-dev wget gnupg2 curl
ARG ubuntu_packages="wget htop default-jre-headless apt-transport-https nvidia-container-toolkit cuda-drivers libopengl0 linux-image-extra-virtual omnisci"
WORKDIR /opt/packages/deb
# Nvidia-Docker
@adubovikov
adubovikov / rds.sh
Created April 1, 2016 12:21 — forked from onyxraven/rds.sh
Amazon RDS Performance Tuning Settings
#XLarge DBInstanceClassMemory = 15892177440 = 14.8GB
#/32 = 496630545 = 473MB
#/64 = 248315272 = 236MB
#/128 = 124157636 = 118MB
#/256 = 62078818 = 59MB
#/512 = 31039409 = 29MB
#/12582880 = 1263 #default same divisor as max_connections = 4041.6MB = 4237924762
#/25165760 = 623 # half of max_connections = 1993.6MB
#/50331520 = 315 # quarter of max_connections = 1008MB = 1056964608
#*(3/4) #default innodb pool size = 11922309120
@TheBigSadowski
TheBigSadowski / anon.js
Last active April 27, 2021 11:18
Random IP address generator for anonymizing data.
randomByte = function() {
return Math.round(Math.random()*256);
}
randomIp = function() {
var ip = randomByte() +'.' +
randomByte() +'.' +
randomByte() +'.' +
randomByte();
if (isPrivate(ip)) return randomIp();
@chrismatthieu
chrismatthieu / registrar.js
Created April 10, 2011 04:25 — forked from mheadd/registrar.js
Node.JS SIP Registrar
var sip = require('sip');
var sys = require('sys');
var redis = require('redis');
//Trim leading and trailing whitespace from string values.
function trim(str) {
return str.replace(/^\s+|\s+$/g, '');
}
sip.start({},function(request) {