Skip to content

Instantly share code, notes, and snippets.

@epiphani
epiphani / CDHTez.md
Last active February 14, 2024 08:03
Getting Tez enabled on CDH5.4+

So Hive in CDH is horribly, painfully slow. Cloudera ships Hive 1.1, which is actually moderately modern. It is, however, very badly configured out of the box and patched with custom code from Cloudera. With a bit of effort, we managed to improve hive performance considerably. We really shouldn't have to do this, but Cloudera is actively working against supporting a performant Hive.

First, building Tez was fairly straightforward. Using the instructions at https://github.com/apache/tez/blob/master/docs/src/site/markdown/install.md, the only change was to use the version string "2.6.0" for the build. I believe that was the default. Don't use the CDH string, it won't work.

At the bottom of the installation instructions, there's mention of the fact that to use the local hadoop jars (rather than those packaged with tez) you must unpack the jars in HDFS rather than using the tarball. In this case, unpack the tez-minimal tarball and upload the contents to /apps/tez-0.7.0 (or whatever you prefer). Don't fo

@marcan
marcan / linux.sh
Last active December 1, 2023 15:18
Linux kernel initialization, translated to bash
#!/boot/bzImage
# Linux kernel userspace initialization code, translated to bash
# (Minus floppy disk handling, because seriously, it's 2017.)
# Not 100% accurate, but gives you a good idea of how kernel init works
# GPLv2, Copyright 2017 Hector Martin <marcan@marcan.st>
# Based on Linux 4.10-rc2.
# Note: pretend chroot is a builtin and affects the current process
# Note: kernel actually uses major/minor device numbers instead of device name
@den-crane
den-crane / Clickhouse-fast-not-exists
Last active May 29, 2024 23:22
Clickhouse fast not exists
ClickHouse server version 18.14.12 revision 54409.
create table data(K Int64, V String) engine=MergeTree order by K;
insert into data select number, toString(number) from numbers(100,100000000);
optimize table data final;
create table buffer(K Int64, V String) engine=Memory;
insert into buffer select number, toString(number) from numbers(0,1000);
@slok
slok / pprof.md
Last active May 21, 2024 07:01
Go pprof cheat sheet

Enable profiling

Default http server

import (
    _ "net/http/pprof"
    "net/http"
)
@atyachin
atyachin / Android_Emulator_AWS_EC2_ARM64_2022.txt
Last active June 8, 2024 20:34
Running headless android emulator on AWS EC2 Ubuntu instance (ARM64 / aarch64) - 2022
Android Emulator (ARM64) on EC2 - 2022
---------------------------------------
1. Launch EC2 ARM based Instance (a1.metal / a1.2xlarge): (16 Gb RAM, 32Gb Disk), Ubuntu Server 22.04 LTS (HVM) ARM x64
2. sudo apt update && sudo apt upgrade
3. sudo apt install default-jdk python3-pip repo python-is-python3 unzip libpcre2-dev adb
4. wget https://dl.google.com/android/repository/commandlinetools-linux-8512546_latest.zip
5. unzip commandlinetools-linux-8512546_latest.zip -d android-sdk
6. sudo mv android-sdk /opt/
7. mkdir /opt/android-sdk/cmdline-tools/latest
8. mv /opt/android-sdk/cmdline-tools/* /opt/android-sdk/cmdline-tools/latest (ignore the error)