vinnix

# Create a folder for our new root structure
$ export centos_root='/centos_image/rootfs'
$ mkdir -p $centos_root
# initialize rpm database
$ rpm --root $centos_root --initdb
# download and install the centos-release package, it contains our repository sources
$ yum reinstall --downloadonly --downloaddir . centos-release
$ rpm --root $centos_root -ivh centos-release*.rpm
$ rpm --root $centos_root --import  $centos_root/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
# install yum without docs and install only the english language files during the process

vinnix

Tuning ZFS + Postgres to outperform EXT4 + Postgres

Please refer to ZFS 2-3x slower than EXT4 to see how ZFS defaults + Postgres defaults severely underperform EXT4 defaults + Postgres defaults (and also to know more about the system on which these benchmarks were performed). This page documents how to tune ZFS + Postgres to give better performance for the tpcb-like benchmark.

BIG FAT WARNING

Please do not copy these settings blindly because I am myself not clear on why/how these settings had the impact they did. For example, I cannot explain why full_page_writes=off independently did not give that much boost, nor did an optimized PG configuration. However, putting both of them together gave a 2-4x boost compared to baseline numbers.

Benchmark results

vinnix

#!/usr/bin/env python
# 
# Simplistic RDS logfile downloader because AWS CLI + Botocore is broken. :-(
#
from __future__ import print_function

import argparse
import os.path
import sys

vinnix

GDB commands by function - simple guide
---------------------------------------
More important commands have a (*) by them.

Startup 
% gdb -help         	print startup help, show switches
*% gdb object      	normal debug 
*% gdb object core 	core debug (must specify core file)
%% gdb object pid  	attach to running process
% gdb        		use file command to load object 

vinnix

##
## Trying to learn something from here:
## http://profmarcello.blogspot.com/search/label/PostGIS%20Raster
##


## Bottom (past) -> Top(now) worklog chronology 

-- Using 2nd patch!

vinnix

1. Clone your fork:

git clone git@github.com:YOUR-USERNAME/YOUR-FORKED-REPO.git

2. Add remote from original repository in your forked repository:

cd into/cloned/fork-repo
git remote add upstream git://github.com/ORIGINAL-DEV-USERNAME/REPO-YOU-FORKED-FROM.git
git fetch upstream

vinnix

$ ./bitcoin-qt --version
Bitcoin Core version v0.15.99.0-5180a86 (64-bit)
Copyright (C) 2009-2017 The Bitcoin Core developers

[vinnix@vnxsam bin]$ valgrind --leak-check=full --show-leak-kinds=all --leak-check=yes ./bitcoind  
==2817== Memcheck, a memory error detector
==2817== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==2817== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==2817== Command: ./bitcoind
==2817== 

vinnix

CREATE OR REPLACE FUNCTION public.json_append(data json, insert_data json)
RETURNS json
IMMUTABLE
LANGUAGE sql
AS $$
    SELECT ('{'||string_agg(to_json(key)||':'||value, ',')||'}')::json
    FROM (
        SELECT * FROM json_each(data)
        UNION ALL
        SELECT * FROM json_each(insert_data)

vinnix

CREATE OR REPLACE FUNCTION public.json_append(data json, insert_data json)
RETURNS json
IMMUTABLE
LANGUAGE sql
AS $$
    SELECT ('{'||string_agg(to_json(key)||':'||value, ',')||'}')::json
    FROM (
        SELECT * FROM json_each(data)
        UNION ALL
        SELECT * FROM json_each(insert_data)

vinnix

$ python get_logs.py rds-db-instance-name rds.log aws-access-key aws-secret-key
$ ./pgbadger -p '%t:%r:%u@%d:[%p]:' ./rds.log

$ open out.html