Skip to content

Instantly share code, notes, and snippets.

View Harkishen-Singh's full-sized avatar
💭
Engineering high-perf tools

Harkishen Singh Harkishen-Singh

💭
Engineering high-perf tools
View GitHub Profile
@Harkishen-Singh
Harkishen-Singh / build_pgcopydb_ubuntu.md
Created March 15, 2024 10:44
Instructions to build dmriti/pgcopydb on a Ubuntu machine.
sudo apt-get install build-essential \
  libgc-dev \
  libselinux-dev \
  libzstd-dev \
  liblz4-dev \
  libxslt-dev \
  libpam-dev \
  libz-dev \
 libreadline-dev \
@Harkishen-Singh
Harkishen-Singh / table_rows.md
Created August 29, 2023 06:03
PLPGSQL function that returns the number of rows of a given array of tables in a 'table_name', 'count' format
CREATE OR REPLACE FUNCTION table_rows(tables text[])
RETURNS TABLE (table_name text, count bigint) AS $$
DECLARE
  query text := '';
BEGIN
  FOR i IN 1..array_length(tables, 1) LOOP
    query := query || format('SELECT %L as table_name, count(*) from %s', tables[i], tables[i]);
    IF i < array_length(tables, 1) THEN
 query := query || ' union all ';
@Harkishen-Singh
Harkishen-Singh / truncate all tables in a schema.md
Last active August 7, 2023 13:47
A PLPGSQL script to truncate all tables in a given list of schemas
-- Usage: SELECT truncate_all_tables_in_schemas(ARRAY['schema1', 'schema2', 'schema3']);
CREATE OR REPLACE FUNCTION truncate_all_tables_in_schemas(schema_names text[]) RETURNS void AS $$
DECLARE 
   schema_name text;
   table_name text; 
BEGIN 
   FOREACH schema_name IN ARRAY schema_names
   LOOP
 FOR table_name IN (SELECT tablename FROM pg_tables WHERE schemaname = schema_name) 
@Harkishen-Singh
Harkishen-Singh / describe all tables in a schema.md
Created July 24, 2023 07:42
Script to run `\d` against all tables in a schema

Script to describe all tables in a schema

#!/bin/bash

# Set these variables to your actual database connection details
DB_NAME="your_database_name"
DB_USER="your_username"
DB_PASS="your_password"
DB_HOST="localhost"
DB_PORT="5432"
@Harkishen-Singh
Harkishen-Singh / db_size_increase.md
Last active August 2, 2023 09:14
A PLPGSQL function to increase database size until a `minimum_db_size` (in GBs) is reached. It does this by cloning `source_schema` into new schemas with prefix `dest_schema`. Useful for creating large datasets.
-- This function increases the DB size upto `minimum_db_size` by cloning the `source_schema` into a new name `dest_schema_<dynamic_count>`
CREATE OR REPLACE FUNCTION increase_db_size_upto(source_schema text, dest_schema text, minimum_db_size int, multiplier_factor int)
RETURNS void AS
$BODY$
DECLARE
    objeto record;
    buffer text;
    db_size int;
    schema_count int;
@Harkishen-Singh
Harkishen-Singh / Duplicate all tables in Postgres.md
Last active May 8, 2023 10:26
Duplicate all tables in Postgres

duplicate all tables under 'public'

do
$$
declare
    r record;
    prefix text := 'copy_1_';
    total integer := (SELECT count(table_name)::integer FROM information_schema.tables WHERE table_schema = 'public');
    current integer := 0;
begin
@Harkishen-Singh
Harkishen-Singh / generate_counter_data_postgres.md
Created October 31, 2022 10:53
This Gist helps to generate Prometheus Counter data behaviour on TimescaleDB/PostgresQL

Generate counter metric data

This example was done on TimescaleDB@2.8.1.

Create function

create or replace function generate_counter_data(_from timestamptz, _till timestamptz, _step interval, _insert_table_name TEXT) returns void as
$$
    declare
        r record;
 counter integer := 0;

Code that ingests duplicates

package main

import (
	"context"
	"time"

	"github.com/go-kit/log"
@Harkishen-Singh
Harkishen-Singh / prometheus_tsdb_index_layout_gist.md
Created December 16, 2021 11:27
This Gist explains Prometheus's tsdb index layout along with some implementation details at various steps.
STEP 1: When AddSymbol() is called
----------------------------------------
alenblen;label_name_1;label_value_1;label_name_2;label_name_3;label_value_2;label_value_3;

remarks: "alenblen" is the space reserved for symbol table metadata, like size of the symbol table, and number of symbols.
This is overwritten in the finishSymbol().

STEP 2: When finishSymbol() is called
----------------------------------------
sudo apt update
sudo apt-get install gcc -y
wget -c https://dl.google.com/go/go1.17.4.linux-amd64.tar.gz -O - | sudo tar -xz -C /usr/local
echo "export PATH=$PATH:/usr/local/go/bin" >> ~/.profile
source ~/.profile
go version