Skip to content

Instantly share code, notes, and snippets.

View corporatepiyush's full-sized avatar

Piyush Katariya corporatepiyush

View GitHub Profile
@corporatepiyush
corporatepiyush / glibc-tuning.sh
Last active May 29, 2024 12:49
Memory optimisations using jemalloc and tcmalloc on Linux
#!/bin/sh
# touch /etc/profile.d/malloc_config.sh
# Get total RAM size in MB
total_ram=$(free -m | awk '/^Mem:/{print $2}')
if [ "$total_ram" -lt 8192 ]; then
# Low RAM settings
export MALLOC_ARENA_MAX=2
@corporatepiyush
corporatepiyush / tune-linux.sh
Last active May 26, 2024 18:52
MacOS and Linux Optimisation
# System V shared memory parameters
kernel.shmmax = 1048576
kernel.shmmin = 1
kernel.shmmni = 128
kernel.shmseg = 32
kernel.shmall = 4096
# IPC parameters
net.core.somaxconn = 8192
fs.file-max = 1048576
@corporatepiyush
corporatepiyush / FileDB.dart
Last active August 15, 2023 14:15
Using File System as Key Value Database
// ignore_for_file: prefer_function_declarations_over_variables
import 'dart:async';
import 'dart:collection';
import 'dart:convert';
import 'dart:io';
import 'dart:typed_data';
import 'package:basics/basics.dart';
import 'package:quiver/cache.dart';
@corporatepiyush
corporatepiyush / fetchSave.js
Created June 29, 2023 15:07
Taking a day wise backup of large SQL Table incrementally
const pg = require('pg-promise')();
const fs = require('fs');
const db = pg({
user: "****",
password: "*****",
host: "******",
port: 5432,
database: "*****",
ssl: {
@corporatepiyush
corporatepiyush / blockAds-linux.sh
Last active May 27, 2024 08:23
Block ads and malware for MacOS and Linux
# Protection from Malicious Domain (ads, malware, hack scripts, fakenews and gambling )
# You may not be doing it but various apps installed on your system doing it internally, so having ad blocker in Browser does not helps.
# Execute below command in your terminal once every week.
sudo chmod 774 /etc/hosts
curl https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/fakenews-gambling-porn/hosts > hosts
sudo mv hosts /etc/hosts
@corporatepiyush
corporatepiyush / dial-pq-via-ssh.go
Created May 27, 2023 09:02 — forked from vinzenz/dial-pq-via-ssh.go
Use postgres via SSH in Golang
package main
import (
"database/sql"
"database/sql/driver"
"fmt"
"net"
"os"
"time"
@corporatepiyush
corporatepiyush / rollups.sql
Created November 14, 2019 18:27 — forked from marcocitus/rollups.sql
Efficient real-time rollups with backfilling in Citus
CREATE TABLE rollups (
name text,
rolled_up_generation bigint default -1
);
-- Create a stub on workers to allow usage as a default in distributed tables
SELECT run_command_on_workers($$
CREATE OR REPLACE FUNCTION current_rollup_generation(rollup_name text)
RETURNS bigint LANGUAGE sql
AS $function$
@corporatepiyush
corporatepiyush / example.sql
Created November 7, 2019 13:19 — forked from marcocitus/example.sql
Safe incremental rollups on Postgres and Citus
-- Create the raw events table
CREATE TABLE page_views (
site_id int,
path text,
client_ip inet,
view_time timestamptz default now(),
view_id bigserial
);
-- Allow fast lookups of ranges of sequence IDs
// suppose you spawned 16 node processes on 16 core CPU
ps axf | grep node | grep -v grep | awk '{print $1}' | xargs -n 1 taskset -cp 0-15
@corporatepiyush
corporatepiyush / pubsub-coordinator.sql
Created September 13, 2018 10:33 — forked from marcocitus/pubsub-coordinator.sql
Prototype for PubSub on PG 10 with Citus 7
/* commands to run on the coordinator */
CREATE EXTENSION citus;
SELECT master_add_node('10.0.0.2', 5432);
SELECT master_add_node('10.0.0.3', 5432);
SELECT start_metadata_sync_to_node(nodename, nodeport) FROM pg_dist_node;
SET citus.replication_model TO 'streaming'
CREATE TABLE events (
event_id bigserial primary key,