Skip to content

Instantly share code, notes, and snippets.

Piyush Katariya corporatepiyush

View GitHub Profile
@corporatepiyush
corporatepiyush / rollups.sql
Created Nov 14, 2019 — forked from marcocitus/rollups.sql
Efficient real-time rollups with backfilling in Citus
View rollups.sql
CREATE TABLE rollups (
name text,
rolled_up_generation bigint default -1
);
-- Create a stub on workers to allow usage as a default in distributed tables
SELECT run_command_on_workers($$
CREATE OR REPLACE FUNCTION current_rollup_generation(rollup_name text)
RETURNS bigint LANGUAGE sql
AS $function$
@corporatepiyush
corporatepiyush / example.sql
Created Nov 7, 2019 — forked from marcocitus/example.sql
Safe incremental rollups on Postgres and Citus
View example.sql
-- Create the raw events table
CREATE TABLE page_views (
site_id int,
path text,
client_ip inet,
view_time timestamptz default now(),
view_id bigserial
);
-- Allow fast lookups of ranges of sequence IDs
View Allocate CPU cores together.sh
// suppose you spawned 16 node processes on 16 core CPU
ps axf | grep node | grep -v grep | awk '{print $1}' | xargs -n 1 taskset -cp 0-15
@corporatepiyush
corporatepiyush / pubsub-coordinator.sql
Created Sep 13, 2018 — forked from marcocitus/pubsub-coordinator.sql
Prototype for PubSub on PG 10 with Citus 7
View pubsub-coordinator.sql
/* commands to run on the coordinator */
CREATE EXTENSION citus;
SELECT master_add_node('10.0.0.2', 5432);
SELECT master_add_node('10.0.0.3', 5432);
SELECT start_metadata_sync_to_node(nodename, nodeport) FROM pg_dist_node;
SET citus.replication_model TO 'streaming'
CREATE TABLE events (
event_id bigserial primary key,
View ngrams.js
const _ = require('ramda')
const ngrams = function (string) {
var r = [];
for (var n = 2; n <= string.length; n++)
for (var i = 0; i <= string.length - n; i++)
r.push(string.substring(i, i + n));
return r;
}
@corporatepiyush
corporatepiyush / findAndCopy.sh
Created Sep 4, 2017
Find file by name and copy to desired destination folder
View findAndCopy.sh
find <directory> -name 'screen.png' | xargs -n1 cp <file/directory to be copied>
@corporatepiyush
corporatepiyush / Parallel.sh
Created Sep 4, 2017
Parallel Processing of Commands or Data
View Parallel.sh
seq 1 10 | xargs -n1 -P5 <any command>
@corporatepiyush
corporatepiyush / CountLines.sh
Created Sep 4, 2017
Count Number of Lines in Project files
View CountLines.sh
find $directory$ -type f -name "*.py" | xargs cat | wc -l
View ListSSHKeys.sh
for keyfile in ~/.ssh/id_*; do ssh-keygen -l -f "${keyfile}"; done | uniq
@corporatepiyush
corporatepiyush / download.sh
Last active Sep 4, 2017
Extract Unique Youtube URL from Given Page or Playlist Page with Jquery
View download.sh
// download audio files and convert them to mp3
cat links.txt | xargs -n1 -P2 youtube-dl --extract-audio --audio-format mp3 --audio-quality 9
You can’t perform that action at this time.