Skip to content

Instantly share code, notes, and snippets.

View corporatepiyush's full-sized avatar

Piyush Katariya corporatepiyush

View GitHub Profile
@corporatepiyush
corporatepiyush / download.sh
Last active September 4, 2017 08:43
Extract Unique Youtube URL from Given Page or Playlist Page with Jquery
// download audio files and convert them to mp3
cat links.txt | xargs -n1 -P2 youtube-dl --extract-audio --audio-format mp3 --audio-quality 9
@corporatepiyush
corporatepiyush / ListSSHKeys.sh
Created September 4, 2017 09:23
List SSH Keys
for keyfile in ~/.ssh/id_*; do ssh-keygen -l -f "${keyfile}"; done | uniq
@corporatepiyush
corporatepiyush / CountLines.sh
Created September 4, 2017 09:24
Count Number of Lines in Project files
find $directory$ -type f -name "*.py" | xargs cat | wc -l
@corporatepiyush
corporatepiyush / Parallel.sh
Created September 4, 2017 09:25
Parallel Processing of Commands or Data
seq 1 10 | xargs -n1 -P5 <any command>
@corporatepiyush
corporatepiyush / findAndCopy.sh
Created September 4, 2017 09:27
Find file by name and copy to desired destination folder
find <directory> -name 'screen.png' | xargs -n1 cp <file/directory to be copied>
@corporatepiyush
corporatepiyush / ngrams.js
Created October 13, 2017 14:11
Ngrams Generators
const _ = require('ramda')
const ngrams = function (string) {
var r = [];
for (var n = 2; n <= string.length; n++)
for (var i = 0; i <= string.length - n; i++)
r.push(string.substring(i, i + n));
return r;
}
@corporatepiyush
corporatepiyush / pubsub-coordinator.sql
Created September 13, 2018 10:33 — forked from marcocitus/pubsub-coordinator.sql
Prototype for PubSub on PG 10 with Citus 7
/* commands to run on the coordinator */
CREATE EXTENSION citus;
SELECT master_add_node('10.0.0.2', 5432);
SELECT master_add_node('10.0.0.3', 5432);
SELECT start_metadata_sync_to_node(nodename, nodeport) FROM pg_dist_node;
SET citus.replication_model TO 'streaming'
CREATE TABLE events (
event_id bigserial primary key,
@corporatepiyush
corporatepiyush / Allocate CPU cores together.sh
Last active December 16, 2020 10:21
Allocating CPU cores
// suppose you spawned 16 node processes on 16 core CPU
ps axf | grep node | grep -v grep | awk '{print $1}' | xargs -n 1 taskset -cp 0-15
@corporatepiyush
corporatepiyush / example.sql
Created November 7, 2019 13:19 — forked from marcocitus/example.sql
Safe incremental rollups on Postgres and Citus
-- Create the raw events table
CREATE TABLE page_views (
site_id int,
path text,
client_ip inet,
view_time timestamptz default now(),
view_id bigserial
);
-- Allow fast lookups of ranges of sequence IDs
@corporatepiyush
corporatepiyush / rollups.sql
Created November 14, 2019 18:27 — forked from marcocitus/rollups.sql
Efficient real-time rollups with backfilling in Citus
CREATE TABLE rollups (
name text,
rolled_up_generation bigint default -1
);
-- Create a stub on workers to allow usage as a default in distributed tables
SELECT run_command_on_workers($$
CREATE OR REPLACE FUNCTION current_rollup_generation(rollup_name text)
RETURNS bigint LANGUAGE sql
AS $function$