Skip to content

Instantly share code, notes, and snippets.

View manuthu's full-sized avatar

Mike manuthu

View GitHub Profile
@manuthu
manuthu / import_parquet_file_to_local.py
Last active July 4, 2024 11:08
RDS Export to S3 dumps the files as parquet files. This script assumes that the parquet files have already been downloaded to the local onprem server.
import os
import pyarrow.parquet as pq
import pandas as pd
import mysql.connector
import concurrent.futures
import logging
import time
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger('DB')
@manuthu
manuthu / postgres_queries_and_commands.sql
Created May 17, 2022 05:51 — forked from rgreenjr/postgres_queries_and_commands.sql
Useful PostgreSQL Queries and Commands
-- show running queries (pre 9.2)
SELECT procpid, age(clock_timestamp(), query_start), usename, current_query
FROM pg_stat_activity
WHERE current_query != '<IDLE>' AND current_query NOT ILIKE '%pg_stat_activity%'
ORDER BY query_start desc;
-- show running queries (9.2)
SELECT pid, age(clock_timestamp(), query_start), usename, query
FROM pg_stat_activity
WHERE query != '<IDLE>' AND query NOT ILIKE '%pg_stat_activity%'
# Install the nifty client
# pip install git+https://github.com/east36/python-nifty-client.git
import logging
from niftyclient import NiftyClient
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger('STK')
@manuthu
manuthu / psql_encoding.markdown
Created February 21, 2022 11:09 — forked from joshteng/psql_encoding.markdown
This solves Postgresql's encoding issue (happened to me when running postgres on my vagrant box) The error happens when trying to create db "rake db:create": Error message: "encoding UTF8 does not match locale en_US; the chosen LC_CTYPE setting requires encoding LATIN1"
sudo su postgres
psql
update pg_database set datistemplate=false where datname='template1';
drop database Template1;
create database template1 with owner=postgres encoding='UTF-8' lc_collate='en_US.utf8' lc_ctype='en_US.utf8' template template0;
update pg_database set datistemplate=true where datname='template1';
#!/usr/bin/env bash
set -e
DB_USER=<db-user>
DB_HOST=<db-host>
DB_NAME=<db-name>
CSV_FILE_NAME=<Reports>
@manuthu
manuthu / sane-caching.nginx.conf
Created May 25, 2021 12:01 — forked from philipstanislaus/sane-caching.nginx.conf
Sample Nginx config with sane caching settings for modern web development
# Sample Nginx config with sane caching settings for modern web development
#
# Motivation:
# Modern web development often happens with developer tools open, e. g. the Chrome Dev Tools.
# These tools automatically deactivate all sorts of caching for you, so you always have a fresh
# and juicy version of your assets available.
# At some point, however, you want to show your work to testers, your boss or your client.
# After you implemented and deployed their feedback, they reload the testing page – and report
# the exact same issues as before! What happened? Of course, they did not have developer tools
# open, and of course, they did not empty their caches before navigating to your site.
#!/bin/bash
set -e
# Database clone script.
# ENV
# dev, staging, demo, prod
ENV=dev

Get the user_id from the console/portal

List all the files belonging to the user . For this case, we use - 6a76bf01-dc90-4996-a846-8472cb086565

rados ls -p .rgw.buckets | grep 6a76bf01-dc90-4996-a846-8472cb086565

Save the files to a log file

-- db=# \d order_payment
-- Table "public.order_payment"
-- Column | Type | Collation | Nullable | Default
-- -----------------+-----------------------------+-----------+----------+---------
-- id | uuid | | not null |
-- order_id | uuid | | |
-- amount_paid | numeric | | |
-- ...
select
@manuthu
manuthu / cloudstack-routers-running-in-a-storagepool.sh
Last active April 15, 2020 15:38
A script to get all the running routers in a given storage pool in XEN.
# Get all running routers. Routers in cloudstack are prefixed with `r-DIGIT`
RUNNING_VMS=$(xe vm-list | grep -A2 -B2 'r-[0-9]' | grep uuid | awk -F':' '{print $2}')
SRID=8c269040-fc42-c12d-b091-8958784ec65e
for vm_id in $RUNNING_VMS
do
xe vbd-list vm-uuid=$vm_id | grep vdi-uuid | awk -F':' '{print $2}' | xargs -I {} xe vdi-list uuid={} | grep $SRID && echo $vm_id
done