Skip to content

Instantly share code, notes, and snippets.

View anilktechie's full-sized avatar
💭
I may be slow to respond.

ak anilktechie

💭
I may be slow to respond.
View GitHub Profile
@anilktechie
anilktechie / pysqlitewrapper.py
Created December 19, 2021 06:00 — forked from rohzart/pysqlitewrapper.py
Simple python sqlite wrapper
import sqlite3 as sql
from contextlib import closing
class dal:
def __init__(self, dbname):
self.dbname = dbname
@staticmethod
def __connection(dbname):
@anilktechie
anilktechie / db.py
Created December 19, 2021 06:00 — forked from nav3van/db.py
Python
import psycopg2
dsn = dict(
dbname='dbname',
host='hostname',
password='password',
port=5432,
user='username'
)
conn = psycopg2.connect(**dsn)
import functools
def excuteImmediately(fn):
@functools.wraps(fn)
def wrapper(*args, **kwargs):
return fn(*args, **kwargs)
return wrapper()
example:
@anilktechie
anilktechie / postgres.py
Created December 19, 2021 06:00 — forked from goldsborough/postgres.py
Python psycopg2 wrapper
###########################################################################
#
## @file postgres.py
#
###########################################################################
import psycopg2
###########################################################################
#
@anilktechie
anilktechie / mysql_wrapper.py
Created December 19, 2021 05:59 — forked from lovesh/mysql_wrapper.py
A MySQLdb wrapper class
import MySQLdb
import MySQLdb.cursors
from config import config
class Cursor(object):
def __init__(self, mysql_cursor):
self.cursor = mysql_cursor
def __iter__(self):
@anilktechie
anilktechie / sql_builder_insert.py
Created December 19, 2021 05:57 — forked from stephane-klein/sql_builder_insert.py
sql_builder_insert Python example
from psycopg2 import sql
import queries
pg_session = queries.Session(
queries.uri(
host='localhost',
port='5439',
dbname='postgres',
user='postgres',
password='password'
@anilktechie
anilktechie / iotworkshop-DataScience.json
Created December 6, 2021 00:35 — forked from rtempleton/iotworkshop-DataScience.json
Zeppelin Notebook for use in Hortonworks IOT workshop
{
"paragraphs": [
{
"text": "%md\n### Run simple queries to check the count of both the iotdata and iotrollup tables.\nClick the play button within each of the paragraphs to rerun/update the content\n",
"user": "admin",
"dateUpdated": "2018-10-05T19:03:40-0400",
"config": {
"tableHide": false,
"editorSetting": {
"language": "markdown",
@anilktechie
anilktechie / WITSML_SNOWFLAKE.sql
Created December 6, 2021 00:35 — forked from rtempleton/WITSML_SNOWFLAKE.sql
Source code for WITSML processing in Snowflake referenced in
--set up your environment
create database my_test_db;
create schema my_test_db.witsml;
use schema my_test_db.witsml;
--create the staging table where all WITSML files are loaded to by Snowpipe
create table witsml_temp (col1 variant);
@anilktechie
anilktechie / SnowflakeCreditMonitoring.xml
Created December 6, 2021 00:35 — forked from rtempleton/SnowflakeCreditMonitoring.xml
This is an importable Nifi template that can be used to monitor Snowflake resources and send reports via Slack. https://medium.com/@ryan_templeton/snowflake-resource-monitor-reports-delivered-to-slack-using-apache-nifi-8dfd4fc4d579
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
<description>Workflow to monitor Snowflake credit consumption and report to Slack. Full article can be found here https://snowflakecomputing.atlassian.net/wiki/spaces/RT/pages/771200336/Creating+a+Snowflake+usage+report+using+Apache+Nifi
Requires existing tables and apps defined in your Snowflake account and your Slack account</description>
<groupId>be279da7-0167-1000-6a67-4d0cb36decc6</groupId>
<name>SnowflakeCreditMonitoring</name>
<snippet>
<connections>
<id>1d50d3ac-3446-3e09-0000-000000000000</id>
@anilktechie
anilktechie / SnowflakeCreditMonitoring.sql
Created December 6, 2021 00:35 — forked from rtempleton/SnowflakeCreditMonitoring.sql
SQL source code used in the Snowflake Resource Monitor reports delivered to Slack using Apache Nifi article: https://medium.com/@ryan_templeton/snowflake-resource-monitor-reports-delivered-to-slack-using-apache-nifi-8dfd4fc4d579
--create a table to hold the metrics you wish to monitor and their threshold values
CREATE TABLE "MY_TEST_DB"."PUBLIC"."THRESHOLDS" ("SERVICE_TYPE" STRING NOT NULL, "CREDITS_BILLED" DOUBLE NOT NULL) COMMENT = 'Used for the Nifi alerting demo';
--insert some sample records into the thresholds table
insert into "MY_TEST_DB"."PUBLIC"."THRESHOLDS" values ('AUTO_CLUSTERING', 10),('PIPE', 10),('MATERIALIZED_VIEW', 10),('WAREHOUSE_METERING', 10);
--query to compare current metrics to threshold values
--This is used for the HOURLY report
select a.*, iff(b.credits_billed is null, 0, b.credits_billed)::string as credits_billed from
"MY_TEST_DB"."PUBLIC"."THRESHOLDS" a left join