Skip to content

Instantly share code, notes, and snippets.

View shantanuo's full-sized avatar

Shantanu Oak shantanuo

65 followers · 149 following
View GitHub Profile
@shantanuo
shantanuo / syllabify-with-index.py
Created May 1, 2011 13:06 — forked from santhoshtr/syllabify-with-index.py
syllabify with word and syllable index
#!/usr/bin/python
# -*- coding: utf-8 -*-
texts =[u"वाराणसी", u"भौगोलिक", u"उपदर्शन"]
signs = [
u'\u0902', u'\u0903', u'\u093e', u'\u093f', u'\u0940', u'\u0941',
u'\u0942', u'\u0943', u'\u0944', u'\u0946', u'\u0947', u'\u0948',
u'\u094a', u'\u094b', u'\u094c', u'\u094d']
limiters = ['.','\"','\'','`','!',';',',','?']
virama = u'\u094d'
@shantanuo
shantanuo / dumper.pl
Created September 20, 2011 09:12
MySQLdump filter. Removes or replaces the DEFINER clauses from a dump
#!/usr/bin/perl
use strict;
use warnings;
use Getopt::Long qw(:config no_ignore_case );
my $replace = undef;
my $delete = undef;
my $help = 0;
GetOptions (
@shantanuo
shantanuo / syllabify-with-index.py
Created February 10, 2012 07:10 — forked from santhoshtr/syllabify-with-index.py
syllabify with word and syllable index
#!/usr/bin/python
# -*- coding: utf-8 -*-
# texts =[u"वाराणसी", u"भौगोलिक", u"उपदर्शन"]
# CREATE TABLE syllabalize (id int(11) DEFAULT NULL, seqn int(11) DEFAULT NULL, akshar varchar(10) DEFAULT NULL) DEFAULT CHARSET=utf8
import codecs
f = codecs.open('testfile.txt', encoding='utf-8')
texts = f.read().split()
signs = [
u'\u0902', u'\u0903', u'\u093e', u'\u093f', u'\u0940', u'\u0941',
@shantanuo
shantanuo / mysql_to_big_query.sh
Last active September 14, 2022 07:12
Copy MySQL table to big query. If you need to copy all tables, use the loop given at the end. Exit with error code 3 if blob or text columns are found. The csv files are first copied to google cloud before being imported to big query.
#!/bin/sh
TABLE_SCHEMA=$1
TABLE_NAME=$2
mytime=`date '+%y%m%d%H%M'`
hostname=`hostname | tr 'A-Z' 'a-z'`
file_prefix="trimax$TABLE_NAME$mytime$TABLE_SCHEMA"
bucket_name=$file_prefix
splitat="4000000000"
bulkfiles=200
@shantanuo
shantanuo / incremental_backup.sh
Created January 19, 2013 08:31
Incremental backup of customer_ticket table and all data backup of other tables
#!/bin/sh
mydb=$1
> $mydb.sql
param="-uroot --compact $mydb"
yesterday=`date '+%Y-%m-%d 00:00:00' -d "1 day ago"`
today=`date '+%Y-%m-%d 00:00:00'`
mywhere="lastupdate>'$yesterday' and lastupdate<'$today'"
# incremental data backup
@shantanuo
shantanuo / helper.sql
Last active December 11, 2015 12:49
The procedure will generate 4,50,000 seconds since the date "2013-01-19". For each record it will save the unix timestamp and corresponding date.
use test;
drop table if exists filler;
drop procedure prc_filler;
CREATE TABLE filler (id INT NOT NULL, msg_timestamp int, mydate date, key(mydate)) ENGINE=Myisam;
DELIMITER $$
CREATE PROCEDURE prc_filler(cnt INT, mydate datetime)
@shantanuo
shantanuo / mysql_to_redshift.sh
Last active January 27, 2020 15:38
Make sure that the package s3cmd is installed and PHP supports pgsql driver. time sh -xv mysql_to_redshift.sh test email_lists This will create a text file createme.sql.txt
#!/bin/sh
# download this script
# make sure s3cmd is installed
# yum install s3cmd
# if yum mentioned above does not work then
# wget http://downloads.sourceforge.net/project/s3tools/s3cmd/1.5.0-alpha1/s3cmd-1.5.0-alpha1.tar.gz
# tar xvf s3cmd-1.5.0-alpha1.tar.gz
# cd s3cmd-1.5.0-alpha1
# python setup.py install
# python 2.6+ is required
@shantanuo
shantanuo / binary_extract_upload.sh
Last active December 14, 2015 03:09
extract data from binary and upload to remote server using curl. The text files used for "load data infile" command are tarred and sql commands are gunzipped.
#!/bin/sh
# call the script as shown below
# sh -xv extract.sh "db_name" "2013-02-23 00:00:00" "2013-02-23 01:59:59"
# or else it will select yesterday binary by default
mystart1=`date -d"1 days ago" +'%Y-%m-%d 00:00:01'`
mystop1=`date +'%Y-%m-%d 00:00:00'`
mystart=${2:-$mystart1}
@shantanuo
shantanuo / schema_convert.sh
Last active February 10, 2021 09:21
script to convert mysql schema to be compatible with data warehouse software
#!/bin/sh
# script to convert mysql schema to be compatible with data warehouse software
# make sure that s3cmd and maatkit utility is installed
db_name=${1:-'test'}
> /root/$db_name.txt
temppath='/mnt/data/pdump1'
host='localhost'
user='maatkit'
@shantanuo
shantanuo / scrap_dict.php
Created March 24, 2013 12:16
Source code of http://saraswaticlasses.net/yubnub/language.html Translate any word to 5 Indian Languages.
language.html
<html>
<body>
<head>
<title> Translate any word to 5 Indian languages</title>
</head>
<form action="welcome.php" method="post">