Skip to content

Instantly share code, notes, and snippets.


Haridas N haridas

View GitHub Profile
haridas /
Created Nov 23, 2018
Ensure a big jsons particular field does't includes null, Helpful as part of datacleanup process.
import json
def read_json_lines(fname, filed_name):
num = 0
doc_size = []
error_docs = []
with open(fname) as f:
while True:
line = f.readline()
if not line:
haridas /
Created Nov 2, 2018
NLP pre-processing - Remove unicode chars from text
import glob
import pandas as pd
files = glob.glob('out-*.json')
def remove_unicode_char(file_name):
f = open(file_name, 'rb').read()
with open(file_name, 'w') as nf:
nf.write(f.decode(encoding="ascii", errors="ignore"))
print ("=> ", file_name)
haridas /
Created Oct 8, 2018
Python debugger on Ipython shell
import os
import ipdb; ipdb.set_trace()
# other codes..
trap 'increment && END=1' 2
increment() {
ls /
echo "Cleaned up"
haridas /
Last active May 8, 2017
Compile python source from source, compile flags, and other settings for data science works.
  1. Ensure all the development files required to build custom bindings, mainly bzip2, and sqlite3 bingings are important.
  2. Build python with enabling the unicode flag usc4
$ sudo apt-get install libbz2-dev libsqlite3-dev
$ ./configure --enable-unicode=ucs4
$ make
$ make install
haridas /
Last active Jan 6, 2017
Remove old files completely from git history
## How to remove the old files/folder from all git commits.
#1. clone the repo freshly
git clone <repo.git>
#2. Do index-filter option to go through all the object indexs and look for give match
# And remove those matching objects.
git filter-branch \
--prune-empty \
--index-filter \
haridas /
Last active Nov 24, 2016
Manage Android sdk from command line

Some times it would be very handy to check / update android sdk from command line. For automation pipelines it surely helps. Here is few commands that can be used to check or update android sdk from command line.

List Installed SDK details

haridas@haridas-HP-ProBook-4440s:~$ android list sdk
Refresh Sources:
  Validate XML
  Parse XML
  Fetched Add-ons List successfully
import os
import sys
import subprocess
from optparse import OptionParser
from datetime import datetime
def run_shell_script(shell_script):
Assuming that the script is comming from trusted source.
View json_unmarshalling.go
package main
import (
type Response struct {
Action string
Node Nodes
haridas /
Created Mar 24, 2015
Ketama based Consistent hashing implementation of python-memcache library.
To Test this Script the start 8 memcache servers using this command.
$ memcached -d -p {PortNumber}
You can’t perform that action at this time.