Skip to content

Instantly share code, notes, and snippets.

Avatar

Karthik Ravindra karthikbgl

View GitHub Profile
@karthikbgl
karthikbgl / spark_dataframe_to_single_file_csv_using_hdfs.py
Created Mar 8, 2018
Saves a spark dataframe into a single csv/delimited file efficiently. Assumes the file storage to be hdfs
View spark_dataframe_to_single_file_csv_using_hdfs.py
import subprocess
def write_to_local_fs(df):
"""
This method writes to local filesystem efficiently without using coalesce or repartition.
The idea is to persist data in cluster format in hdfs (or whatever file storage) and write to local file system.
Write header to the file in the local file system
:param: df: the dataframe being sent as argument
"""
@karthikbgl
karthikbgl / regex_matcher.js
Last active Jul 22, 2017
Regex pattern matches using jquery
View regex_matcher.js
(function($) {
$.fn.regexFinder = function(text) {
/*
Phone number formats
XXX-XXX-XXXX
XXX.XXX.XXXX
XXX XXX XXXX
*/
var cc_re = /(?:4[0-9]{12}(?:[0-9]{3})?|[25][1-7][0-9]{14}|6(?:011|5[0-9][0-9])[0-9]{12}|3[47][0-9]{13}|3(?:0[0-5]|[68][0-9])[0-9]{11}|(?:2131|1800|35\d{3})\d{11})/g;
var date_re = /\d{2}\/\d{4}/g;
@karthikbgl
karthikbgl / combinations_substring.py
Created Oct 24, 2015
All possible combinations of substrings
View combinations_substring.py
import itertools
def yieldSubset(s):
return sorted(list(set([''.join(c) for i in range(len(s)+1) for c in itertools.combinations(s, i) if c])))
View smallest_multiple.py
import fractions
def smallest_multiple( n):
return reduce(lambda x, y: x*y/fractions.gcd(x, y), range(2, n+1))
@karthikbgl
karthikbgl / console_log_ie_fix.js
Last active Aug 29, 2015
console.log messages in code are bad for Internet Explorer <= 9. This plugin would avoid the application from crashing when console.log messages are present in the javascript code.
View console_log_ie_fix.js
;(function () {
// console.log statements cause issues in Internet Explorer <= 9.
// This piece of code prevents the application to fail
// Source for console: https://developer.mozilla.org/en-US/docs/Web/API/Console
var fns = [
"assert",
"count",
"debug",
"dir",
"dirxml",
@karthikbgl
karthikbgl / handler_500.py
Created Jun 25, 2015
Override the standard 500 error page to display the stack trace if certain conditions are met. The advantage of overriding the server error handler is, it keeps everything else intact - example all the error emails, etc.
View handler_500.py
from django.views.debug import technical_500_response
from django.views.defaults import server_error
def server_error_handler(request):
ip_address = request.META.get("HTTP_X_FORWARDED_FOR", request.META.get("REMOTE_ADDR"))
if ip_address in settings.ALLOWED_IPS_FOR_DEBUG_SCREEN: #or request.user.is_staff
return technical_500_response(request, *sys.exc_info())
return server_error(request)
@karthikbgl
karthikbgl / seconds_to_string
Created Oct 23, 2014
Time elapsed in seconds
View seconds_to_string
def secondsToStr(t):
'''
This snippet converts the time elapsed in
seconds to HH:MM:SS.mmm
Example:
>>> secondsToStr(123456)
'34:17:36.000'
'''
View functools_partials.py
'''
This is an example of functools.partials
Typical usage is when you have a function that needs multiple parameters,
of which one is a constant, you can use a partial.
Example:
Return only the values in an iterable which match the regex 'c.t' where . is a single character.
Input:
['cat', 'cut', 'abc', 'def']
@karthikbgl
karthikbgl / migrations.py
Last active Aug 29, 2015
Editing south migrations to run custom sql
View migrations.py
# -*- coding: utf-8 -*-
import datetime
from south.db import db
from south.v2 import DataMigration
from django.db import models
class Migration(DataMigration):
def forwards(self, orm):
db.start_transaction()
@karthikbgl
karthikbgl / sunsigns.py
Last active Aug 29, 2015
Sunsigns models and create view using django
View sunsigns.py
#models.py
CAPRICORN = 'Capricorn'
AQUARIUS = 'Aquarius'
PISCES = 'Pisces'
ARIES = 'Aries'
TAURUS = 'Taurus'
GEMINI = 'Gemini'
CANCER = 'Cancer'
LEO = 'Leo'
You can’t perform that action at this time.