Skip to content

Instantly share code, notes, and snippets.

View fulibacsi's full-sized avatar

András Fülöp fulibacsi

View GitHub Profile
@fulibacsi
fulibacsi / failstate.py
Last active April 23, 2021 00:18
Non-blocking exception catching context manager with step skipping and lightweight logging and reporting capabilities.
"""Non-blocking exception catching context manager with step skipping and
lightweight logging and reporting capabilities.
2021 - Andras Fulop @fulibacsi"""
import json
import sys
import traceback
import warnings
@fulibacsi
fulibacsi / randomframe.py
Last active August 22, 2019 13:31
Generate random pandas DataFrame
import datetime
import numpy as np
import pandas as pd
from pandas.util.testing import makeIntIndex, rands_array
class RandomDataFrame:
@fulibacsi
fulibacsi / chunker.py
Created March 7, 2018 14:39
chunker function in python
def chunk(seq, size):
return (seq[pos:pos + size] for pos in range(0, len(seq), size))
@fulibacsi
fulibacsi / custom_deduplication.py
Last active March 22, 2023 06:51
deduplication on pandas dataframe with custom function
import pandas as pd
from tqdm import tqdm
def jaccard_sim(doc1, doc2, thres=0.9):
return len(doc1 & doc2) / len(doc1 | doc2) > thres
def duplicated(df, textcol, func, **kwargs):
"""Simlarly to pd.duplicated it finds the duplicated rows
@fulibacsi
fulibacsi / Dynamic plotting
Last active January 25, 2017 17:55
Dynamic plotting class with multiline support
%matplotlib notebook
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import colors as mcolors
class DynamicPlot():
colors = list(mcolors.cnames.values())