Skip to content

Instantly share code, notes, and snippets.

View BrunoGomesCoelho's full-sized avatar
Learning to learn

Bruno Gomes Coelho BrunoGomesCoelho

Learning to learn
View GitHub Profile
@BrunoGomesCoelho
BrunoGomesCoelho / conda_ds.sh
Last active January 12, 2019 19:24
Quickly get a simple data science environement running with conda
wget https://repo.anaconda.com/archive/Anaconda3-5.2.0-Linux-x86_64.
sh Anaconda3-5.2.0-Linux-x86_64.sh
conda update -n base conda
# Activate a environment then inside it run:
conda install pandas matplotlib jupyter notebook scipy scikit-learn nb_conda seaborn
# Add some extra things...
# jupyter extensions
conda install -c conda-forge jupyter_contrib_nbextensions
@BrunoGomesCoelho
BrunoGomesCoelho / faster_csv_concat.py
Created July 21, 2019 22:06
Read and concat various pandas dataframes in parallel. All credits to @zemekeneng on stackoverflow
from multiprocessing import Pool # for reading the CSVs faster
def my_read_csv(filename):
# Helper function for the parellel load_csvs
return pd.read_csv(filename)
def load_csvs(prefix):
"""Reads and joins all our CSV files into one big dataframe.
We do it in parallel to make it faster, since otherwise it takes some time.
Idea from: https://stackoverflow.com/questions/36587211/easiest-way-to-read-csv-files-with-multiprocessing-in-pandas
@BrunoGomesCoelho
BrunoGomesCoelho / clone.sh
Created August 29, 2019 18:40
Clone non-empty folder
# taken from https://stackoverflow.com/a/20538655
git init
git remote add origin PATH/TO/REPO
git fetch
git checkout -t origin/master
@BrunoGomesCoelho
BrunoGomesCoelho / reduce_mem_usage.py
Created August 31, 2019 18:23
Reduce memory usage Pandas
"""Famous kaggle reduce mem usage script.
NOT MINE - taken from https://www.kaggle.com/gemartin/load-data-reduce-memory-usage
"""
import pandas as pd
import numpy as np
def reduce_mem_usage(df):
""" iterate through all the columns of a dataframe and modify the data type
@BrunoGomesCoelho
BrunoGomesCoelho / embed_plotly.py
Created April 11, 2022 01:55
Embed a plotly figure in jupyter
def embed_plotly(fig):
"""
See https://plotly.com/python/static-image-export/
"""
# not sure if this import will work outside jupyter
from IPython.display import Image
img_bytes = fig.to_image(format="png")
return Image(img_bytes)