This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import pandas as pd | |
def read_json_to_df(file_path): | |
# df = pd.read_json(path_or_buf=file_path,orient='records',lines=True) | |
df = pd.read_json(path_or_buf=file_path, orient='records') | |
return df | |
def read_json_list_to_df(json_list): | |
df = pd.DataFrame.from_records(json_list) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from collections import defaultdict | |
# method 1 | |
my_dict = defaultdict(int) | |
my_dict[key] += 1 | |
# method 2 | |
my_dict = {} | |
my_dict[key] = my_dict.get(key, 0) + 1 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Provide the path where to download | |
wget https://www.dropbox.com/s/twmtutniktom7tu/VisualDialog_val2018.zip?dl=0 -P ./data | |
# Provide the name of the file | |
wget https://www.dropbox.com/s/twmtutniktom7tu/VisualDialog_val2018.zip?dl=0 -O VisualDialog_val2018.zip |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# To copy all files recursively in a folder except .pth files | |
# Be careful no "/" in source_dir whle "/" in final_dir | |
rsync -avr --exclude=*.pth source_dir final_dir/ | |
# For a file in just one folder | |
cp -r !(*.png) dest |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rm checkpoint_{6..8}.pth |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tail -f logs.txt | |
watch -n1 nvidia-smi | |
htop | |
watch -n 1 free -h |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
slit horizontal ctrl+b " | |
unsplit horizontal ctrl+b x |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import argparse | |
args = parser.parse_args() | |
for arg in vars(args): | |
print("{:<20}: {}".format(arg, getattr(args, arg))) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
RUN /bin/bash -c "apt-get install tmux -y" | |
RUN /bin/bash -c "apt-get install htop -y" | |
# Trying conda environment | |
RUN /opt/conda/bin/conda create -y --name myenv python=3.6.8 && \ | |
/opt/conda/bin/conda clean -ya | |
RUN echo "source activate myenv" > ~/.bashrc | |
ENV PATH /opt/conda/envs/myenv/bin:$PATH |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import time | |
start = time.time() | |
"the code you want to test stays here" | |
end = time.time() | |
print(end - start) |