This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## ls only directories | |
# https://unix.stackexchange.com/questions/1645/is-there-any-option-with-ls-command-that-i-see-only-the-directories | |
ls -ld -- */ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Byte-compiled / optimized / DLL files | |
__pycache__/ | |
*.py[cod] | |
*$py.class | |
.idea/ | |
.DS_Store/ | |
# C extensions | |
*.so | |
# Distribution / packaging |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
today="$(date '+%Y-%m-%d')" | |
yesterday="$(date -d yesterday '+%Y-%m-%d')" | |
echo $today | |
echo $yesterday |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
git submodule deinit <path_to_submodule> | |
git rm <path_to_submodule> | |
git commit-m "Removed submodule " | |
rm -rf .git/modules/<path_to_submodule> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import argparse | |
def parse_args(): | |
.... | |
return args | |
args = parse_args() | |
config = vars(args) | |
for arg in vars(config): |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Refer https://fizzylogic.nl/2017/11/06/edit-jupyter-notebooks-over-ssh/ | |
# On server | |
jupyter notebook --no-browser --port=8080 | |
# On local machine | |
ssh -N -L 8080:localhost:8080 <remote_user>@<remote_host> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import torch | |
import logging | |
from pytorch_transformers import BertModel, BertTokenizer | |
from pytorch_transformers import * | |
from typing import List | |
CLS_TOKEN = "[CLS]" | |
SEP_TOKEN = "[SEP]" | |
logger = logging.getLogger(__name__) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Also refer https://stackoverflow.com/a/1176225 | |
import sys | |
class Foobar: | |
pass | |
def str_to_class(str): | |
return getattr(sys.modules[__name__], str) | |
str_to_class("Foobar") |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def save_h5_data(file_path, data_split="train", **data_dump): | |
hf = h5py.File(file_path, 'w') | |
hf.attrs['split'] = data_split | |
for key in data_dump: | |
hf.create_dataset(key, data=data_dump[key]) | |
hf.close() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Yield successive n-sized | |
# chunks from l. | |
def divide_chunks(l, n): | |
# looping till length l | |
for i in range(0, len(l), n): | |
yield l[i:i + n] | |
test_list = list(range(100)) |