This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# Install required softwares | |
sudo apt-get update | |
sudo apt-get install fish -y | |
sudo chsh -s /usr/bin/fish | |
# Setup static IP - | |
# LAN - 192.168.0.99 | |
# WLAN - 192.168.0.199 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env bash | |
# Install Open MPI v4 | |
wget https://download.open-mpi.org/release/open-mpi/v4.0/openmpi-4.0.0.tar.gz | |
tar -xvf openmpi-4.0.0.tar.gz | |
cd openmpi-4.0.0 | |
./configure --prefix=/usr/local | |
make all | |
sudo make install | |
sudo ldconfig |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env bash | |
# Remove and Install nvidia-docker2 | |
#======================================================================================================================================== | |
# If you have nvidia-docker 1.0 installed: we need to remove it and all existing GPU containers | |
docker volume ls -q -f driver=nvidia-docker | xargs -r -I{} -n1 docker ps -q -a -f volume={} | xargs -r docker rm -f | |
sudo apt-get purge -y nvidia-docker | |
# Add the package repositories | |
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \ | |
sudo apt-key add - |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# No Proper documentation on installing horovod with spark support | |
# Reference - https://github.com/uber/horovod/blob/master/docs/spark.md | |
# Corrupt Java installtion - | |
sudo apt-add-repository ppa:webupd8team/java | |
sudo apt-get update | |
sudo apt-get install oracle-java8-installer | |
# Install Spark | |
wget http://mirrors.estointernet.in/apache/spark/spark-2.4.0/spark-2.4.0-bin-hadoop2.7.tgz |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Install fish | |
cd /etc/yum.repos.d/ | |
sudo wget https://download.opensuse.org/repositories/shells:fish:release:2/RedHat_RHEL-6/shells:fish:release:2.repo | |
sudo yum install fish | |
# Update locatedb | |
sudo ionice -c3 updatedb | |
# Install g++ | |
sudo yum install gcc72-c++ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env bash | |
# Install Open MPI v4 | |
wget https://download.open-mpi.org/release/open-mpi/v4.0/openmpi-4.0.0.tar.gz | |
tar -xvf openmpi-4.0.0.tar.gz | |
cd openmpi-4.0.0 | |
./configure --prefix=/usr/local | |
make all | |
sudo make install | |
sudo ldconfig |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def humanize_time_delta(td_object): | |
seconds = td_object | |
periods = [ | |
('year', 60 * 60 * 24 * 365), | |
('month', 60 * 60 * 24 * 30), | |
('day', 60 * 60 * 24), | |
('hour', 60 * 60), | |
('minute', 60), | |
('second', 1), | |
('milli_second', 1 / 10 ** 3), |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import typing | |
import tensorflow as tf | |
def validate_fetch(fetch_ops: typing.Union[str, list, dict, tf.Tensor]): | |
if isinstance(fetch_ops, str): | |
if not len(fetch_ops) > 0: | |
raise Exception("Fetch Op is an empty string") | |
return fetch_ops | |
elif isinstance(fetch_ops, tf.Tensor): |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import re | |
def generate_label_from_name(name: str): | |
""" | |
:param name: Name for which label has to be generate | |
:return: | |
""" | |
caps_split = sum([list(match) for match in re.findall('([A-Z][^A-Z][a-z]*)|([0-9][A-Z][^A-Z][a-z]*)|([0-9][A-Z][a-z]*)', name)], []) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from tensorflow.examples.tutorials.mnist import input_data | |
import pandas as pd | |
import numpy as np | |
def convert_tensorflow_mnist_train_data_to_csv(path_to_save:str): | |
mnist = input_data.read_data_sets("/tmp/data/", one_hot=True) | |
data = pd.DataFrame(mnist.train.images) | |
label = pd.DataFrame(mnist.train.labels) | |
df = pd.concat([data, label], axis=1) | |
df.columns = ['D_{}'.format(i) for i in range(784)] + ['L_{}'.format(i) for i in range(10)] |