Skip to content

Instantly share code, notes, and snippets.

View arundasan91's full-sized avatar

Arun Das arundasan91

View GitHub Profile
@arundasan91
arundasan91 / bazel_install.md
Created December 10, 2016 05:26
Install Bazel and Java OpenJDK 1.8 in Ubuntu 14.04

Install OpenJDK 1.8

sudo add-apt-repository ppa:openjdk-r/ppa
sudo apt-get update && sudo apt-get -y install openjdk-8-jdk

Install Bazel

echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add -
@arundasan91
arundasan91 / torch_build_1
Created February 23, 2017 06:46
Build is successful, dual xeon, NVIDIA M40, plenty RAM
CMake Warning at /usr/share/cmake-3.5/Modules/FindCUDA/make2cmake.cmake:65 (message):
Removing non-existent dependency file: generic/THCTensorMathScan.h
CMake Warning at /usr/share/cmake-3.5/Modules/FindCUDA/make2cmake.cmake:65 (message):
Removing non-existent dependency file: generic/THCTensorMasked.h
@arundasan91
arundasan91 / opencv_video_to_frames.py
Created April 14, 2017 19:56
Function to extract frames from input video file using OpenCV and save them as separate frames in an output directory.
def video_to_frames(input_loc, output_loc):
"""Function to extract frames from input video file and save them as separate frames in an output directory.
Args:
input_loc: Input video file.
output_loc: Output directory to save the frames.
Returns:
None
"""
heat_template_version: 2015-10-15
description: Implementation of Apache MXNET Distributed Deep Learning Appliance for OpenStack.
parameters:
key_name:
type: string
label: Key name
description: Name of a key pair to enable SSH access to the instance
default: default
@arundasan91
arundasan91 / download_youtube_video.py
Created April 14, 2017 19:57
Function to download video's from YouTube to a specific location. Requires 'youtube-dl'.
def download_video(url, output_dir, filename, ext='mp4'):
"""Function to download video's from YouTube to a specific location.
Args:
url: YouTube URL.
output_dir: Output directory to save the video.
filename: Filename of the video.
ext: File type to save the video to. Default to mp4.
"""
import os
#!/bin/bash
ls -1 *.tiff | parallel convert '{}' '{.}.jpg'
find . -name "*.tiff" -print0 | xargs -0 -I {} rm {}
@arundasan91
arundasan91 / launch_jupyter_at_boot.sh
Created October 17, 2017 00:53
Script to launch Jupyter notebook at boot time of VMs or Containers.
#!/bin/bash
echo '1'
export PATH=$PATH:/home/student/.local/bin/
echo '2'
su - student -c "bash -c '/home/student/.local/bin/jupyter notebook'"
echo 'Launched Jupyter Notebook'
#touch /home/student/testrclocal
sed -i "s|PasswordAuthentication no|PasswordAuthentication yes|g" /etc/ssh/sshd_config
echo '3'
export HOSTIP=`hostname -I`
@arundasan91
arundasan91 / core-site.xml
Created December 10, 2016 05:33
core-site.xml file for integrating openstack swift with hadoop
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
<property>
<name>fs.swift.impl</name>
<value>org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem</value>
</property>
<property>
@arundasan91
arundasan91 / install-packages-CC-Ubuntu1604-GPU.sh
Last active February 21, 2018 01:30
This bash script will create OpenStack `openrc` file for the current tenant, source it, download required files from OpenStack Swift Object Storage and then runs the scripts to install TensorFlow 1.3, OpenCV from source and some other user defined packages.
#!/usr/bin/env bash
# OpenRC extraction
function extract_json_key {
RESULT=$(echo "$2" | sed "s/.*$1\": \"//g" | sed 's/".*//g')
echo "$RESULT"
}
JSON_VENDOR_DATA=$(curl -s http://169.254.169.254/openstack/latest/vendor_data.json)
SITE=$(extract_json_key "site" "$JSON_VENDOR_DATA")
#!/bin/bash
sudo apt update
sudo apt install libhdf5-dev
sudo apt-get install -y opencl-headers build-essential protobuf-compiler libprotoc-dev libboost-all-dev libleveldb-dev hdf5-tools libhdf5-serial-dev libopencv-core-dev libopencv-highgui-dev libsnappy-dev libatlas-base-dev cmake libstdc++6-4.8-dbg libgoogle-glog0v5 libgoogle-glog-dev libgflags-dev liblmdb-dev git python-pip gfortran
sudo sh -c "sudo echo '/usr/local/cuda/lib64' > /etc/ld.so.conf.d/cuda_hack.conf"
sudo ldconfig /usr/local/cuda/lib64
pip install --upgrade --user pip
pip install scikit-image --user
cd /home/$USER/packages/
git clone https://github.com/BVLC/caffe.git