Skip to content

Instantly share code, notes, and snippets.

Samarth Bhargav samarthbhargav

Block or report user

Report or block samarthbhargav

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@samarthbhargav
samarthbhargav / CUDA Availability in Torch
Created Sep 24, 2018
One liner to test if cuda is available
View CUDA Availability in Torch
python -c "import torch; print(torch.cuda.is_available())"
@samarthbhargav
samarthbhargav / python_stop_thread.py
Last active Mar 8, 2018
A 'stoppable' thread for Python
View python_stop_thread.py
# source: https://www.safaribooksonline.com/library/view/python-cookbook-2nd/0596007973/ch09s03.html
import threading
class TestThread(threading.Thread):
def _ _init_ _(self, name='TestThread'):
""" constructor, setting initial variables """
self._stopevent = threading.Event( )
self._sleepperiod = 1.0
threading.Thread._ _init_ _(self, name=name)
def run(self):
@samarthbhargav
samarthbhargav / scrape_xkcd.py
Last active Dec 12, 2017
XKCD Scraper - using BeautifulSoup and requests
View scrape_xkcd.py
from bs4 import BeautifulSoup
import requests
import shutil
import glob
from os import listdir
from os.path import isfile, join
def save_image(url, filename):
response = requests.get(url, stream=True)
@samarthbhargav
samarthbhargav / timer.py
Created Dec 18, 2014
a simple profiler for python
View timer.py
# source: http://www.huyng.com/posts/python-performance-analysis/
import time
class Timer(object):
def __init__(self, verbose=False):
self.verbose = verbose
def __enter__(self):
self.start = time.time()
return self
@samarthbhargav
samarthbhargav / powerset.py
Created Dec 4, 2014
Recursive generation of a power set
View powerset.py
# as seen in 6.00.2x, Week 6, Lecture 1, Video 4
def powerset(iterable):
if len(iterable) == 0:
return [[]]
smaller = powerset(iterable[1:])
withElem = []
for s in smaller:
withElem.append( s + [iterable[0]])
@samarthbhargav
samarthbhargav / .bashrc
Last active Mar 3, 2018
bashrc for Hadoop
View .bashrc
# Hadoop Properties
export HADOOP_PREFIX=/usr/local/hadoop # installation location
export HADOOP_HOME=$HADOOP_PREFIX
export HADOOP_COMMON_HOME=$HADOOP_PREFIX
export HADOOP_CONF_DIR=$HADOOP_PREFIX/etc/hadoop
export HADOOP_HDFS_HOME=$HADOOP_PREFIX
export HADOOP_MAPRED_HOME=$HADOOP_PREFIX
export HADOOP_YARN_HOME=$HADOOP_PREFIX
export PATH=$PATH:$HADOOP_PREFIX/bin
View yarn-site.xml
<?xml version="1.0"?>
<configuration>
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>8192</value>
</property>
<property>
<name>yarn.scheduler.maximum-allocation-mb</name>
<value>4096</value>
</property>
View mapred-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>localhost:10020</value>
View hdfs-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>dfs.datanode.data.dir</name>
<value>/usr/local/hadoop/hdfs/data/</value>
<description>Comma separated list of paths on the local filesystem of a DataNode where it should store its blocks.</description>
</property>
<property>
<name>dfs.namenode.name.dir</name>
View core-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost/</value>
<description>NameNode URI</description>
</property>
<property>
<name>hadoop.tmp.dir</name>
You can’t perform that action at this time.