neeraj neeraj1928

## pyspark-virtualenv-jupyter-ubuntutahr-setup.md

      
              1 file
            
          
              0 forks
            
          
              1 comment
            
          
              5 stars
            
          
                jayant91089
                / pyspark-virtualenv-jupyter-ubuntutahr-setup.md
            
            
              Last active
              December 15, 2018 12:54
            
              
                Describes how to setup spark 2.1.0 for use with python (pyspark) via jupyter notebook. Does not assume root access. Uses virtualenv.
              
          
    Get prebuilt spark

mkdir spark_install && cd spark_install
wget http://d3kbcqa49mib13.cloudfront.net/spark-2.1.0-bin-hadoop2.7.tgz
tar -xzvf spark-2.1.0-bin-hadoop2.7.tgz 
cd spark-2.1.0-bin-hadoop2.7/
Test prebuilt spark (this should open a spark console, use Ctrl+C to exit )


## min-char-rnn.py
"""
Minimal character-level Vanilla RNN model. Written by Andrej Karpathy (@karpathy)
BSD License
"""
import numpy as np

# data I/O
data = open('input.txt', 'r').read() # should be simple plain text file
chars = list(set(data))
data_size, vocab_size = len(data), len(chars)
	"""
	Minimal character-level Vanilla RNN model. Written by Andrej Karpathy (@karpathy)
	BSD License
	"""
	import numpy as np

	# data I/O
	data = open('input.txt', 'r').read() # should be simple plain text file
	chars = list(set(data))
	data_size, vocab_size = len(data), len(chars)