Skip to content

Instantly share code, notes, and snippets.

@89465127
89465127 / merge_files.scala
Created May 16, 2013 17:00
scala merge files
import scala.io.Source
var filea = Source.fromFile("filea").mkString
var fileb = Source.fromFile("fileb").mkString
var merged = filea + fileb
print(merged)
@89465127
89465127 / git_on_windows.html
Created May 14, 2013 03:25
How to get up and running with git on windows, FAST.
<ul>
<li>Install <a href="http://chocolatey.org/">Chocolatey</a></li>
<li>Open powershell (as admin) and enter "cinst git.commandline"</li>
<li>Environment variable: GIT_SSH = C:\git\bin\ssh.exe</li>
<li>Put your keys into ~/.ssh</li>
</ul>
Optional steps:<br />
Add C:\git\bin to your PATH, if needed<br />
SSH stuff can be found in C:\git\bin
@89465127
89465127 / mongo_random.py
Created May 13, 2013 19:39
Pymongo implementation as described here: http://cookbook.mongodb.org/patterns/random-attribute/ Note that you need to have the field 'random' indexed in mongo: http://api.mongodb.org/python/current/api/pymongo/collection.html#pymongo.collection.Collection.ensure_index Could be extended to use other formal parameters from find: http://api.mongod…
def get_random(collection, query={}):
'''Gets a random record from a mongo collection'''
random_num = random.random()
query.update({'random' : {'$gte' : random_num}})
result = collection.find_one(query)
if not result:
query.update({'random' : {'$lt' : random_num}})
result = collection.find_one(query)
return result
rsync -avz /path/to/local/sync/folder -e "ssh -i /path/to/ssh/key" ubuntu@ec2instance:/path/to/remote/sync/folder
@89465127
89465127 / spark_ubuntu.bash
Last active December 16, 2015 19:39
How to unzip and install Spark on Ubuntu.
# Download and unzip spark to your home directory
# You may want to run update if it's on a fresh OS
sudo apt-get update
# Install dependencies (& verify)
sudo apt-get install default-jdk scala git
java -version; javac -version; scala -version; git --version
# Configure - run these in your open terminal and add them to ~/.bashrc
@89465127
89465127 / write_line.py
Created April 29, 2013 18:56
Printing to a single line in Python, in separate steps, flushing output.
from __future__ import print_function
import sys
import time
print('Executing...', end='')
sys.stdout.flush()
time.sleep(2)
print('Done.')
@89465127
89465127 / multi.py
Created April 26, 2013 19:03
Multiprocessing example, iterating over each line in a file.
import multiprocessing
import StringIO
# function to be run on each line of file
def f(x):
print x,
# Create mock file
mock_file = StringIO.StringIO()
for i in range(20):
@89465127
89465127 / mock_file.py
Created April 26, 2013 18:58
How to mock a file in memory for testing and experimenting.
import StringIO
# Create mock file
mock_file = StringIO.StringIO()
for i in range(20):
mock_file.write("{}\n".format(i))
mock_file.seek(0)
# Do something
for line in mock_file:
@89465127
89465127 / error.py
Created April 26, 2013 17:38
Simple example showing how you can still raise an error in Python, but report more details with it.
other_var = 42
try:
print 1/0
except Exception as e:
print "\nError encountered. Other var = {}\n".format(other_var)
raise e
print 'You will not see this text'
@89465127
89465127 / cli_progress.py
Last active January 9, 2019 10:43
Simple example of how to get a python progress bar in your program.
import time
from progress.bar import Bar # sudo pip install progress
bar = Bar('Processing', max=20, suffix='%(index)d/%(max)d - %(percent).1f%% - %(eta)ds')
for i in range(20):
time.sleep(.05) # Do some work
bar.next()
bar.finish()