Skip to content

Instantly share code, notes, and snippets.

Mark Betz Markbnj

Block or report user

Report or block Markbnj

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@Markbnj
Markbnj / environment-specific-settings.py
Last active Aug 29, 2015
Environment-specific settings for python 2 modules
View environment-specific-settings.py
# Environment-specific settings for python 2 modules
# This code placed in __init__.py will allow a module to maintain
# environment-specific settings in files located in a settings folder
# within the module.
#
# Example:
#
# ├── mymodule.py
# ├── __init__.py
@Markbnj
Markbnj / ec2-discover-instances.py
Last active Aug 29, 2015
Discover ec2 instances by tag and tag value
View ec2-discover-instances.py
from boto import ec2
class Resource(object):
"""
Represents a single resource returned from the Discovery class's
get_resources method.
"""
def __init__(self, host_name, private_host_name, ip, private_ip, tags):
self.__host_name = host_name
self.__private_host_name = private_host_name
@Markbnj
Markbnj / debug-scrapyd-ipdb.py
Last active Aug 29, 2015
User ipdb to debug a running spider in scrapyd
View debug-scrapyd-ipdb.py
# The scrapyd daemon adds a lot of power and flexibility to scrapy, but it also adds a layer of indirection
# (another process) that can make debugging a running spider troublesome. Use this code to launch scrapyd
# in ipdb (or pdb). You'll be able to breakpoint and catch exceptions inside the spiders it launches.
import scrapyd.runner
import ipdb
ipdb.runcall(scrapyd.runner.main)
@Markbnj
Markbnj / simple-google-places-wrapper.py
Created Aug 11, 2015
A simple google places wrapper using requests
View simple-google-places-wrapper.py
import requests
import json
GOOGLE_API_KEY = "<A google API key>"
TEXT_SEARCH_TEMPLATE = "https://maps.googleapis.com/maps/api/place/textsearch/json?key={}&query={}"
NEARBY_SEARCH_TEMPLATE = "https://maps.googleapis.com/maps/api/place/nearbysearch/json?key={}&location={},{}&radius={}"
DETAIL_SEARCH_ID_TEMPLATE = "https://maps.googleapis.com/maps/api/place/details/json?key={}&placeid={}"
@Markbnj
Markbnj / geonames-org-wrapper.py
Created Aug 11, 2015
A simple geonames.org wrapper in python
View geonames-org-wrapper.py
import requests
import json
# For information on endpoints and arguments see the geonames
# API documentation at:
#
# http://www.geonames.org/export/web-services.html
class Geonames(object):
@Markbnj
Markbnj / set-proxy.sh
Created Aug 11, 2015
Set http/https proxy address on a console session
View set-proxy.sh
# sets the http/s proxy for the current shell to
# the passed argument, which should be in the form
# http://xxx.xxx.xxx.xxx:xxxx
export http_proxy=$1
export https_proxy=$http_proxy
export no_proxy="localhost,127.0.0.1"
@Markbnj
Markbnj / set-up-phantomjs.py
Created Aug 11, 2015
Set up phantomjs driver for selenium, example
View set-up-phantomjs.py
# An example showing how to set up phantomjs for use with Selenium
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
# These arguments will be passed on the phantomjs command line
# when webdriver launches it. See the reference page at
# http://phantomjs.org/api/command-line.html for more settings.
#
@Markbnj
Markbnj / docker-inject-ec2-host-ip.sh
Created Aug 11, 2015
Inject the ec2 instance private IP into a docker container at start
View docker-inject-ec2-host-ip.sh
# Sometimes you need to get the IP of the ec2 instance a container
# is running on. This script will get the IP from the local metadata
# service and inject it as an environment var.
localip=$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4)
export HOST_IP=$localip
sudo docker run -i -t -e "HOST_IP=$localip" --name="test" -h="test" some-repo
@Markbnj
Markbnj / update-launch-es.sh
Created Aug 11, 2015
Update elasticsearch.yml and launch the service
View update-launch-es.sh
# Script to update elasticsearch.yml and launch the service. In this
# scenario the service is being launched inside a container on ec2
# but the technique is easily generalizable.
#
# Expects the following environment vars to be set inside the container
#
# HOST_IP - the IP address of the host running the container
# AWS_ACCESS_KEY - the AWS account access key
# AWS_SECRET_KEY - the AWS account secret key
# CLUSTER_NAME - the name of the ES cluster to join
@Markbnj
Markbnj / Dockerfile
Created Aug 11, 2015
Simple dockerfile for an Ubuntu-based base image for python services
View Dockerfile
# DockerFile for building a base image for python services
FROM ubuntu:15.04
MAINTAINER Someone <someone@some.where>
# common stuff
RUN echo "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d && \
mkdir /home/root && \
dpkg-reconfigure locales && \
locale-gen en_US en_US.UTF-8 && \
You can’t perform that action at this time.