Skip to content

Instantly share code, notes, and snippets.

View arthurpham's full-sized avatar

Arthur Pham arthurpham

32 followers · 20 following
View GitHub Profile
@arthurpham
arthurpham / gist:8391944
Created January 12, 2014 23:07 — forked from c93614/gist:3848993
// requires
var utils = require('utils');
var casper = require('casper').create()
var casper = require('casper').create({
verbose: true,
logLevel: "debug"
});
// setup globals
var email = casper.cli.options['email'] || 'REPLACE THIS EMAIL';
@arthurpham
arthurpham / Vagrantfile
Last active August 29, 2015 14:01
Setup a hadoop cluster with Vagrant. See http://blog.cloudera.com/blog/2013/04/how-to-use-vagrant-to-set-up-a-virtual-hadoop-cluster/
# -*- mode: ruby -*-
# vi: set ft=ruby :
$master_script = <<SCRIPT
#!/bin/bash
cat > /etc/hosts <<EOF
127.0.0.1 localhost
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
@arthurpham
arthurpham / docker-compose.yml
Last active November 13, 2016 15:29
scrapy-cluster-docker-cloud-stack
crawler:
environment:
- REDIS_PORT=6379
image: 'istresearch/scrapy-cluster:crawler-dev-alpine'
links:
- kafka
- redis
- zookeeper
restart: always
target_num_containers: 3
@arthurpham
arthurpham / sum.cpp
Last active November 11, 2016 22:05
Loop
#include <iostream>
#include <string>
#include <ctime>
#define N 1000
double array_sum(double a[N][N])
{
int i,j;
double s;
@arthurpham
arthurpham / remove-docker-containers.md
Last active August 28, 2018 14:52
How to remove unused Docker containers and images on windows
  1. Delete all images

FOR /f "tokens=*" %i IN ('docker images -q -f "dangling=true"') DO docker rmi %i

  1. Delete all containers

FOR /f "tokens=*" %i IN ('docker ps -aq') DO docker rm %i

@arthurpham
arthurpham / docker-compose.yml
Last active November 22, 2016 03:11
Docker compose for Rancher
nginx-proxy:
image: jwilder/nginx-proxy:latest
ports:
- "80:80"
- "443:443"
volumes:
- '/home/docker/nginx-proxy/ssl:/etc/nginx/certs:ro'
- '/etc/nginx/vhost.d'
- '/usr/share/nginx/html'
- '/var/run/docker.sock:/tmp/docker.sock:ro'
@arthurpham
arthurpham / docker-compose.yml
Last active February 6, 2018 16:59
For Rancher # This compose file stands up Scrapy Cluster with an # associated ELK Stack. You should run a few crawls and then import the # `export.json` file into your Kibana objects
logstash-crawler:
labels:
io.rancher.container.pull_image: always
io.rancher.scheduler.affinity:container_label_soft_ne: io.rancher.stack_service.name=$$$${stack_name}/$$$${service_name}
io.rancher.scheduler.affinity:host_label: role=crawler
command:
- -f
- /etc/logstash/conf.d/logstash.conf
image: arthurpham/logstash-scrapy-cluster-docker:latest
links:
@arthurpham
arthurpham / rancher-compose.yml
Last active November 21, 2016 20:05
logstash-crawler:
scale: 12
redis:
scale: 1
health_check:
port: 6379
interval: 2000
initializing_timeout: 60000
unhealthy_threshold: 3
strategy: recreate
@arthurpham
arthurpham / lftp-mirror
Created April 12, 2017 03:27 — forked from bensbrowning/lftp-mirror
lftp mirror script for cron
#!/bin/bash
login="user"
pass="pass"
host="foo.bar.com"
remote_dir="/path/on/foo/bar/"
local_dir="/home/you/foobar-mirror/"
log_file="/tmp/lftp-mirror.log"
@arthurpham
arthurpham / _Aggregate_Barchart_Options_Data_.py
Created April 16, 2017 23:39 — forked from BlackArbsCEO/_Aggregate_Barchart_Options_Data_.py
# -*- coding: utf-8 -*-
"""
Created on Fri Sep 2 13:26:48 2016
@author: Brian Christopher, CFA [Blackarbs LLC]
"""
import time
import pandas as pd
import numpy as np