minhchuduc

#!/usr/bin/env bash
if [ $# -eq 0 ]; then
	DEFAULT_USER=clouduser
else
	DEFAULT_USER=$1
fi
echo $DEFAULT_USER

if [ -f /bin/tempfile ]; then # is Ubuntu/Debian
	SSH_KEYFILE=`tempfile`

minhchuduc

Info

This guide sets up a non-clustered Nutch crawler, which stores its data via HBase. We will not learn how to setup Hadoop et al., but just the bare minimum to crawl and index websites on a single machine.

Terms

• Nutch - the crawler (fetches and parses websites)
• HBase - filesystem storage for Nutch (Hadoop component, basically)

minhchuduc

A few show tricks to find slow queries in mongodb

Enable profiling

First, you have to enable profiling

> db.setProfilingLevel(1)

Now let it run for a while. It collects the slow queries ( > 100ms) into a capped collections, so queries go in and if it's full, old queries go out, so don't be surprised that it's a moving target...

minhchuduc

Deploying Kubernetes Clusters with Terraform and CoreOS

Kubernetes is complex software and setting up a new cluster can be difficult. While there are easy approaches like Google Container Engine, you may want to customize your cluster in various ways, or simply understand how it all works. A great way to do this is to define your cluster as code using Terraform. In this talk, you'll learn how to use Terraform to deploy Kubernetes on CoreOS and EC2.

Takeaway

Terraform and CoreOS are a great combination for deploying Kubernetes clusters.

Abstract

minhchuduc

Set up screen

1. docker run --detach --interactive --tty --privileged --net=host fedora /bin/bash
2. docker attach $CONTAINER_ID
3. yum install screen

Create screen session

1. docker attach $CONTAINER_ID
2. screen -S test
3. Detach from screen

minhchuduc

tmux shortcuts & cheatsheet

start new:

tmux

start new with session name:

tmux new -s myname

minhchuduc

import logging
import uuid
import time

from mesos.interface import Scheduler
from mesos.native import MesosSchedulerDriver
from mesos.interface import mesos_pb2

logging.basicConfig(level=logging.INFO)

minhchuduc

# Ubuntu 14.04 don't have nsenter - the straight forward way required me to install build tools and etc.
# I preferred to keep the system clean and install nsenter in a container and then copy the command to the host
# Note - its also possible to run nsenter from a container (didn't tried) https://github.com/jpetazzo/nsenter

# start a container
docker run --name nsenter -it ubuntu:14.04 bash

## in the docker
apt-get update
apt-get install git build-essential libncurses5-dev libslang2-dev gettext zlib1g-dev libselinux1-dev debhelper lsb-release pkg-config po-debconf autoconf automake autopoint libtool

minhchuduc

#
# Enable CEPH repos as described at http://ceph.com/docs/master/install/get-packages/#rpm
# Install ceph-deploy package
#

export DATA_DEV=sdb
export JRNL_DEV=sdc
export FS_TYPE=xfs
export CEPH_S_NODE=ceph-s
export CEPH_RELEASE=giant

minhchuduc

#! /bin/sh

### Must have Gcloud sdk installed and configured

###Create a micro instance as ansible master
gcloud compute --project $PROJECT_NAME instances create "ansible" --zone "us-central1-b" --machine-type "f1-micro" --network "default" --maintenance-policy "MIGRATE" --scopes "https://www.googleapis.com/auth/userinfo.email" "https://www.googleapis.com/auth/compute" "https://www.googleapis.com/auth/devstorage.full_control" --tags "http-server" "https-server" --no-boot-disk-auto-delete

###or a centos like in the tutorial
gcloud compute --project $PROJECT_NAME instances create "ansible-master" --zone "us-central1-b" --machine-type "g1-small" --network "default" --maintenance-policy "MIGRATE" --scopes "https://www.googleapis.com/auth/userinfo.email" "https://www.googleapis.com/auth/compute" "https://www.googleapis.com/auth/devstorage.full_control" --tags "http-server" "https-server" --image "https://www.googleapis.com/compute/v1/projects/centos-cloud/global/images/centos-7-v20140926" --no-boot-disk-auto-de