bfleming-ciena

"""
 Requeriments:

     $ sudo pip install boto dnspython

 Edit ~/.boto to use your AWS credentials

"""
import time
import sys

bfleming-ciena

#!/usr/bin/env python
import boto
from boto.route53.record import ResourceRecordSets
import logging

conn = boto.connect_ec2()
DNS_EXCLUSION_TAG = 'ExcludeFromDNS'   # If this tag exists on an instance, no DNS values will be populated
DNS_TAGS = ['ShortName', 'Name']       # This is the list of instance tags we want to populate DNS entries from
DNS_SUFFIX = 'YOUR_SUBDOMAIN_HERE'     # Suffix under which to create DNS records
ROUTE53_ZONE_ID = 'YOUR_ZONE_ID_HERE'  # The zone id from route53 of the zone we will be adding these entries under

bfleming-ciena

Checking if required software is installed.

Vagrant 1.6 or newer installed.
Virtualbox 4.2 or newer installed.

/home/brian/.panamax/desktop: line 208: .version: No such file or directory

Creating a new CoreOS VM...
Downloading images
######################################################################## 100.0%

bfleming-ciena

// Use Gists to store code you would like to remember later on
console.log(window); // log the "window" object to the console

bfleming-ciena

Packages

• lxml - Pythonic binding for the C libraries libxml2 and libxslt.
• boto - Python interface to Amazon Web Services
• Django - Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design.
• Fabric - Library and command-line tool for streamlining the use of SSH for application deployment or systems administration task.
• PyMongo - Tools for working with MongoDB, and is the recommended way to work with MongoDB from Python.
• Celery - Task queue to distribute work across threads or machines.
• pytz - pytz brings the Olson tz database into Python. This library allows accurate and cross platform timezone calculations using Python 2.4 or higher.

Guides

bfleming-ciena

Producer

Setup
bin/kafka-topics.sh --zookeeper esv4-hcl197.grid.linkedin.com:2181 --create --topic test-rep-one --partitions 6 --replication-factor 1
bin/kafka-topics.sh --zookeeper esv4-hcl197.grid.linkedin.com:2181 --create --topic test --partitions 6 --replication-factor 3

Single thread, no replication

bin/kafka-run-class.sh org.apache.kafka.clients.tools.ProducerPerformance test7 50000000 100 -1 acks=1 bootstrap.servers=esv4-hcl198.grid.linkedin.com:9092 buffer.memory=67108864 batch.size=8196

bfleming-ciena

servers-spec=
zookeeper-data-directory=/zookeeper/data
zookeeper-install-directory=/zookeeper/*
zookeeper-log-directory=
log-index-directory=/zookeeper/log-index
cleanup-period-ms=200000
check-ms=2000
backup-period-ms=600000
client-port=2181
cleanup-max-files=20

bfleming-ciena

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "S3Only",
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [

bfleming-ciena

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ProtectTaggedVPC",
      "Effect": "Deny",
      "Action": [
        "ec2:DeleteVpc"
      ],
      "Resource": [

bfleming-ciena

{
  "Version": "2012-10-17",
  "Statement": {
    "Effect": "Allow",
    "Action": "sts:AssumeRole",
    "Resource": "arn:aws:iam::acct:role/S3-Admin"
  }
}