hengkiardo

AWS S3 bucket and IAM policy recipes

• Recipes
  • Anonymous GET access
  • Anonymous GET access - match HTTP referrer
  • Full access for specific IAM user/role
  • GET/PUT/DELETE access to specific path within a bucket
  • Restricted LIST & PUT/DELETE access to specific path within a bucket
  • Full access (and S3 console) for specific IAM users
  • Bucket and object delete deny
  • CloudTrail log receive

hengkiardo

CloudFormation example for API Gateway integration to Lambda function

Template that will create the following:

• API Gateway endpoint:
  • A single root method, accepting POST requests only with Lambda proxy integration to a function.
• In-line Lambda function echoing back requesting users IP address to API Gateway requests:
  • IAM role for Lambda allowing CloudWatch logs access.
  • Permissions for Lambda that allow API Gateway endpoint to successfully invoke function.
• CloudWatch logs group for Lambda, with 90 day log retention.

After standing up the template, you should be able to curl a POST request to the URL listed as the apiGatewayInvokeURL output value.

hengkiardo

Linux - create "Default (Linux).sublime-mousemap" in ~/.config/sublime-text-3/Packages/User
Mac - create "Default (OSX).sublime-mousemap" in ~/Library/Application Support/Sublime Text 3/Packages/User
Win - create "Default (Windows).sublime-mousemap" in %appdata%\Sublime Text 3\Packages\User

[
    {
        "button": "button1", 
        "count": 1, 
        "modifiers": ["ctrl"],
        "press_command": "drag_select",

hengkiardo

Installing Cassandra on Mac OS X

Caution!

Version Number might change!! The versions of all softwares mentioned here, including Cassandra will change as newer versions are launched.

Install Homebrew

Homebrew is a great little package manager for OS X. If you haven't already, installing it is pretty easy:

hengkiardo

#MongoDB 3.2.x Replica Sets on AWS EC2 A MongoDB replica set provides a mechanism to allow for a reliable database services. The basic replica set consists of three servers, a primary, a secondary and an arbitrator. The primary and secondary both hold a copy of the data. The arbitrator is normally a low spec server which just monitors the other servers and help with the failover process. In production, there can be more than three servers.

To setup mongo as a replica set on Amazon Web Services EC2 you need to first setup a security group with ssh on port 22 and mongodb on port 27017. You then need to create three servers. Select Ubuntu 14.04 LTS x64 and a micro (or bigger depending on your database size, ideally you should have enough memory to match your database size) instance for the primary and secondary and a nano instance for the arbitrator.

##Adjust the File System on each Server The operating system by default will update the last access time on a file. In a high data throughput database application

hengkiardo

Before starting please have a mongo replica set initiated. I have mongo running at localhost:27170 and elasticsearch at localhost:9200

The test dataset I used is available on the mongodb website.

I am using MongoDB 2.6 and Elasticsearch 2.2.0 on a Windows machine.

Here I have tested three cases:

• An index already exists, mc-test-index in this case

hengkiardo

storage:
    dbPath: "./data"
    directoryPerDB: true
    journal:
        enabled: true
    engine: "wiredTiger"
    wiredTiger:
        engineConfig: 
            cacheSizeGB: 8 
            journalCompressor: zlib

hengkiardo

#!/usr/bin/env sh

# checks to see if running 
launchctl list | grep mongo
 
launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.mongodb.plist
launchctl remove homebrew.mxcl.mongodb

pkill -f mongod

hengkiardo

// mongoose 4.3.x
var mongoose = require('mongoose');
 
/* 
 * Mongoose by default sets the auto_reconnect option to true.
 * We recommend setting socket options at both the server and replica set level.
 * We recommend a 30 second connection timeout because it allows for 
 * plenty of time in most operating environments.
 */
var options = { server: { socketOptions: { keepAlive: 1, connectTimeoutMS: 30000 } }, 

hengkiardo

#!/bin/sh

# fix permissions (wrong if docker mounted volume)
chown -R elasticsearch:elasticsearch /var/lib/elasticsearch

# now switch to elasticsearch user and run in foreground
echo Starting: /usr/share/elasticsearch/bin/elasticsearch -Des.default.config=$CONF_FILE -Des.default.path.home=/usr/share/elasticsearch -Des.default.path.logs=$LOG_DIR -Des.default.path.data=/var/lib/elasticsearch -Des.default.path.work=/tmp/elasticsearch -Des.default.path.conf=$CONF_DIR $@
su elasticsearch -s /bin/sh -c "/usr/share/elasticsearch/bin/elasticsearch -Des.default.config=$CONF_FILE -Des.default.path.home=/usr/share/elasticsearch -Des.default.path.logs=$LOG_DIR -Des.default.path.data=/var/lib/elasticsearch -Des.default.path.work=/tmp/elasticsearch -Des.default.path.conf=$CONF_DIR $@"