Skip to content

Instantly share code, notes, and snippets.

View ivanlei's full-sized avatar

Ivan Leichtling ivanlei

44 followers · 10 following
  • Agoric
View GitHub Profile
@ivanlei
ivanlei / keybase.md
Created March 8, 2017 18:46
keybase.md

Keybase proof

I hereby claim:

  • I am ivanlei on github.
  • I am ivanlei (https://keybase.io/ivanlei) on keybase.
  • I have a public key whose fingerprint is ED33 D65D 38F7 4B7B 53B6 34D3 3C08 7CBB 189C C81C

To claim this, I am signing this object:

@ivanlei
ivanlei / crash.log
Created March 5, 2017 15:13
terraform crash.log
2017/03/05 10:12:56 [INFO] Terraform version: 0.8.8 403a86dc557fae52f8e39676b11e1e4356b7d1a2
2017/03/05 10:12:56 [INFO] CLI args: []string{"/Volumes/dat/Users/ivanlei/bin/terraform", "plan"}
2017/03/05 10:12:56 [DEBUG] Detected home directory from env var: /Volumes/dat/Users/ivanlei
2017/03/05 10:12:56 [DEBUG] Detected home directory from env var: /Volumes/dat/Users/ivanlei
2017/03/05 10:12:56 [DEBUG] Attempting to open CLI config file: /Volumes/dat/Users/ivanlei/.terraformrc
2017/03/05 10:12:56 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2017/03/05 10:12:56 [DEBUG] Detected home directory from env var: /Volumes/dat/Users/ivanlei
2017/03/05 10:12:56 [DEBUG] plugin: waiting for all plugin processes to complete...
panic: runtime error: index out of range
@ivanlei
ivanlei / debug spew
Created March 5, 2017 15:13
$ terraform plan
2017/03/05 10:12:56 [INFO] Terraform version: 0.8.8 403a86dc557fae52f8e39676b11e1e4356b7d1a2
2017/03/05 10:12:56 [INFO] CLI args: []string{"/Volumes/dat/Users/ivanlei/bin/terraform", "plan"}
2017/03/05 10:12:56 [DEBUG] Detected home directory from env var: /Volumes/dat/Users/ivanlei
2017/03/05 10:12:56 [DEBUG] Detected home directory from env var: /Volumes/dat/Users/ivanlei
2017/03/05 10:12:56 [DEBUG] Attempting to open CLI config file: /Volumes/dat/Users/ivanlei/.terraformrc
2017/03/05 10:12:56 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2017/03/05 10:12:56 [DEBUG] Detected home directory from env var: /Volumes/dat/Users/ivanlei
2017/03/05 10:12:56 [DEBUG] plugin: waiting for all plugin processes to complete...
panic: runtime error: index out of range
@ivanlei
ivanlei / crash spew
Created March 5, 2017 15:11
terraform crash
```shell
$ terraform plan
2017/03/05 10:11:05 [INFO] Terraform version: 0.8.8 403a86dc557fae52f8e39676b11e1e4356b7d1a2
2017/03/05 10:11:05 [INFO] CLI args: []string{"/Volumes/dat/Users/ivanlei/bin/terraform", "plan"}
2017/03/05 10:11:05 [DEBUG] Detected home directory from env var: /Volumes/dat/Users/ivanlei
2017/03/05 10:11:05 [DEBUG] Detected home directory from env var: /Volumes/dat/Users/ivanlei
2017/03/05 10:11:05 [DEBUG] Attempting to open CLI config file: /Volumes/dat/Users/ivanlei/.terraformrc
2017/03/05 10:11:05 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2017/03/05 10:11:05 [DEBUG] Detected home directory from env var: /Volumes/dat/Users/ivanlei
2017/03/05 10:11:05 [DEBUG] plugin: waiting for all plugin processes to complete...
@ivanlei
ivanlei / debug spew
Created March 5, 2017 15:02
terraform crash debug spew
$ export TF_LOG=TRACE
$ terraform plan
2017/03/05 10:01:09 [INFO] Terraform version: 0.8.8 403a86dc557fae52f8e39676b11e1e4356b7d1a2
2017/03/05 10:01:09 [INFO] CLI args: []string{"/Volumes/dat/Users/ivanlei/bin/terraform", "plan"}
2017/03/05 10:01:09 [DEBUG] Detected home directory from env var: /Volumes/dat/Users/ivanlei
2017/03/05 10:01:09 [DEBUG] Detected home directory from env var: /Volumes/dat/Users/ivanlei
2017/03/05 10:01:09 [DEBUG] Attempting to open CLI config file: /Volumes/dat/Users/ivanlei/.terraformrc
2017/03/05 10:01:09 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2017/03/05 10:01:09 [DEBUG] Detected home directory from env var: /Volumes/dat/Users/ivanlei
2017/03/05 10:01:09 [DEBUG] plugin: waiting for all plugin processes to complete...
@ivanlei
ivanlei / crash.log
Created March 5, 2017 14:59
terraform crash
2017/03/05 09:59:20 [INFO] Terraform version: 0.8.8 403a86dc557fae52f8e39676b11e1e4356b7d1a2
2017/03/05 09:59:20 [INFO] CLI args: []string{"/Volumes/dat/Users/ivanlei/bin/terraform", "plan"}
2017/03/05 09:59:20 [DEBUG] Detected home directory from env var: /Volumes/dat/Users/ivanlei
2017/03/05 09:59:20 [DEBUG] Detected home directory from env var: /Volumes/dat/Users/ivanlei
2017/03/05 09:59:20 [DEBUG] Attempting to open CLI config file: /Volumes/dat/Users/ivanlei/.terraformrc
2017/03/05 09:59:20 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2017/03/05 09:59:20 [DEBUG] Detected home directory from env var: /Volumes/dat/Users/ivanlei
2017/03/05 09:59:20 [DEBUG] plugin: waiting for all plugin processes to complete...
panic: runtime error: index out of range
@ivanlei
ivanlei / crash.log
Created March 5, 2017 14:49
terraform crash
2017/03/05 09:48:24 [INFO] Terraform version: 0.8.8 403a86dc557fae52f8e39676b11e1e4356b7d1a2
2017/03/05 09:48:24 [INFO] CLI args: []string{"/Volumes/dat/Users/ivanlei/bin/terraform", "plan"}
2017/03/05 09:48:24 [DEBUG] Detected home directory from env var: /Volumes/dat/Users/ivanlei
2017/03/05 09:48:24 [DEBUG] Detected home directory from env var: /Volumes/dat/Users/ivanlei
2017/03/05 09:48:24 [DEBUG] Attempting to open CLI config file: /Volumes/dat/Users/ivanlei/.terraformrc
2017/03/05 09:48:24 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2017/03/05 09:48:24 [DEBUG] Detected home directory from env var: /Volumes/dat/Users/ivanlei
2017/03/05 09:48:24 [DEBUG] plugin: waiting for all plugin processes to complete...
panic: runtime error: index out of range
@ivanlei
ivanlei / gist:f81630084d2be0f22362
Last active August 2, 2016 07:17
S3->SNS->SQS->Logstash

S3 support SNS notifications for new objects. These SNS notifications can fan out to SQS queues.

Logstash can read from inputs including:

but neither of these are sufficient:

  • S3 input has very low performance in attempting to read for a bucket with a high number of writes. The S3 reader lands up spending most of its time listing the bucket contents vs. reading objects.
  • SQS input works well. It works well with a single logstash process or a cluster of multiple processes. However, the SQS input doesn't understand the format of an SNS notification object for S3 changes.
@ivanlei
ivanlei / gist:f7131796abdae2e3b087
Last active September 16, 2015 19:01
keybase.md
### Keybase proof
I hereby claim:
* I am ivanlei on github.
* I am ivanlei (https://keybase.io/ivanlei) on keybase.
* I have a public key whose fingerprint is A809 2A4A FC1D 5E1D 346F 4A5C 0827 0A86 9915 D0B8
To claim this, I am signing this object:
@ivanlei
ivanlei / boto keyring setup
Last active January 2, 2016 15:29
This is a script takes AWS credentials csv file containing an AWS keypair and: * Uses keyring package to store the Secret Access Key * Outputs the proper config lines for .boto file to use the keyring
# -*- coding: utf-8 -*-
#
# This is a script takes AWS credentials csv file containing an AWS keypair and:
# * Uses keyring package to store the Secret Access Key
# * Outputs the proper config lines for .boto file to use the keyring
#
import sys
import keyring
import optparse