Skip to content

Instantly share code, notes, and snippets.

View fred's full-sized avatar

Frederico Araujo fred

  • Thailand
View GitHub Profile
@fred
fred / reindex_cloudtrail.rb
Created August 26, 2015 08:29
Reindex Cloudtrail logs in ElasticSearch, for Kibana usage
require 'aws-sdk'
require 'json'
require 'uri'
# Your current cloudtrail S3 preffix
BASE="AWSLogs/xxxxxxxxxxxxxxx/CloudTrail/ap-southeast-1"
BUCKET_NAME="bucket_name"
AWS_ACCESS_ID = 'XXXXXXXXXXXXXX'
AWS_SECRET_KEY = 'xxxxxxxxxxxxxxxxxxxxx'
  • Zero downtime during upgrades and scaling out system (such as zero seconds downtime, while app still runs)

  • Highly Scalable with Terabytes of data with high performance and low cost with shards

  • Automatic Replica Set member take-over in case of failing primary, with automatic recovery for returning hosts.

  • Automatic promotion to primary and de promotion for any member of a replica set based on performance and latency.

  • Journaling, allows safe recovery in case of hard-disk failure.

  • Strong consistency, "Afford to loose data" in mongoDB is myth. Data is not lost, it's only not persisted due to failures on the developers code.

  • When Sharding is not feasible, different Documents (rails models) can be on different Replica sets (different database or set) altogether, since there are no joins in Mongo.

  • Background indexes.

/Users/fred/.rvm/gems/ruby-2.0.0-p353/gems/rest-2.7.1/lib/rest/wrappers/base_wrapper.rb:21:in `rescue in post_file': HTTP 400 Error: In JSON, unexpected end of JSON input at position 0. (Rest::Wrappers::RestClientExceptionWrapper)
from /Users/fred/.rvm/gems/ruby-2.0.0-p353/gems/rest-2.7.1/lib/rest/wrappers/base_wrapper.rb:6:in `post_file'
from /Users/fred/.rvm/gems/ruby-2.0.0-p353/gems/rest-2.7.1/lib/rest/client.rb:219:in `block in post_file'
from /Users/fred/.rvm/gems/ruby-2.0.0-p353/gems/rest-2.7.1/lib/rest/client.rb:142:in `perform_op'
from /Users/fred/.rvm/gems/ruby-2.0.0-p353/gems/rest-2.7.1/lib/rest/client.rb:218:in `post_file'
from /Users/fred/.rvm/gems/ruby-2.0.0-p353/gems/iron_core-1.0.3/lib/iron_core/client.rb:258:in `post_file'
from /Users/fred/.rvm/gems/ruby-2.0.0-p353/gems/iron_worker_ng-1.2.0/lib/iron_worker_ng/api_client.rb:43:in `codes_create'
from /Users/fred/.rvm/gems/ruby-2.0.0-p353/gems/iron_worker_ng-1.2.0/lib/iron_worker_ng/client.rb:112:in `codes_create'
from /Users/fred/.rvm/ge
@fred
fred / bashrc
Created March 18, 2014 05:44
show git branch in bash prompt, nicer bash prompt
# add to home .bashrc
if [[ $- != *i* ]] ; then
# Shell is non-interactive. Be done now!
return
fi
safe_term=${TERM//[^[:alnum:]]/?}
match_lhs=""
[[ -f ~/.dir_colors ]] && match_lhs="${match_lhs}$(<~/.dir_colors)"
namespace :workers do
namespace :clockwork do
desc "Stop clockwork"
task :stop do
on roles(:app) do
within release_path do
with rails_env: fetch(:rails_env) do
execute :bundle, :exec, :clockworkd, "-c lib/clockwork.rb --pid-dir=#{cw_pid_dir} --log-dir=#{cw_log_dir} stop"
end
end
@fred
fred / _search.html.haml
Created June 21, 2014 06:44
Activeadmin MongoID with Fulltext Search on Sidebar
/ file: app/views/admin/users/_search.html.haml
#panel_contents
= form_for(admin_users_path, method: :get) do |f|
.filter_form_field.filter_string
= label_tag :query, "Full Text Search"
= text_field_tag :query, params[:query], { placeholder: "Keyword to search" }
.buttons
= submit_tag "Search"
_bundler_complete()
{
if [[ ! `which bundle` ]]; then
return
fi
local cur prev commands
commands="help install update package exec config check list show outdated console open viz init gem platform"
cur="${COMP_WORDS[COMP_CWORD]}"
prev="${COMP_WORDS[COMP_CWORD-1]}"
@fred
fred / config.fish
Created March 24, 2015 07:16
My ~/.config/fish/config.fish
# ~/.config/fish/config.fish
# for mate and subl
set PATH $HOME/bin $PATH
set PATH /usr/local/bin /usr/local/sbin $PATH
# RBENV
set PATH $HOME/.rbenv/bin $PATH
set PATH $HOME/.rbenv/shims $PATH
rbenv rehash >/dev/null ^&1
@fred
fred / omise_export.rb
Last active August 29, 2015 14:20
exporting charges and refunds to a CSV format files (charges.csv, refunds.csv)
#!/usr/bin/env ruby
# how to run:
# ruby ./omise_export.rb
require 'date'
require 'json'
# CHANGE the next 4 variables
# just be aware of timezone overlapping between months, Omise will parse dates as UTC time
skey = 'your secret key'
@fred
fred / omise_export.rb
Last active August 29, 2015 14:21
Full Transactions in CSV format
#!/usr/bin/env ruby
# how to run:
# ruby ./omise_export.rb
# Works on Mac OS X
# It will generate (and overwrite) a csv file with name full_transactions.csv
require 'date'
require 'json'
# Update your valid Live Secret Key, this is only a test key: