Skip to content

Instantly share code, notes, and snippets.

View y-ken's full-sized avatar

Y.Kentaro y-ken

62 followers · 3 following
View GitHub Profile
@y-ken
y-ken / acl.yml
Created June 11, 2018 05:06 — forked from danielnorberg/ acl.yml
Digdag secrets
# System secret ACL policy
---
acl:
operators:
# TD
td:
secrets:
- td.*
td_load:
secrets:
@y-ken
y-ken / fluentd_s3.conf
Created October 15, 2017 07:14
FluentdからS3に1日単位のファイルと1時間単位のファイルで書き出したい時
#
<match apache.**>
@type copy
<store>
@type s3
...snip...
buffer_path /var/log/fluent/s3_hourly
time_slice_format %Y%m%d%h
</store>
@y-ken
y-ken / file0.txt
Last active August 29, 2015 14:06
iOSアプリ開発の処方箋 : Unable to find a specification for `AFNetworking (~> 2.0)`. ref: http://qiita.com/y-ken/items/2e4c3fed1d29e8f5d6ba
$ rake
================================================================================
A new version of RubyMotion is available. Run `sudo motion update' to upgrade.
================================================================================
rake aborted!
Pod::Informative: [!] Unable to find a specification for `AFNetworking (~> 2.0)`.
/Users/y-ken/Documents/htdocs/motion-octofeed/vendor/bundle/ruby/2.1.0/gems/cocoapods-0.23.0/lib/cocoapods/resolver.rb:195:in `find_cached_set'
/Users/y-ken/Documents/htdocs/motion-octofeed/vendor/bundle/ruby/2.1.0/gems/cocoapods-0.23.0/lib/cocoapods/resolver.rb:153:in `block (2 levels) in find_dependency_specs'
@y-ken
y-ken / file0.conf
Last active December 18, 2015 01:50
FluentdでApacheのエラーログを収集する正規表現 ref: http://qiita.com/y-ken/items/b302b73d0721162ff401
<source>
type tail
path /var/log/httpd/error_log
format /^(\[(?<time>[^\]]*)\] \[(?<level>[^\]]+)\] (\[client (?<host>[^\]]*)\] )?(?<message>.*)|(?<message>.*))$/
time_format %a %b %d %H:%M:%S %Y
tag apache.error
pos_file /var/log/td-agent/apache_error.pos
</source>
# 送り先を Fluentd の標準ログへ出力します
@y-ken
y-ken / readme.md
Last active August 29, 2015 14:04
FluentdでApache2のエラーログを収集する正規表現
@y-ken
y-ken / how-to-call-fluentd-role-name.md
Last active August 29, 2015 14:03
I'd like to make a common name for each Fluentd role.

Name

  • Fluentd Cluster ... whole of fluentd system
  • Fluentd Node ... type of role
  • Fluentd Instance ... each fluentd process

Role

  1. Forwarder ... forwarding log to Aggregator node
  2. Aggregator ... Aggregate log from forwarder node
@y-ken
y-ken / bench_out_parser.rb
Last active August 29, 2015 14:02 — forked from sonots/bench_out_parser.rb
fluent-plugin-parser を用いてベンチマークするサンプルです
require_relative './test/helper'
require 'benchmark'
Fluent::Test.setup
def create_driver(config, tag = 'foo.bar')
Fluent::Test::OutputTestDriver.new(Fluent::ParserOutput, tag).configure(config)
end
# setup
time = Time.now.to_i
@y-ken
y-ken / fluentd-v0.conf
Last active August 29, 2015 14:02
How do you plan for using Filter for Fluentd v1?
<match apache.access>
type record_reformer
enable_ruby false
tag reform.apache.access
<record>
hostname ${hostname}
</record>
</match>
<match reform.apache.access>
@y-ken
y-ken / 01_how-to-pivot-facet-in-elasticsearch-like-solr4.md
Last active August 29, 2015 14:01
How to do "Pivot Faceting" like Solr-4.0 in elasticsearch-1.1.1 or later version?

次の記事のような集計をelasticsearchで実現するクエリが分からず困っております。

期待する結果は、SQLで言う所の次のクエリに相応するものです。

SELECT `都道府県`, `物件種別`, COUNT(*) FROM sample_data GROUP BY `都道府県`, `物件種別`;
@y-ken
y-ken / docker_aufs_benchmark.md
Created February 20, 2014 07:31
bonnie++を使って、Docker(LXC+aufs)のディスクI/O性能ベンチマークを行いました。