Skip to content

Instantly share code, notes, and snippets.

@y-ken
y-ken / acl.yml
Created Jun 11, 2018 — forked from danielnorberg/ acl.yml
Digdag secrets
View acl.yml
# System secret ACL policy
---
acl:
operators:
# TD
td:
secrets:
- td.*
td_load:
secrets:
@y-ken
y-ken / fluentd_s3.conf
Created Oct 15, 2017
FluentdからS3に1日単位のファイルと1時間単位のファイルで書き出したい時
View fluentd_s3.conf
#
<match apache.**>
@type copy
<store>
@type s3
...snip...
buffer_path /var/log/fluent/s3_hourly
time_slice_format %Y%m%d%h
</store>
@y-ken
y-ken / file0.txt
Last active Aug 29, 2015
iOSアプリ開発の処方箋 : Unable to find a specification for `AFNetworking (~> 2.0)`. ref: http://qiita.com/y-ken/items/2e4c3fed1d29e8f5d6ba
View file0.txt
$ rake
================================================================================
A new version of RubyMotion is available. Run `sudo motion update' to upgrade.
================================================================================
rake aborted!
Pod::Informative: [!] Unable to find a specification for `AFNetworking (~> 2.0)`.
/Users/y-ken/Documents/htdocs/motion-octofeed/vendor/bundle/ruby/2.1.0/gems/cocoapods-0.23.0/lib/cocoapods/resolver.rb:195:in `find_cached_set'
/Users/y-ken/Documents/htdocs/motion-octofeed/vendor/bundle/ruby/2.1.0/gems/cocoapods-0.23.0/lib/cocoapods/resolver.rb:153:in `block (2 levels) in find_dependency_specs'
@y-ken
y-ken / file0.conf
Last active Dec 18, 2015
FluentdでApacheのエラーログを収集する正規表現 ref: http://qiita.com/y-ken/items/b302b73d0721162ff401
View file0.conf
<source>
type tail
path /var/log/httpd/error_log
format /^(\[(?<time>[^\]]*)\] \[(?<level>[^\]]+)\] (\[client (?<host>[^\]]*)\] )?(?<message>.*)|(?<message>.*))$/
time_format %a %b %d %H:%M:%S %Y
tag apache.error
pos_file /var/log/td-agent/apache_error.pos
</source>
# 送り先を Fluentd の標準ログへ出力します
@y-ken
y-ken / readme.md
Last active Aug 29, 2015
FluentdでApache2のエラーログを収集する正規表現
@y-ken
y-ken / how-to-call-fluentd-role-name.md
Last active Aug 29, 2015
I'd like to make a common name for each Fluentd role.
View how-to-call-fluentd-role-name.md

Name

  • Fluentd Cluster ... whole of fluentd system
  • Fluentd Node ... type of role
  • Fluentd Instance ... each fluentd process

Role

  1. Forwarder ... forwarding log to Aggregator node
  2. Aggregator ... Aggregate log from forwarder node
@y-ken
y-ken / bench_out_parser.rb
Last active Aug 29, 2015 — forked from sonots/bench_out_parser.rb
fluent-plugin-parser を用いてベンチマークするサンプルです
View bench_out_parser.rb
require_relative './test/helper'
require 'benchmark'
Fluent::Test.setup
def create_driver(config, tag = 'foo.bar')
Fluent::Test::OutputTestDriver.new(Fluent::ParserOutput, tag).configure(config)
end
# setup
time = Time.now.to_i
@y-ken
y-ken / fluentd-v0.conf
Last active Aug 29, 2015
How do you plan for using Filter for Fluentd v1?
View fluentd-v0.conf
<match apache.access>
type record_reformer
enable_ruby false
tag reform.apache.access
<record>
hostname ${hostname}
</record>
</match>
<match reform.apache.access>
@y-ken
y-ken / 01_how-to-pivot-facet-in-elasticsearch-like-solr4.md
Last active Aug 29, 2015
How to do "Pivot Faceting" like Solr-4.0 in elasticsearch-1.1.1 or later version?
View 01_how-to-pivot-facet-in-elasticsearch-like-solr4.md

次の記事のような集計をelasticsearchで実現するクエリが分からず困っております。

期待する結果は、SQLで言う所の次のクエリに相応するものです。

SELECT `都道府県`, `物件種別`, COUNT(*) FROM sample_data GROUP BY `都道府県`, `物件種別`;
@y-ken
y-ken / docker_aufs_benchmark.md
Created Feb 20, 2014
bonnie++を使って、Docker(LXC+aufs)のディスクI/O性能ベンチマークを行いました。
View docker_aufs_benchmark.md