Skip to content

Instantly share code, notes, and snippets.

View toyama0919's full-sized avatar

Toyama Hiroshi toyama0919

29 followers · 18 following
View GitHub Profile
@toyama0919
toyama0919 / autoscaling.md
Last active January 28, 2023 06:10

Autoscaling色々

はじめに

  • autoscaling全般についてはなします
  • terraformのコードがいっぱい出てきます

はなすこと

  • application autoscaling
@toyama0919
toyama0919 / gist:5a62890cc512cdcde06f4596542992df
Last active July 23, 2018 16:11
Embulk-0.8.34をJava9で実行

embulk previewの場合

Running an external shell script: embulk_init.sh
2017-09-26 12:49:28.635 +0900: Embulk v0.8.34
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$2 (file:/Users/toyama-h/bin/embulk) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
@toyama0919
toyama0919 / fluentd
Last active December 19, 2016 03:57
<source>
type tail
format none
path /var/log/presto/presto.log
pos_file /var/log/td-agent/presto.pos
tag presto
</source>
<filter presto>
@type record_transformer
@toyama0919
toyama0919 / embulk_error.md
Last active July 29, 2016 00:52

発生するエラーはこれ。

2016-07-29 09:34:28.957 +0900: Embulk v0.8.11
2016-07-29 09:34:30.302 +0900 [INFO] (0001:transaction): Loaded plugin embulk-input-s3 (0.2.8)
2016-07-29 09:34:31.190 +0900 [INFO] (0001:transaction): Loaded plugin embulk/output/elasticsearch_ruby from a load path
2016-07-29 09:34:32.575 +0900 [INFO] (0001:transaction): Using local thread executor with max_threads=8 / output tasks 4 = input tasks 1 * 4
2016-07-29 09:34:32.601 +0900 [INFO] (0001:transaction): {done:  0 / 1, running: 0}
org.embulk.exec.PooledBufferAllocator$BufferDoubleReleasedException: Detected double release() call of a buffer
  at org.embulk.exec.PooledBufferAllocator$NettyByteBufBuffer.release(PooledBufferAllocator.java:55)
@toyama0919
toyama0919 / gist:415b32202916d905909d
Last active November 11, 2015 10:59
embulk-input-postgresqlからprestogresにつなげるとエラーになる

embulk-input-postgresqlからprestogresにつなげるとエラーになる。

やりたいこと

  • embulkからprestoにqueryしたい。

思いつく実現方法

  • embulk-input-postgreqlからprestogresにSQLを実行する
  • embulk-input-prestoを作る
@toyama0919
toyama0919 / file0.txt
Created February 8, 2014 09:25
dockerがmacに対応したのでインストール ref: http://qiita.com/toyama0919/items/b4249d910a5ece927208
brew update
brew tap homebrew/binary
brew install docker
@toyama0919
toyama0919 / elasticsearch.yml
Created January 8, 2014 10:38
elasticsearchでjsonをimportする(高速) ref: http://qiita.com/toyama0919/items/8f637794b257f9c7bbd1
http.compression: true
@toyama0919
toyama0919 / Gemfile
Created November 21, 2013 16:19
railsからredshiftに接続する ref: http://qiita.com/toyama0919/items/8de293cb11178a8b999e
gem "activerecord-redshift-adapter"
@toyama0919
toyama0919 / file0.txt
Created November 19, 2013 11:32
elasticsearchのパフォーマンスをnewrelicでモニタリングする(centos) ref: http://qiita.com/toyama0919/items/c52f67818d18ce3e4ac2
yum install python-setuptools
yum install python-devel
yum --enablerepo=epel install libyaml-devel
easy_install pip
@toyama0919
toyama0919 / file0.txt
Created November 19, 2013 09:23
elasticsearchの便利なプラグイン集 ref: http://qiita.com/toyama0919/items/383b97bdd6cfcd098ce4
sudo su - root
cd /usr/share/elasticsearch/
bin/plugin -install elasticsearch/elasticsearch-cloud-aws
bin/plugin -install mobz/elasticsearch-head
bin/plugin -install lukas-vlcek/bigdesk
bin/plugin -install royrusso/elasticsearch-HQ
yum install python-setuptools
easy_install esclient