Logstash输出到aws elasticsearch service

README.md

1. 在本机安装logstash
2. 使用rvm安装jruby.
3. 进入logstash安装目录（如/usr/share/logstash）， 运行sudo -E bin/logstash-plugin install --no-verify --version 6.4.0 logstash-output-amazon_es 安装logstash-output-amazon_es插件。如果logstash-plugin安装很慢的话，请安装 haveged 后尝试。见此issue, 此文章
4. 在aws iam中创建一个用户，赋予编程访问，和AmazonESFullAccess权限，将此用户设为elasticsearch域的主用户，获取access_key和secret（如果已经配置了elasticsearch域的主用户，则用该用户的access_key和secret）
5. 配置logstash pipeline, 如

input {
  file {
      path => "/var/log/nginx/access.log"
      start_position => "beginning"
  }
}
filter {
  grok {
    patterns_dir => "/etc/logstash/patterns"
    match => { "message" => "%{NGINXACCESS}" }
  }
}
output {
  stdout {}
  amazon_es {
   hosts => ["$ES_host"]
   ssl => true
   region => "$ES_region"
   aws_access_key_id => '$access_key_id'
   aws_secret_access_key => '$secret_access_key'
   index => "nginx-access-logs-%{+YYYY.MM.dd}"
 }
}