Jo-Hue
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cat << 'EOF' > /etc/zabbix/zabbix_agentd.conf.d/percentiles-discovery.json | |
| cat percentiles-discovery.json | |
| {"data":[ | |
| { | |
| "{#PCTQUERY}": "nginx.access.url:*_count*", "{#PCTFIELD}":"nginx.access.request_time", "{#PCTRANGE}":"-1hour","{#PCTINDEX}":"filebeat-*", | |
| "{#PCTNAME}": "ES _count query latencies last 1hour"}, | |
| { | |
| "{#PCTQUERY}": "nginx.access.url:*_count*", "{#PCTFIELD}":"nginx.access.request_time", "{#PCTRANGE}":"-5mins","{#PCTINDEX}":"filebeat-*", | |
| "{#PCTNAME}": "ES _count query latencies last 5 minutes"}, | |
| { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cat << 'EOF' >> /etc/zabbix/zabbix_agentd.conf.d/elasticsearch.conf | |
| UserParameter=elastic.discovery.percentiles, cat /etc/zabbix/zabbix_agentd.conf.d/percentiles-discovery.json | |
| EOF |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cat << 'EOF' > /etc/zabbix/zabbix_agentd.conf.d/elasticsearch.conf | |
| UserParameter=elastic.percentiles[*], /etc/zabbix/zabbix_agentd.conf.d/es-percentiles.sh "$1" "$2" "$3" "$4" "$5" "$6" "$7" "$8" "$9" | |
| EOF |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ./es-percentiles.sh "*" nginx.access.request_time -1hours | |
| #=> | |
| { | |
| "1.0": 0, | |
| "5.0": 0.0010000000474974513, | |
| "25.0": 0.0010000000474974513, | |
| "50.0": 0.0020000000949949026, | |
| "75.0": 0.013000000268220901, | |
| "95.0": 0.024000000208616257, | |
| "99.0": 0.027000000700354576 |
Jo-Hue
/ specify a query, the field, time range and index.
Created
September 16, 2019 11:26
specify a query, the field, time range and index.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cd /etc/zabbix/zabbix_agentd.conf.d | |
| curl https://raw.githubusercontent.com/FreekPaans/zabbix-configs/master/es-percentiles.sh > es-percentiles.sh | |
| chmod +x es-percentiles.sh |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| curl -s -d '{"size":0,"aggs":{"zabbix_aggregate":{"percentiles":{'\ | |
| '"field":"nginx.access.request_time",'\ | |
| '"percents":[50,99]}}}}' \ | |
| localhost:9200/*/_search -H "Content-type: application/json" | jq . | |
| #=> | |
| { | |
| "took": 272, | |
| "timed_out": false, | |
| "num_reduce_phases": 3, | |
| "_shards": { |
Jo-Hue
/ log data should now be getting indexed
Created
September 11, 2019 15:38
log data should now be getting indexed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| curl -s 'localhost:9200/filebeat-*/_search?size=1' | jq '.hits.hits[0]._source.nginx.access' | |
| #=> | |
| { | |
| "response_code": "200", | |
| "upstream": { | |
| "connect_time": 0, | |
| "response_time": 0.004, | |
| "header_time": 0.004 | |
| }, | |
| "method": "GET", |
Jo-Hue
/ The complete filebeat.inputs section
Created
September 11, 2019 15:27
The complete filebeat.inputs section
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| filebeat.inputs: | |
| - type: log | |
| enabled: false | |
| paths: | |
| - /var/log/*.log | |
| - type: log | |
| enabled: true | |
| paths: | |
| - /var/log/nginx/*upstream_time.log* | |
| exclude_files: ['.gz$'] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| - type: log | |
| enabled: true | |
| paths: | |
| - /var/log/nginx/*upstream_time.log* | |
| exclude_files: ['.gz$'] | |
| pipeline: nginx-response-time |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cat << EOF > pipeline.json | |
| { | |
| "description": "pipeline for parsing nginx response times", | |
| "processors": [ | |
| { | |
| "set": { | |
| "field": "log_line", | |
| "value": "{{message}}" | |
| } | |
| }, |
NewerOlder