Skip to content

Instantly share code, notes, and snippets.

@thisismitch
Last active February 28, 2020 21:11
Show Gist options
  • Save thisismitch/3429023e8438cc25b86c to your computer and use it in GitHub Desktop.
Save thisismitch/3429023e8438cc25b86c to your computer and use it in GitHub Desktop.
Filebeat configuration with comments removed
{
"mappings": {
"_default_": {
"_all": {
"enabled": true,
"norms": {
"enabled": false
}
},
"dynamic_templates": [
{
"template1": {
"mapping": {
"doc_values": true,
"ignore_above": 1024,
"index": "not_analyzed",
"type": "{dynamic_type}"
},
"match": "*"
}
}
],
"properties": {
"@timestamp": {
"type": "date"
},
"message": {
"type": "string",
"index": "analyzed"
},
"offset": {
"type": "long",
"doc_values": "true"
},
"geoip" : {
"type" : "object",
"dynamic": true,
"properties" : {
"location" : { "type" : "geo_point" }
}
}
}
}
},
"settings": {
"index.refresh_interval": "5s"
},
"template": "filebeat-*"
}
filebeat:
prospectors:
-
paths:
- /var/log/secure
- /var/log/messages
# - /var/log/*.log
input_type: log
document_type: syslog
registry_file: /var/lib/filebeat/registry
output:
logstash:
hosts: ["elk_server_private_ip:5044"]
bulk_max_size: 1024
tls:
certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]
shipper:
logging:
files:
rotateeverybytes: 10485760 # = 10MB
filebeat:
prospectors:
-
paths:
- /var/log/auth.log
- /var/log/syslog
# - /var/log/*.log
input_type: log
document_type: syslog
registry_file: /var/lib/filebeat/registry
output:
logstash:
hosts: ["elk_server_private_ip:5044"]
bulk_max_size: 1024
tls:
certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]
shipper:
logging:
files:
rotateeverybytes: 10485760 # = 10MB
@uw920
Copy link

uw920 commented Feb 27, 2018

{
"error" : {
"root_cause" : [
{
"type" : "mapper_parsing_exception",
"reason" : "No handler for type [string] declared on field [message]"
}
],
"type" : "mapper_parsing_exception",
"reason" : "Failed to parse mapping [default]: No handler for type [string] declared on field [message]",
"caused_by" : {
"type" : "mapper_parsing_exception",
"reason" : "No handler for type [string] declared on field [message]"
}
},
"status" : 400
}

@wixaw
Copy link

wixaw commented Mar 15, 2018

I have same error

@dumityty
Copy link

I got the same error when running Elasticsearch 6.X
"string" is no longer an acceptable type so it needs to be changed to TEXT and the index to TRUE rather than "analyzed":

...
"message": {
    "type": "text",
    "index": true
},
...

Then I got a different error:
"Failed to parse mapping [_default_]: Enabling [_all] is disabled in 6.0. As a replacement, you can use [copy_to] on mapping fields to create your own catch all field.",
Apparently the _all field no longer exists and you can either not create it at all or if you want to use copy_to to create your own _all field:
https://discuss.elastic.co/t/elasticsearch-6-and-the-disappearing--all-field/85871/6

Once I removed the _all bit it worked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment