Skip to content

Instantly share code, notes, and snippets.

@r39132
Created February 11, 2017 01:41
Show Gist options
  • Save r39132/30cc62c74b3ba23039a622c31016766f to your computer and use it in GitHub Desktop.
Save r39132/30cc62c74b3ba23039a622c31016766f to your computer and use it in GitHub Desktop.
ElasticSearch 2.3 --> 5.1 Migration : new IP fields do not support ipv6
I recently migrated from AWS ES 2.3 to 5.1.
I followed the instructions on [http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-version-migration.html]
TLDR, i snapshotted my 2.3 ES cluster to S3 and then restored to the new 5.1 cluster from S3.
However, I ran into a problem. I added an *ip* field to my indexes, which included indexes brought over from 2.3 as well as new indexes created on 5.1. Here's an sample mapping:
curl -XPUT "localhost:80/cars/_mapping/transactions" -d'
{
"properties": {
"my_ip": { "type": "ip"}
}
}'
After adding the this field, my mapping looks like
deploy@es-proxy-00:~$ curl -XGET 'localhost:80/cars/transactions/_mapping'
{
"cars": {
"mappings": {
"transactions": {
"properties": {
"color": {
"type": "string"
},
"make": {
"type": "string"
},
"my_ip": {
"type": "ip"
},
"price": {
"type": "long"
},
"sold": {
"type": "date",
"format": "strict_date_optional_time||epoch_millis"
}
}
}
}
}
}
If I try to insert an ipv6 value into the indexes created on 5.1, it works. If I try to insert an ipv6 value into one of the indexes brought over from 2.3, I get an exception that the address is not ipv4. To be clear, the migration from 2.3 to 5.1 works otherwise. I can use new types like scaled_float, but not a type that has changed between 2.x. and 5.x to support ipv6 in addition to ipv4 addresses.
deploy@es-proxy-00:~$ cat test_ip_a.sh
curl -XPOST "localhost:80/cars/transactions/AVlxOqfz-i1bJQCCyerj" -d'
{
"my_ip": "2607:f8b0:400e:c00::22b"
}'
deploy@es-proxy-00:~$ ./test_ip_a.sh
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "failed to parse [my_ip]"
}
],
"type": "mapper_parsing_exception",
"reason": "failed to parse [my_ip]",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "failed to parse ip [2607:f8b0:400e:c00::22b], not a valid ipv4 address (4 dots)"
}
},
"status": 400
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment