Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save renshuki/dc29585b686356b95939b1e1b7b1328a to your computer and use it in GitHub Desktop.
Save renshuki/dc29585b686356b95939b1e1b7b1328a to your computer and use it in GitHub Desktop.
Elasticsearch CSV import via File Data Visualizer (ML) - Convert latitude and longitude column to geo_point using ingest pipeline
  1. Choose your CSV file to import including your coordinates data and upload it

location.csv

lat,long, timestamp
41.12,-71.34,1569476964
38.85896,-106.01665,1569476964
65.47629,18.61576,1569476964
  1. Click on import and go to the Advanced tab
  2. Set the mapping
{
  " timestamp": {
    "type": "date",
    "format": "epoch_second"
  },
  "@timestamp": {
    "type": "date"
  },
  "経度": {
    "type": "long"
  },
  "緯度": {
    "type": "long"
  },
  "location": {
    "type": "geo_point"
  }
}
  1. Define the ingest pipeline (to set location geo_point)
{
  "description" : "sets location geo_point with lat, lon",
  "processors": [{
  "set": {
    "field": "location",
    "value": "{{lat}}, {{lon}}"
  }
  }]
}
  1. Import
@shkazmi89
Copy link

It not working, please tell if th ingest pipeline should also contain data format?

I have set mapping to:
{
"@timestamp": {
"type": "date"
},
"Country": {
"type": "keyword"
},
"Day_Since_First_Case": {
"type": "long"
},
"Lat": {
"type": "long"
},
"Lon": {
"type": "long"
},
"location": {
"type": "geo_point"
}
"date": {
"type": "date",
"format": "iso8601"
},
"iso_code": {
"type": "keyword"
},
"new_cases": {
"type": "long"
},
and Ingest pipeline as following:
{
"description" : "sets location geo_point with lat, lon",
"processors": [{
"set": {
"field": "location",
"value": "{{lat}}, {{lon}}"
}
}]
}

Error parsing JSON
Error parsing mappings: Unexpected string in JSON at position 254

@renshuki
Copy link
Author

@shkazmi89
Error parsing mappings: Unexpected string in JSON at position 254 looks like a JSON parsing issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment