Skip to content

Instantly share code, notes, and snippets.

@utkarshmalik211
Last active April 1, 2019 09:36
Show Gist options
  • Save utkarshmalik211/da21ffb819f8afd9f409b8e09ce33784 to your computer and use it in GitHub Desktop.
Save utkarshmalik211/da21ffb819f8afd9f409b8e09ce33784 to your computer and use it in GitHub Desktop.
Minimal Python ElasticSearch scroll example WITHOUT client library
data = []
base_url="https://<elasticsearch-cluster-name>.us-west-2.es.amazonaws.com/"
payload = {
"size":10000,
"query":{
"match_all": {}
}
}
response = requests.post(base_url+"<index-name>/_search?scroll=1m", json=payload)
response = json.loads(response.content.decode())
# subsequent requests
scroll = {
"scroll" : "1m",
"scroll_id" : response['_scroll_id']
}
data.extend(response['hits']['hits'])
while (response['hits']['hits'] != []):
response = requests.post(f'{base_url}_search/scroll', json=scroll)
response = json.loads(response.content.decode())
data.extend(response['hits']['hits'])
scroll_id = response['_scroll_id']
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment