Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
Elasticsearch Python bulk index API example
>>> import itertools
>>> import string
>>> from elasticsearch import Elasticsearch,helpers
es = Elasticsearch()
>>> # k is a generator expression that produces
... # a series of dictionaries containing test data.
... # The test data are just letter permutations
... # created with itertools.permutations.
... #
... # We then reference k as the iterator that's
... # consumed by the elasticsearch.helpers.bulk method.
>>> k = ({'_type':'foo', '_index':'test','letters':''.join(letters)}
... for letters in itertools.permutations(string.letters,2))
>>> # calling k.next() shows examples
... # (while consuming the generator, of course)
>>> # each dict contains a doc type, index, and data (at minimum)
>>> k.next()
{'_type': 'foo', 'letters': 'ab', '_index': 'test'}
>>> k.next()
{'_type': 'foo', 'letters': 'ac', '_index': 'test'}
>>> # create our test index
>>> es.indices.create('test')
{u'acknowledged': True}
>>> helpers.bulk(es,k)
(2650, [])
>>> # check to make sure we got what we expected...
>>> es.count(index='test')
{u'count': 2650, u'_shards': {u'successful': 1, u'failed': 0, u'total': 1}}
@zyxwu
zyxwu commented Apr 3, 2016

but when I index documents one by one, everything's fine
res = es.index(index=INDEX, doc_type=DOC_TYPE, id=ind, body=JS_message)

@k2xl
k2xl commented Mar 22, 2017

Would this work for a python list of JSON documents?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment