AmitPhulera/es_upgrade_steps.md

## es_upgrade_steps.md

      
    Raw
  

              es_upgrade_steps.md
            
          
    ES Upgrade Doc For Local Dev Systems

The ES 5 setup on local dev system is a two step process, first one being installing Elasticsearch 5 and then populating the indices with data.
Step 1) - Installing Elasticsearch 5


For folks running ES inside docker, installing elasticsearch is a straightforward process.
./scripts/docker stop elasticsearch2
./scripts/docker up -d elasticsearch5


For people who are running Elasticsearch without docker


Ensure that you have stopped Elasticsearch 2.


Remove existing reference to Elasticsearch 2 from $PATH from your ~/.zshrc. You can find it by
grep elasticsearch ~/.zshrc

The output should look like, if you have Elasticsearch 2 configured in $PATH
export PATH=/your/path/elasticsearch-2.4.6/bin:$PATH


If you find the output similar to what is mentioned above, then remove that line.


Download Elasticsearch 5 and untar it
curl https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.16.tar.gz --output elasticsearch-5.6.16.tar.gz
tar -xvzf elasticsearch-5.6.16.tar.gz


Put the folder somewhere you can find it. Take note of tha path (pwd) and add the following to your ~/.zshrc:
export PATH=/your/path/elasticsearch-5.6.16/bin:$PATH


Open a new shell or source zshrc file using the command below to ensure the changes are applied.
source ~/.zshrc


Install Phonetic plugin
elasticsearch-plugin install analysis-phonetic


Run elasticsearch with
elasticsearch


Step 2) - Populating data in Elasticsearch


Paste these django settings in HQ's localsettings.py. This will ensure that the indices are created with the newer names.

Ensure that you are not editing docker/localsettings.py

ES_APPS_INDEX_MULTIPLEXED = False
ES_CASE_SEARCH_INDEX_MULTIPLEXED = False
ES_CASES_INDEX_MULTIPLEXED = False
ES_DOMAINS_INDEX_MULTIPLEXED = False
ES_FORMS_INDEX_MULTIPLEXED = False
ES_GROUPS_INDEX_MULTIPLEXED = False
ES_SMS_INDEX_MULTIPLEXED = False
ES_USERS_INDEX_MULTIPLEXED = False


ES_APPS_INDEX_SWAPPED = True
ES_CASE_SEARCH_INDEX_SWAPPED = True
ES_CASES_INDEX_SWAPPED = True
ES_DOMAINS_INDEX_SWAPPED = True
ES_FORMS_INDEX_SWAPPED = True
ES_GROUPS_INDEX_SWAPPED = True
ES_SMS_INDEX_SWAPPED = True
ES_USERS_INDEX_SWAPPED = True


NOTE - After Third party hosters are done with the upgrade, you will be asked to remove these settings.


Update elasticsearch major version on localsettings if it is set
ELASTICSEARCH_MAJOR_VERSION = 5


Make sure you are on lastest master branch.


Create and Populate the indexes using the reindexers. Run the following command
./manage.py ptop_preindex --reset

OR
If you have issues with the above command, you can use ptop_reindexer_v2 with which you can reindex one index at a time.
./manage.py ptop_reindexer_v2 {app, case-search, sql-case, domain, sql-form, group, sms, user} --reset


Run HQ locally and ensure that things powered by Elasticsearch are working properly.


Congratulations 🎉 you have sucessfully upgraded Elasticsearch on your local system.

You might encounter following issues -

During reindex, if the size of documents on your local es is too large

ERROR reindexer Error sending bulk payload to Elasticsearch: TransportError(413, '<html>\r\n<head><title>413 Request Entity Too Large</title></head>\r\n<body>\r\n<center><h1>413 Request Entity Too Large</h1></center>\r\n<hr><center>nginx/1.23.3</center>\r\n</body>\r\n</html>\r\n')

You can set a smaller chunk size while reindexing
./manage.py ptop_reindexer_v2 app --chunksize=10


Error creating index

elasticsearch2.exceptions.RequestError: TransportError(400, 'mapper_parsing_exception', 'Failed to parse mapping [case]: The [string] type is removed in 5.0 and automatic upgrade failed because parameters [null_value] are not supported for automatic upgrades. You should now use either a [text] or [keyword] field instead for field [value]')

Have you pulled latest master? Checkout master and do a git pull
git checkout master
git pull

If you run into any other issues, feel free to reach out Amit Phulera on Slack.