Skip to content

Instantly share code, notes, and snippets.

@xSakix

xSakix/elasticsearch-install.ipynb

Created August 29, 2021 09:40
Show Gist options
  • Save xSakix/74695d559505bcfd41774a9064358cb5 to your computer and use it in GitHub Desktop.
Save xSakix/74695d559505bcfd41774a9064358cb5 to your computer and use it in GitHub Desktop.
Download ZIP
elasticsearch - install.ipynb
Display the source blob
Display the rendered blob
Raw
elasticsearch-install.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "elasticsearch - install.ipynb",
"provenance": [],
"collapsed_sections": [],
"authorship_tag": "ABX9TyPTefcnKSJ245oytDC4JWdo",
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/xSakix/74695d559505bcfd41774a9064358cb5/elasticsearch-install.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-VwOvgtBQCiy"
},
"source": [
"#Elastic search in Collab\n",
"\n",
"Had to install elastic search in colab for 'reasons' and this is the way it worked for me. Might be usefull for someone else also.\n",
"\n",
"Works with 7.9.2. Probably could be run also with 7.14.0, but didn't have time to debug the issues. If you want, you can try and just run the instance under the 'elasticsearch' user to get the proper error log."
]
},
{
"cell_type": "code",
"metadata": {
"id": "_mHSIejH6qYF"
},
"source": [
"#7.9.1 works with ES 7.9.2\n",
"!pip install -Iv elasticsearch==7.9.1"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "UKfq3beg9Yqq",
"outputId": "90cdece2-b83b-43c8-e55e-cc13769f537f"
},
"source": [
"#download ES 7.92 and extract\n",
"%%bash\n",
"\n",
"wget -q https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-7.9.2-linux-x86_64.tar.gz\n",
"wget -q https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-7.9.2-linux-x86_64.tar.gz.sha512\n",
"shasum -a 512 -c elasticsearch-oss-7.9.2-linux-x86_64.tar.gz.sha512 \n",
"tar -xzf elasticsearch-oss-7.9.2-linux-x86_64.tar.gz"
],
"execution_count": 2,
"outputs": [
{
"output_type": "stream",
"text": [
"elasticsearch-oss-7.9.2-linux-x86_64.tar.gz: OK\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "g7Rej_GcIwQr",
"outputId": "294da89e-3fe9-49a1-81d8-f26f7e2d5064"
},
"source": [
"# create user elasticsearch and group elasticsearch, under which will the ES instance be running\n",
"# ES can't run under root\n",
"!sudo useradd elasticsearch\n",
"!sudo grep elasticsearch /etc/passwd \n",
"!sudo groupadd elasticsearch\n",
"!sudo usermod -a -G elasticsearch elasticsearch\n",
"!grep elasticsearch /etc/group"
],
"execution_count": 3,
"outputs": [
{
"output_type": "stream",
"text": [
"elasticsearch:x:1000:1000::/home/elasticsearch:/bin/sh\n",
"groupadd: group 'elasticsearch' already exists\n",
"elasticsearch:x:1000:elasticsearch\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "b4MTsnKwKL6z"
},
"source": [
"# change the directory rights to user:group \n",
"!sudo chown elasticsearch:elasticsearch -R elasticsearch-7.9.2"
],
"execution_count": 4,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "DyHSAedl9zUe",
"outputId": "86c622f9-633c-4563-f22c-d3dfdcec402f"
},
"source": [
"#run ES instance as a daemon\n",
"%%bash --bg\n",
"sudo -H -u elasticsearch elasticsearch-7.9.2/bin/elasticsearch"
],
"execution_count": 5,
"outputs": [
{
"output_type": "stream",
"text": [
"Starting job # 0 in a separate thread.\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "6ofYNhQ594zd"
},
"source": [
"# give time to start up\n",
"import time\n",
"time.sleep(20)"
],
"execution_count": 6,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "BIqw6ThT99Ae",
"outputId": "7ca9fdea-34ee-4a20-c1d0-e45c3b997539"
},
"source": [
"#print the process\n",
"%%bash\n",
"\n",
"ps -ef | grep elastic"
],
"execution_count": 7,
"outputs": [
{
"output_type": "stream",
"text": [
"root 151 149 0 09:37 ? 00:00:00 sudo -H -u elasticsearch elasticsearch-7.9.2/bin/elasticsearch\n",
"elastic+ 152 151 99 09:37 ? 00:00:20 /content/elasticsearch-7.9.2/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms1g -Xmx1g -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/elasticsearch-13828270847204549773 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=logs/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m -XX:MaxDirectMemorySize=536870912 -Des.path.home=/content/elasticsearch-7.9.2 -Des.path.conf=/content/elasticsearch-7.9.2/config -Des.distribution.flavor=oss -Des.distribution.type=tar -Des.bundled_jdk=true -cp /content/elasticsearch-7.9.2/lib/* org.elasticsearch.bootstrap.Elasticsearch\n",
"root 386 384 0 09:37 ? 00:00:00 grep elastic\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "QNsmgFZR99uF",
"outputId": "ba1a636b-ca38-42fe-d1db-87b3af150166"
},
"source": [
"#test the instance\n",
"%%bash\n",
"\n",
"curl -sX GET \"localhost:9200/\""
],
"execution_count": 8,
"outputs": [
{
"output_type": "stream",
"text": [
"{\n",
" \"name\" : \"80196037624b\",\n",
" \"cluster_name\" : \"elasticsearch\",\n",
" \"cluster_uuid\" : \"LQGhp_Y6TNGzHsw0lt4JEA\",\n",
" \"version\" : {\n",
" \"number\" : \"7.9.2\",\n",
" \"build_flavor\" : \"oss\",\n",
" \"build_type\" : \"tar\",\n",
" \"build_hash\" : \"d34da0ea4a966c4e49417f2da2f244e3e97b4e6e\",\n",
" \"build_date\" : \"2020-09-23T00:45:33.626720Z\",\n",
" \"build_snapshot\" : false,\n",
" \"lucene_version\" : \"8.6.2\",\n",
" \"minimum_wire_compatibility_version\" : \"6.8.0\",\n",
" \"minimum_index_compatibility_version\" : \"6.0.0-beta1\"\n",
" },\n",
" \"tagline\" : \"You Know, for Search\"\n",
"}\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "TC87zw7s_xbd",
"outputId": "f938e32a-8405-4955-b144-ac6d95765732"
},
"source": [
"# test the python client/lib\n",
"from elasticsearch import Elasticsearch\n",
"es = Elasticsearch()\n",
"es.ping()"
],
"execution_count": 9,
"outputs": [
{
"output_type": "stream",
"text": [
"/usr/local/lib/python3.7/dist-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.6) or chardet (3.0.4) doesn't match a supported version!\n",
" RequestsDependencyWarning)\n"
],
"name": "stderr"
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"True"
]
},
"metadata": {},
"execution_count": 9
}
]
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment