Skip to content

Instantly share code, notes, and snippets.

@yesnik
Last active July 25, 2023 13:19
Show Gist options
  • Save yesnik/df992790bfd7cd141065625dd4282968 to your computer and use it in GitHub Desktop.
Save yesnik/df992790bfd7cd141065625dd4282968 to your computer and use it in GitHub Desktop.
Elasticsearch with Docker

Elasticsearch Docs

URL for Dev Tools: http://0.0.0.0:5601/app/dev_tools

Add data to Elasticsearch

You can add data to Elasticsearch by sending JSON objects (documents) to Elasticsearch over HTTP. Whether you have structured or unstructured text, numerical data, or geospatial data, Elasticsearch efficiently stores and indexes it in a way that supports fast searches.

We can use the Kibana Dev Tools console or any HTTP client to submit REST requests to Elasticsearch,

Use Elastic Agent to collect data from hosts or containers that you need to monitor.

Submit an HTTP post request that contains a JSON document:

POST /customer/_doc/1
{
  "name": "John Doe"
}

This request automatically creates the customer index, adds a new document that has an ID of 1, and stores and indexes the name field.

Get data by ID

The new document is available immediately from any node in the cluster. You can retrieve it with a GET request that specifies its document ID:

GET /customer/_doc/1

Add data in bulk

Instead of adding documents one at a time, you can use the _bulk endpoint to add multiple documents in one request. This minimizes network roundtrips and is significantly faster than adding documents one at a time.

Want to index some of your own data? You can upload data from a CSV, TSV, JSON file or use Elastic integrations to collect data from popular services and platforms like Nginx, AWS, and MongoDB. To check what’s available, select Add integrations on the Kibana home page.

POST /product/_bulk?pretty
{ "create":{ } }
{ "title": "Book", "price":205}
{ "create":{ } }
{ "title": "Super book", "price":999}
{ "create":{ } }
{ "title": "Pencil", "price":55}
{ "create":{ } }
{ "title": "Book with pencil", "price":455}
{ "create":{ } }
{ "title": "Paper", "price":345}

Search and sort data

Indexed documents are available for search in near real-time. To search for specific terms within a field, you can use a match query. For example, the following request searches the title field to find products whose titles contain pencil or book:

GET /product/_search
{
  "query": { "match": { "title": "pencil book" } }
}

To construct more complex queries, you can use a bool query to combine multiple query criteria. You can designate criteria as required (must match), desirable (should match), or undesirable (must not match).

For example, the following request searches the product index for items that have price 999, but excludes items that have title 'Pencil':

GET /product/_search
{
  "query": {
    "bool": {
      "must": [
        { "match": { "price": "999" } }
      ],
      "must_not": [
        { "match": { "title": "Pencil" } }
      ]
    }
  }
}

Elasticsearch Installation with Docker

See Docs

  1. Pull the Elasticsearch Docker image:
    docker pull docker.elastic.co/elasticsearch/elasticsearch:8.8.2
    
  2. The following commands start a single-node Elasticsearch cluster for development or testing. Create a new docker network for Elasticsearch and Kibana:
    docker network create elastic
    
  3. Start Elasticsearch in Docker. Copy from the terminal a password for elastic user and an enrollment token for enrolling Kibana. Save them in a secure location. These values are shown only when you start Elasticsearch for the first time.
    docker run --name es-node01 --net elastic -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" -t docker.elastic.co/elasticsearch/elasticsearch:8.8.2
    
  4. Copy the http_ca.crt security certificate from your Docker container to your local machine.
    docker cp es-node01:/usr/share/elasticsearch/config/certs/http_ca.crt .
    
  5. Open a new terminal and verify that you can connect to your Elasticsearch cluster by making an authenticated call, using the http_ca.crt file that you copied from your Docker container. Enter the password for the elastic user when prompted.
    curl --cacert http_ca.crt -u elastic https://localhost:9200
    

Kibana Installation with Docker

Kibana URL: http://0.0.0.0:5601/

See docs

  1. In a new terminal session, start Kibana and connect it to your Elasticsearch container. Pull the Kibana Docker image:
    docker pull docker.elastic.co/kibana/kibana:8.8.2
    
  2. Start Kibana in Docker:
    docker run --name kib-01 --net elastic -p 5601:5601 docker.elastic.co/kibana/kibana:8.8.2
    
    You'll see in terminal something like this: Go to http://0.0.0.0:5601/?code=507538 to get started.
  3. To access Kibana, click the generated link in your terminal.
    • In your browser, paste the enrollment token that you copied when starting Elasticsearch and click the button to connect your Kibana instance with Elasticsearch.
    • Log in to Kibana as the elastic user with the password that was generated when you started Elasticsearch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment