Skip to content

Instantly share code, notes, and snippets.

@makuk66
Last active February 6, 2023 11:20
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save makuk66/0812f70b77aa92230c203cec41acac64 to your computer and use it in GitHub Desktop.
Save makuk66/0812f70b77aa92230c203cec41acac64 to your computer and use it in GitHub Desktop.
docker-compose example including creation of a collection
# a docker-compose.yml to start a simple cluster with three ZooKeeper nodes and three Solr nodes
# and create a collection
#
# To use:
#
# mkdir mycluster
# cd mycluster
# curl --output docker-compose.yml https://raw.githubusercontent.com/docker-solr/docker-solr-examples/master/docker-compose/docker-compose.yml
# docker-compose up
#
version: '3.7'
services:
solr1:
container_name: solr1
image: solr:8.2
ports:
- "8981:8983"
environment:
- ZK_HOST=zoo1:2181,zoo2:2181,zoo3:2181
networks:
- solr
volumes:
- solr1data:/var/solr
depends_on:
- zoo1
- zoo2
- zoo3
solr2:
image: solr:8.2
container_name: solr2
ports:
- "8982:8983"
environment:
- ZK_HOST=zoo1:2181,zoo2:2181,zoo3:2181
networks:
- solr
volumes:
- solr2data:/var/solr
depends_on:
- zoo1
- zoo2
- zoo3
solr3:
image: solr:8.2
container_name: solr3
ports:
- "8983:8983"
environment:
- ZK_HOST=zoo1:2181,zoo2:2181,zoo3:2181
ports:
- 8983:8983
networks:
- solr
volumes:
- solr3data:/var/solr
depends_on:
- zoo1
- zoo2
- zoo3
create-collection:
image: solr:8.2
container_name: create-collection
environment:
- SOLR_HOST=solr1
networks:
- solr
command:
- bash
- "-e"
- "-x"
- "-c"
- "wait-for-solr.sh --max-attempts 10 --wait-seconds 5 --solr-url http://$$SOLR_HOST:8983/; solr create_collection -c mycollection -p 8983"
zoo1:
image: zookeeper:3.5
container_name: zoo1
restart: always
hostname: zoo1
ports:
- 2181:2181
environment:
ZOO_MY_ID: 1
ZOO_SERVERS: server.1=0.0.0.0:2888:3888;2181 server.2=zoo2:2888:3888;2181 server.3=zoo3:2888:3888;2181
networks:
- solr
volumes:
- zoo1data:/data
- zoo1datalog:/datalog
zoo2:
image: zookeeper:3.5
container_name: zoo2
restart: always
hostname: zoo2
ports:
- 2182:2181
environment:
ZOO_MY_ID: 2
ZOO_SERVERS: server.1=0.0.0.0:2888:3888;2181 server.2=0.0.0.0:2888:3888;2181 server.3=zoo3:2888:3888;2181
networks:
- solr
volumes:
- zoo2data:/data
- zoo2datalog:/datalog
zoo3:
image: zookeeper:3.5
container_name: zoo3
restart: always
hostname: zoo3
ports:
- 2183:2181
environment:
ZOO_MY_ID: 3
ZOO_SERVERS: server.1=zoo1:2888:3888;2181 server.2=zoo2:2888:3888;2181 server.3=0.0.0.0:2888:3888;2181
networks:
- solr
volumes:
- zoo3data:/data
- zoo3datalog:/datalog
networks:
solr:
volumes:
solr1data:
solr2data:
solr3data:
zoo1data:
zoo1datalog:
zoo2data:
zoo2datalog:
zoo3data:
zoo3datalog:
@telendt
Copy link

telendt commented Feb 5, 2023

@makuk66: I'm trying to do something similar - to create a collection with solr create-collection from a different container (host) like you do here, but I want that solr1 to run embedded zookeeper. The problem is that create_collection doesn't work in this case as it gets the zookeeper connection URL from solr1 Solr instance, which claims the address is localhost:9983. That address of course is not reachable on create-collection host:

create-collection  |
create-collection  | Connecting to ZooKeeper at localhost:9983 ...
create-collection  | INFO  - 2023-02-05 15:20:30.865; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
create-collection  |
create-collection  | ERROR: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper localhost:9983 within 15000 ms

Do you know if there is a workaround for this problem?

@telendt
Copy link

telendt commented Feb 6, 2023

It looks like there is no way to specify a custom ZK_HOST for solr create_collection so instead I went with solr zk upconfig + collection create.

create-collections:
  image: solr:8.11.2
  environment:
    - SOLR_HOST=solr1
    - ZK_HOST=solr1:9983
    - COLLECTION_NAME=mycollection
  networks:
    - solr
  command:
    - bash
    - "-exc"
    - |
      wait-for-solr.sh --max-attempts 10 --wait-seconds 5 --solr-url http://$${SOLR_HOST}:8983/

      solr zk upconfig -n "$${COLLECTION_NAME}_conf" -d "PATH_TO_COLLECTION_CONFIG"

      curl --get -sSf "http://$${SOLR_HOST}:8983/solr/admin/collections?action=CREATE&numShards=1&replicationFactor=1" \
        --data-urlencode "name=$${COLLECTION_NAME}" \
        --data-urlencode "collection.configName=$${COLLECTION_NAME}_conf"

More in this thread:
https://lists.apache.org/thread/vwv9d011vfqo5v8vwrrtzppcnknvn57b

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment