docker-remote-context-mount-error.md

Error mounting file with remote docker context

I am getting the error error mounting "/root/public/litesiem/core/elasticsearch/config/elasticsearch.yml" to rootfs at "/usr/share/elasticsearch/config/elasticsearch.yml" while trying to bring up a compose service on a remote host.

$ docker --version

Docker version 20.10.17, build 100c701

$ docker-compose --version

docker-compose version 1.29.2, build 5becea4c

$ docker context ls

NAME        DESCRIPTION                               DOCKER ENDPOINT               KUBERNETES ENDPOINT   ORCHESTRATOR
default *   Current DOCKER_HOST based configuration   unix:///var/run/docker.sock                         swarm
remote                                                ssh://remote-host                                           

./docker-compose.yaml

services:
  elasticsearch:
    build:
      context: ./elasticsearch
      args:
        ELASTIC_VERSION: ${ELASTIC_VERSION}
    volumes:
      - "${ROOT:-.}/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml"
      - "elasticsearch:/usr/share/elasticsearch/data"
    expose:
      - "9200"
    environment:
      discovery.type: single-node
      ES_JAVA_OPTS: '-Xms512m -Xmx512m'
    healthcheck:
      test: curl -s -f http://elasticsearch:9200/_nodes/http || exit 1
      interval: 30s
      timeout: 10s
      retries: 3

./elastisearch/Dockerfile

ARG ELASTIC_VERSION

FROM docker.elastic.co/elasticsearch/elasticsearch:${ELASTIC_VERSION}

$ ls elastisearch/config/elasticsearch.yml

elasticsearch/config/elasticsearch.yml

$ dc up elasticsearch

Removing core_elasticsearch_1
Recreating c7d11cd6431f_core_elasticsearch_1 ... done
Attaching to core_elasticsearch_1
elasticsearch_1  | {"type": "server", "timestamp": "2022-09-07T14:58:11,089Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "3e0cfd887eea", "message": "version[7.17.4], pid[7], build[default/docker/79878662c54c886ae89206c685d9f1051a9d6411/2022-05-18T18:04:20.964345128Z], OS[Linux/5.4.0-124-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/18.0.1.1/18.0.1.1+2-6]" }
elasticsearch_1  | {"type": "server", "timestamp": "2022-09-07T14:58:11,111Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "3e0cfd887eea", "message": "JVM home [/usr/share/elasticsearch/jdk], using bundled JDK [true]" }
elasticsearch_1  | {"type": "server", "timestamp": "2022-09-07T14:58:11,112Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "3e0cfd887eea", "message": "JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=ALL-UNNAMED, -Djava.security.manager=allow, -XX:+UseG1GC, -Djava.io.tmpdir=/tmp/elasticsearch-1475342965083098439, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Des.cgroups.hierarchy.override=/, -Xms512m, -Xmx512m, -XX:MaxDirectMemorySize=268435456, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]" }

root@remote:/# stat /root/public/litesiem/core/elasticsearch/config/elasticsearch.yml

  File: /root/public/litesiem/core/elasticsearch/config/elasticsearch.yml
  Size: 704       	Blocks: 8          IO Block: 4096   regular file
Device: fd14h/64788d	Inode: 6422566     Links: 1
Access: (0664/-rw-rw-r--)  Uid: ( 1000/ UNKNOWN)   Gid: ( 1000/ UNKNOWN)
Access: 2022-09-07 13:46:24.360471347 +0000
Modify: 2022-08-11 18:50:45.912672048 +0000
Change: 2022-09-07 13:46:24.360471347 +0000
 Birth: 2022-09-07 13:46:24.360471347 +0000

$ ROOT=/root/public/litesiem/core DOCKER_CONTEXT=remote docker-compose -f docker-compose.yaml up elasticsearch

Removing core_elasticsearch_1
Recreating 3e0cfd887eea_core_elasticsearch_1 ... error

ERROR: for 3e0cfd887eea_core_elasticsearch_1  Cannot start service elasticsearch: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/root/public/litesiem/core/elasticsearch/config/elasticsearch.yml" to rootfs at "/usr/share/elasticsearch/config/elasticsearch.yml": mount /root/public/litesiem/core/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type

ERROR: for elasticsearch  Cannot start service elasticsearch: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/root/public/litesiem/core/elasticsearch/config/elasticsearch.yml" to rootfs at "/usr/share/elasticsearch/config/elasticsearch.yml": mount /root/public/litesiem/core/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
ERROR: Encountered errors while bringing up the project.

What I have tried

• Deleting old containers/images for the service
• Deleting are re-rsyncing the config file/folder
• Removed the volume mount mode ro,z

Solved by upgrading Compose. Who would've thought!

sudo apt-get install docker-compose-plugin
ROOT=/root/public/litesiem/core docker --context remote compose -f docker-compose.yaml up elasticsearch