Testing REX-Ray with Docker 1.13 Plugins

not-working.md

Using REX-Ray with Docker 1.13 Plugins

current status is FAILING

The rexay.sock file is never created under the /var/run/docker/plugins directory. However, everything in /var/run/rexray and /var/run/libstorage all seem to be in working order. /var/log/rexray/rexray.log has the error:

time="2017-01-11T20:46:34Z" level=panic msg="error initializing instance ID cache" inner.lsx="/var/lib/libstorage/lsx-linux" inner.args=[scaleio instanceID] inner.inner.Stderr=[101 114 114 111 114 58 32 101 114 114 111 114 32 103 101 116 116 105 110 103 32 105 110 115 116 97 110 99 101 32 73 68 58 32 112 114 111 98 108 101 109 32 103 101 116 116 105 110 103 32 115 100 99 32 103 117 105 100 10]

Standup a functional ScaleIO environment

git clone https://github.com/codedellemc/vagrant
cd vagrant/scaleio
vi Vagrantfile

change rexrayinstall to "False"

vagrant up --provider virtualbox

Install Docker on Tie Breaker

vagrant ssh tb
sudo yum update -y
curl -fsSL https://test.docker.com/ | sh
sudo usermod -aG docker vagrant && sudo systemctl enable docker.service && sudo systemctl start docker

Create the REX-Ray Configuration File

sudo mkdir -p /etc/rexray

sudo tee -a /etc/rexray/config.yml << EOF
libstorage:
  service: scaleio
  integration:
    volume:
      operations:
        mount:
          preempt: true
scaleio:
  endpoint: https://192.168.50.12/api
  insecure: true
  useCerts: true
  userName: admin
  password: 'Scaleio123'
  systemName: cluster1
  protectionDomainName: pdomain
  storagePoolName: pool1
  thinOrThick: ThinProvisioned
EOF

Create a REX-Ray Dockerfile to Containerize it and be used with ScaleIO

sudo mkdir -p /home/vagrant/rr

sudo tee -a /home/vagrant/rr/Dockerfile << EOF
FROM centos

RUN yum update -y
RUN yum install numactl libaio xfsprogs e4fsprogs -y

ENV RR_VERSION latest
ENV RR_RELEASE stable

RUN curl -sSL https://dl.bintray.com/emccode/rexray/install | sh -s -- $RR_RELEASE $RR_VERSION

RUN echo -e '#!/bin/sh\n\nexec rexray start -f' > /docker-entrypoint.sh && chmod +x /docker-entrypoint.sh

ENTRYPOINT ["/docker-entrypoint.sh"]
EOF

Build the Docker 1.13 Plugin rootfs

cd rr
sudo docker build -t rootfs .
id=$(sudo docker create rootfs true) 
sudo mkdir -p /var/lib/docker/plugins/$id/rootfs
sudo docker export "$id" | sudo tar -x -C /var/lib/docker/plugins/$id/rootfs
sudo chgrp -R docker /var/lib/docker/plugins/
sudo docker rm -vf "$id"
sudo docker rmi rootfs

Create the Docker 1.13 config.json

sudo tee -a /var/lib/docker/plugins/$id/config.json << EOF
{
        "manifestVersion": "v0",
        "description": "REX-Ray for Dell EMC ScaleIO",
        "documentation": "http://libstorage.readthedocs.io/en/stable/user-guide/storage-providers/#scaleio",
        "entrypoint": ["/docker-entrypoint.sh"],
        "interface" : {
          "types": ["docker.volumedriver/1.0"],
          "socket": "rexray.sock"
        },
        "network": {
          "type": "host"
        },

        "mounts": [
          {
            "source": "/etc/rexray",
            "destination": "/etc/rexray",
            "type": "bind",
                            "options": ["rbind"]
          },
                     {
                            "source": "/var/lib/libstorage",
                            "destination": "/var/lib/libstorage",
                            "type": "bind",
                            "options": ["shared", "rbind"]
                     },
                     {
                            "source": "/var/lib/rexray",
                            "destination": "/var/lib/rexray",
                            "type": "bind",
                            "options": ["shared", "rbind"]
                     },
                     {
                            "source": "/var/run/rexray",
                            "destination": "/var/run/rexray",
                            "type": "bind",
                            "options": ["rbind"]
                     },
                     {
                            "source": "/var/run/libstorage",
                            "destination": "/var/run/libstorage",
                            "type": "bind",
                            "options": ["rbind"]
                     },
                     {
                            "source": "/var/log/rexray",
                            "destination": "/var/log/rexray",
                            "type": "bind",
                            "options": ["rbind"]
                     },
                     {
                            "source": "/var/run/docker/plugins",
                            "destination": "/var/run/docker/plugins",
                            "type": "bind",
                            "options": ["rbind"]
                     },
                     {
                            "source": "/dev",
                            "destination": "/dev",
                            "type": "bind",
                            "options": ["rbind"]
                     },
                     {
                            "source": "/bin/emc",
                            "destination": "/bin/emc",
                            "type": "bind",
                            "options": ["rbind"]
                     },
                     {
                            "source": "/opt/emc/scaleio/sdc",
                            "destination": "/opt/emc/scaleio/sdc",
                            "type": "bind",
                            "options": ["rbind"]
                     }
        ],

        "args": {
          "name": "args",
          "description": "command line arguments",
          "value": ["rexray start -f"]
        },

        "env": [
          {
            "name": "RR_VERSION",
            "description": "The REX-Ray Version",
            "value": "latest"
          },
         {
                "name": "RR_RELEASE",
                "description": "The REX-Ray Release Train",
                "value": "stable"
         }
        ],
              "capabilities": ["CAP_SYS_ADMIN"]
}
EOF

Create all the folders needed by REX-Ray on the host system

sudo mkdir -p /var/lib/libstorage && sudo mkdir -p /var/lib/rexray && sudo mkdir -p /var/log/rexray && sudo mkdir -p /var/run/rexray && sudo mkdir -p /var/run/libstorage

Create and Start the Docker Plugin

sudo docker plugin create rexray/scaleio /var/lib/docker/plugins/$id
sudo docker plugin enable rexray/scaleio