Volume migration for a job that previously ran PX-Developer and is now running PX-Essentials or PX-Enterprise should be done
with a job that is not currently running.
For this example, I am going to use a simple REDIS container to migrate its /data volume to CSI volume in Nomad.
NOTE: It is important that the 'name' value matches the 'name' column in the pxctl volume status output and that the
'id' value matches the 'id' column from pxctl volume status
root@localdev:/vagrant/localdev# pxctl volume list | grep 'redis\|NAME'
ID NAME SIZE HA SHARED ENCRYPTED PROXY-VOLUME IO_PRIORITY STATUS SNAP-ENABLED
1107963972586667525 redis-cache 1 GiB 1 no no no MEDIUM up - attached on 10.0.2.15 no
Before we do anything, assuming we have data in our redis container at /data, lets verify that's the case:
# Exec in to our redis container's alloc
root@localdev:/vagrant/localdev# nomad exec -i 94f5c /bin/bash
root@79ca3843e4f6:/data#
# Create a file we want to make sure exists post-migration
root@79ca3843e4f6:/data# date --rfc-3339=ns > date.txt
root@79ca3843e4f6:/data# ls -lah
total 12K
drwxr-xr-x. 2 redis redis 4.0K Oct 19 02:08 .
drwxr-xr-x 1 root root 4.0K Oct 19 02:04 ..
-rw-r--r-- 1 root root 36 Oct 19 02:08 date.txt
# Exit out of our container, and stop the redis job
root@localdev:/vagrant/localdev# nomad stop --purge redis
# pxctl volume status should show our redis vol as detached
root@localdev:/vagrant/localdev# pxctl volume list | grep 'redis\|NAME'
ID NAME SIZE HA SHARED ENCRYPTED PROXY-VOLUME IO_PRIORITY STATUS SNAP-ENABLED
1054097411774934751 redis-cache 1 GiB 1 no no no MEDIUM up - detached no
This is the new CSI volume definition that we will register in Nomad using nomad volume create:
id = "1107963972586667525" # Should match 'id' col from 'pxctl volume status'
name = "redis-cache" # Should match 'name' col from 'pxctl volume status'
type = "csi"
plugin_id = "portworx"
capability {
access_mode = "single-node-reader-only"
attachment_mode = "file-system"
}
capability {
access_mode = "single-node-writer"
attachment_mode = "file-system"
}
mount_options {
fs_type = "ext4"
mount_flags = ["noatime"]
}Register the volume like this (NOTE: The output MUST match the 'id' field in pxctl volume list, otherwise it
created a new volume, rather than registering an existing one):
# Create redis volume
root@localdev:/vagrant/localdev# nomad volume register redis.volume.hcl
Created external volume 1107963972586667525 with ID 1107963972586667525
# View Nomad volume status
root@localdev:/vagrant/localdev# nomad volume status
Container Storage Interface
ID Name Plugin ID Schedulable Access Mode
1107963972586667525 redis-cache portworx true <none>
# View pxctl volume status
root@localdev:/vagrant/localdev# pxctl volume list | grep 'redis\|NAME'
ID NAME SIZE HA SHARED ENCRYPTED PROXY-VOLUME IO_PRIORITY STATUS SNAP-ENABLED
1107963972586667525 redis-cache 1 GiB 1 no no no MEDIUM up - detached no
Everything looks great, so we should be able to update our Nomad Job now.
Update your nomad job HCL (an example is below).
- Remove the 'mounts' field from
job.task.group.confg
- Add new value for
job.task.group.volume
- Add new value for
job.task.group.volume_mount
job "redis" {
datacenters = ["localdev"]
group "cache" {
count = 1
# ADD For CSI Volume of this task (see redis.volume.hcl)
volume "redis-cache" {
type = "csi"
source = "1107963972586667525"
attachment_mode = "file-system"
access_mode = "single-node-writer"
}
network {
port "db" {
to = 6379
}
}
task "redis" {
driver = "docker"
config {
image = "redis:7"
# mounts = [ # REMOVE START
# {
# type = "volume"
# target = "/test"
# source = "redis-cache"
# readonly = false
# volume_options = [{
# driver_config = {
# name = "pxd"
# options = [{
# name = "redis-cache"
# size = "1"
# repl = "1"
# io_priority = "medium"
# io_profile = "auto"
# }]
# }
# }]
# }
# ] # REMOVE END
ports = ["db"]
}
resources {
cpu = 500
memory = 256
}
# ADD For CSI Volume mount of this task (see redis.volume.hcl)
volume_mount {
volume = "redis-cache"
destination = "/data"
}
}
}
}You can now re-run your Nomad job, and it shoudl automatically use our newly registered redis-cache volume:
root@localdev:/vagrant/localdev# nomad run --detach redis.hcl
Job registration successful
Evaluation ID: b98e4f3a-c3c2-5502-6c93-d8f3962c66e4
