Skip to content

Instantly share code, notes, and snippets.

@rbocchinfuso
Last active August 15, 2017 19:29
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rbocchinfuso/58a786ce1964cf3d512b5de91f5cdf62 to your computer and use it in GitHub Desktop.
Save rbocchinfuso/58a786ce1964cf3d512b5de91f5cdf62 to your computer and use it in GitHub Desktop.
zfs migration process
Create source and target zfs pools for testing
sourcepool is a raidz1 stripe comprised of 4x1GB volumes and targetpool is a single 5GB device
[ root@unknown:/ ] $ zpool create sourcepool raidz1 c1t0d0 c1t1d0 c1t2d0 c1t3d0
[ root@unknown:/ ] $ zpool create targetpool c1t4d0
Check pools after creation
[ root@unknown:/ ] $ zpool status -v sourcepool
pool: sourcepool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
sourcepool ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
c1t0d0 ONLINE 0 0 0
c1t1d0 ONLINE 0 0 0
c1t2d0 ONLINE 0 0 0
c1t3d0 ONLINE 0 0 0
errors: No known data errors
[ root@unknown:/ ] $ zpool status -v targetpool
pool: targetpool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
targetpool ONLINE 0 0 0
c1t4d0 ONLINE 0 0 0
errors: No known data errors
[ root@unknown:/ ] $ zpool status -x
all pools are healthy
[ root@unknown:/ ] $ zpool list
NAME SIZE ALLOC FREE CAP HEALTH ALTROOT
rpool 31.8G 6.11G 25.6G 19% ONLINE -
sourcepool 3.94G 130K 3.94G 0% ONLINE -
targetpool 4.97G 77K 4.97G 0% ONLINE -
[ root@unknown:/ ] $ df -h
Filesystem size used avail capacity Mounted on
rpool/ROOT/s10_1-13 31G 4.1G 25G 15% /
/devices 0K 0K 0K 0% /devices
ctfs 0K 0K 0K 0% /system/contract
proc 0K 0K 0K 0% /proc
mnttab 0K 0K 0K 0% /etc/mnttab
swap 1.4G 356K 1.4G 1% /etc/svc/volatile
objfs 0K 0K 0K 0% /system/object
sharefs 0K 0K 0K 0% /etc/dfs/sharetab
/usr/lib/libc/libc_hwcap2.so.1
29G 4.1G 25G 15% /lib/libc.so.1
fd 0K 0K 0K 0% /dev/fd
swap 1.4G 72K 1.4G 1% /tmp
swap 1.4G 28K 1.4G 1% /var/run
rpool/export 31G 32K 25G 1% /export
rpool/export/home 31G 32K 25G 1% /export/home
rpool 31G 42K 25G 1% /rpool
sourcepool 2.9G 45K 2.9G 1% /sourcepool
targetpool 4.9G 31K 4.9G 1% /targetpool
create source filesystem on sourcepool
[ root@unknown:/ ] $ zfs create sourcepool/fs01
[ root@unknown:/ ] $ zfs list
NAME USED AVAIL REFER MOUNTPOINT
rpool 6.23G 25.0G 42.5K /rpool
rpool/ROOT 4.16G 25.0G 31K legacy
rpool/ROOT/s10_1-13 4.16G 25.0G 4.16G /
rpool/dump 1.00G 25.0G 1.00G -
rpool/export 64K 25.0G 32K /export
rpool/export/home 32K 25.0G 32K /export/home
rpool/swap 1.06G 25.1G 1.00G -
sourcepool 254K 2.90G 44.9K /sourcepool
sourcepool/fs01 44.9K 2.90G 44.9K /sourcepool/fs01
targetpool 97K 4.89G 31K /targetpool
[ root@unknown:/ ] $ df -h
Filesystem size used avail capacity Mounted on
rpool/ROOT/s10_1-13 31G 4.2G 25G 15% /
/devices 0K 0K 0K 0% /devices
ctfs 0K 0K 0K 0% /system/contract
proc 0K 0K 0K 0% /proc
mnttab 0K 0K 0K 0% /etc/mnttab
swap 1.0G 356K 1.0G 1% /etc/svc/volatile
objfs 0K 0K 0K 0% /system/object
sharefs 0K 0K 0K 0% /etc/dfs/sharetab
/usr/lib/libc/libc_hwcap2.so.1
29G 4.2G 25G 15% /lib/libc.so.1
fd 0K 0K 0K 0% /dev/fd
swap 1.0G 72K 1.0G 1% /tmp
swap 1.0G 28K 1.0G 1% /var/run
rpool/export 31G 32K 25G 1% /export
rpool/export/home 31G 32K 25G 1% /export/home
rpool 31G 42K 25G 1% /rpool
sourcepool 2.9G 45K 2.9G 1% /sourcepool
targetpool 4.9G 31K 4.9G 1% /targetpool
sourcepool/fs01 2.9G 45K 2.9G 1% /sourcepool/fs01
[ root@unknown:/ ] $
create some dummy data on fs01
[ root@unknown:/ ] $ cd /sourcepool/fs01/
[ root@unknown:/sourcepool/fs01 ] $ ls -al
total 8
drwxr-xr-x 2 root root 2 Nov 17 19:44 .
drwxr-xr-x 3 root root 3 Nov 17 19:44 ..
[ root@unknown:/sourcepool/fs01 ] $ touch {a..z}
[ root@unknown:/sourcepool/fs01 ] $ ls -al
total 34
drwxr-xr-x 2 root root 28 Nov 17 19:45 .
drwxr-xr-x 3 root root 3 Nov 17 19:44 ..
-rw-r--r-- 1 root root 0 Nov 17 19:45 a
-rw-r--r-- 1 root root 0 Nov 17 19:45 b
-rw-r--r-- 1 root root 0 Nov 17 19:45 c
-rw-r--r-- 1 root root 0 Nov 17 19:45 d
-rw-r--r-- 1 root root 0 Nov 17 19:45 e
-rw-r--r-- 1 root root 0 Nov 17 19:45 f
-rw-r--r-- 1 root root 0 Nov 17 19:45 g
-rw-r--r-- 1 root root 0 Nov 17 19:45 h
-rw-r--r-- 1 root root 0 Nov 17 19:45 i
-rw-r--r-- 1 root root 0 Nov 17 19:45 j
-rw-r--r-- 1 root root 0 Nov 17 19:45 k
-rw-r--r-- 1 root root 0 Nov 17 19:45 l
-rw-r--r-- 1 root root 0 Nov 17 19:45 m
-rw-r--r-- 1 root root 0 Nov 17 19:45 n
-rw-r--r-- 1 root root 0 Nov 17 19:45 o
-rw-r--r-- 1 root root 0 Nov 17 19:45 p
-rw-r--r-- 1 root root 0 Nov 17 19:45 q
-rw-r--r-- 1 root root 0 Nov 17 19:45 r
-rw-r--r-- 1 root root 0 Nov 17 19:45 s
-rw-r--r-- 1 root root 0 Nov 17 19:45 t
-rw-r--r-- 1 root root 0 Nov 17 19:45 u
-rw-r--r-- 1 root root 0 Nov 17 19:45 v
-rw-r--r-- 1 root root 0 Nov 17 19:45 w
-rw-r--r-- 1 root root 0 Nov 17 19:45 x
-rw-r--r-- 1 root root 0 Nov 17 19:45 y
-rw-r--r-- 1 root root 0 Nov 17 19:45 z
[ root@unknown:/sourcepool/fs01 ] $
Install pipe view so you can monitor transfer status
pkgadd -d http://get.opencsw.org/now
/opt/csw/bin/pkgutil -U
/opt/csw/bin/pkgutil -y -i pv
/usr/sbin/pkgchk -L CSWpv # list files
/opt/csw/bin/pv
Migration Process
This process allows for incrmental syncronoization. Applications can stay up until the final snapshot and sync.
Inital Sync
Take snashot
[ root@unknown:/ ] $ zfs snapshot sourcepool/fs01@snapshot01
[ root@unknown:/ ] $ zfs list -t snapshot
NAME USED AVAIL REFER MOUNTPOINT
sourcepool/sourcefs01@snapshot01 0 - 49.4K -
Sync
[ root@unknown:/ ] $ zfs send sourcepool/fs01@snapshot01 | /opt/csw/bin/pv | zfs receive targetpool/fs01
76.1KiB 0:00:00 [ 1.5MiB/s] [ <=> ]
[ root@unknown:/ ] $
After the sync you will see the fs01 appear in the targetpool
[ root@unknown:/ ] $ df -h
Filesystem size used avail capacity Mounted on
rpool/ROOT/s10_1-13 31G 4.2G 25G 15% /
/devices 0K 0K 0K 0% /devices
ctfs 0K 0K 0K 0% /system/contract
proc 0K 0K 0K 0% /proc
mnttab 0K 0K 0K 0% /etc/mnttab
swap 1.0G 356K 1.0G 1% /etc/svc/volatile
objfs 0K 0K 0K 0% /system/object
sharefs 0K 0K 0K 0% /etc/dfs/sharetab
/usr/lib/libc/libc_hwcap2.so.1
29G 4.2G 25G 15% /lib/libc.so.1
fd 0K 0K 0K 0% /dev/fd
swap 1.0G 72K 1.0G 1% /tmp
swap 1.0G 28K 1.0G 1% /var/run
rpool/export 31G 32K 25G 1% /export
rpool/export/home 31G 32K 25G 1% /export/home
rpool 31G 42K 25G 1% /rpool
sourcepool 2.9G 49K 2.9G 1% /sourcepool
targetpool 4.9G 32K 4.9G 1% /targetpool
sourcepool/fs01 2.9G 49K 2.9G 1% /sourcepool/fs01
targetpool/fs01 4.9G 34K 4.9G 1% /targetpool/fs01
[ root@unknown:/ ] $ zfs list
NAME USED AVAIL REFER MOUNTPOINT
rpool 6.23G 25.0G 42.5K /rpool
rpool/ROOT 4.16G 25.0G 31K legacy
rpool/ROOT/s10_1-13 4.16G 25.0G 4.16G /
rpool/dump 1.00G 25.0G 1.00G -
rpool/export 64K 25.0G 32K /export
rpool/export/home 32K 25.0G 32K /export/home
rpool/swap 1.06G 25.1G 1.00G -
sourcepool 163K 2.90G 49.4K /sourcepool
sourcepool/fs01 49.4K 2.90G 49.4K /sourcepool/fs01
sourcepool/fs01@snapshot01 0 - 49.4K -
targetpool 140K 4.89G 32K /targetpool
targetpool/fs01 34K 4.89G 34K /targetpool/fs01
targetpool/fs01@snapshot01 0 - 34K -
[ root@unknown:/ ] $
Cutover
Shutdown all applications
Unmount source filesystem(s)
Note: Be sure you are not in the filesystem you are trying to unmount
[ root@unknown:/ ] $ zfs umount sourcepool/fs01
[ root@unknown:/ ] $ df -h
Filesystem size used avail capacity Mounted on
rpool/ROOT/s10_1-13 31G 4.2G 25G 15% /
/devices 0K 0K 0K 0% /devices
ctfs 0K 0K 0K 0% /system/contract
proc 0K 0K 0K 0% /proc
mnttab 0K 0K 0K 0% /etc/mnttab
swap 1017M 356K 1017M 1% /etc/svc/volatile
objfs 0K 0K 0K 0% /system/object
sharefs 0K 0K 0K 0% /etc/dfs/sharetab
/usr/lib/libc/libc_hwcap2.so.1
29G 4.2G 25G 15% /lib/libc.so.1
fd 0K 0K 0K 0% /dev/fd
swap 1017M 72K 1017M 1% /tmp
swap 1017M 28K 1017M 1% /var/run
rpool/export 31G 32K 25G 1% /export
rpool/export/home 31G 32K 25G 1% /export/home
rpool 31G 42K 25G 1% /rpool
sourcepool 2.9G 49K 2.9G 1% /sourcepool
targetpool 4.9G 32K 4.9G 1% /targetpool
targetpool/fs01 4.9G 34K 4.9G 1% /targetpool/fs01
[ root@unknown:/ ] $
take new snapshot
[ root@unknown:/ ] $ zfs snapshot sourcepool/fs01@snapshot02
[ root@unknown:/ ] $ zfs list -t snapshot
NAME USED AVAIL REFER MOUNTPOINT
sourcepool/fs01@snapshot01 28.4K - 49.4K -
sourcepool/fs01@snapshot02 0 - 49.4K -
targetpool/fs01@snapshot01 0 - 34K -
[ root@unknown:/ ] $
Sync incremental differences between two snapshots
[ root@unknown:/ ] $ zfs send -i sourcepool/fs01@snapshot01 sourcepool/fs01@snapshot02 | /opt/csw/bin/pv | zfs receive targetpool/fs01
33.2KiB 0:00:00 [ 679KiB/s] [ <=> ]
[ root@unknown:/ ] $
Make targetpool the sourcepool
Export sourepool and targetpool
[ root@unknown:/ ] $ zpool export sourcepool
[ root@unknown:/ ] $ zpool status -v
pool: rpool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
c0d0s0 ONLINE 0 0 0
errors: No known data errors
pool: targetpool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
targetpool ONLINE 0 0 0
c1t4d0 ONLINE 0 0 0
errors: No known data errors
[ root@unknown:/ ] $ zpool export targetpool
[ root@unknown:/ ] $ zpool status -v
pool: rpool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
c0d0s0 ONLINE 0 0 0
errors: No known data errors
[ root@unknown:/ ] $
List pools available for import
[ root@unknown:/ ] $ zpool import
pool: targetpool
id: 6974299001134861399
state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:
targetpool ONLINE
c1t4d0 ONLINE
pool: sourcepool
id: 16188486505158652390
state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:
sourcepool ONLINE
raidz1-0 ONLINE
c1t0d0 ONLINE
c1t1d0 ONLINE
c1t2d0 ONLINE
c1t3d0 ONLINE
import original sourcepool as old_sourcepool
[ root@unknown:/ ] $ zpool import sourcepool old_sourcepool
import targetpool as sourcepool
[ root@unknown:/ ] $ zpool import targetpool sourcepool
validate pool swap
[ root@unknown:/ ] $ zpool status -v
pool: old_sourcepool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
old_sourcepool ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
c1t0d0 ONLINE 0 0 0
c1t1d0 ONLINE 0 0 0
c1t2d0 ONLINE 0 0 0
c1t3d0 ONLINE 0 0 0
errors: No known data errors
pool: rpool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
c0d0s0 ONLINE 0 0 0
errors: No known data errors
pool: sourcepool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
sourcepool ONLINE 0 0 0
c1t4d0 ONLINE 0 0 0
errors: No known data errors
[ root@unknown:/ ] $
snapshots intact
[ root@unknown:/ ] $ zfs list -t snapshot
NAME USED AVAIL REFER MOUNTPOINT
old_sourcepool/fs01@snapshot01 28.4K - 49.4K -
old_sourcepool/fs01@snapshot02 0 - 49.4K -
sourcepool/fs01@snapshot01 19K - 34K -
sourcepool/fs01@snapshot02 0 - 34K -
[ root@unknown:/ ] $
devices mounted
[ root@unknown:/ ] $ df -h
Filesystem size used avail capacity Mounted on
rpool/ROOT/s10_1-13 31G 4.2G 25G 15% /
/devices 0K 0K 0K 0% /devices
ctfs 0K 0K 0K 0% /system/contract
proc 0K 0K 0K 0% /proc
mnttab 0K 0K 0K 0% /etc/mnttab
swap 1014M 356K 1013M 1% /etc/svc/volatile
objfs 0K 0K 0K 0% /system/object
sharefs 0K 0K 0K 0% /etc/dfs/sharetab
/usr/lib/libc/libc_hwcap2.so.1
29G 4.2G 25G 15% /lib/libc.so.1
fd 0K 0K 0K 0% /dev/fd
swap 1013M 72K 1013M 1% /tmp
swap 1013M 28K 1013M 1% /var/run
rpool/export 31G 32K 25G 1% /export
rpool/export/home 31G 32K 25G 1% /export/home
rpool 31G 42K 25G 1% /rpool
old_sourcepool 2.9G 49K 2.9G 1% /old_sourcepool
old_sourcepool/fs01 2.9G 49K 2.9G 1% /old_sourcepool/fs01
sourcepool 4.9G 32K 4.9G 1% /sourcepool
sourcepool/fs01 4.9G 34K 4.9G 1% /sourcepool/fs01
[ root@unknown:/ ] $ zfs list
NAME USED AVAIL REFER MOUNTPOINT
old_sourcepool 211K 2.90G 49.4K /old_sourcepool
old_sourcepool/fs01 77.8K 2.90G 49.4K /old_sourcepool/fs01
old_sourcepool/fs01@snapshot01 28.4K - 49.4K -
old_sourcepool/fs01@snapshot02 0 - 49.4K -
rpool 6.24G 25.0G 42.5K /rpool
rpool/ROOT 4.17G 25.0G 31K legacy
rpool/ROOT/s10_1-13 4.17G 25.0G 4.17G /
rpool/dump 1.00G 25.0G 1.00G -
rpool/export 64K 25.0G 32K /export
rpool/export/home 32K 25.0G 32K /export/home
rpool/swap 1.06G 25.1G 1.00G -
sourcepool 187K 4.89G 32K /sourcepool
sourcepool/fs01 53K 4.89G 34K /sourcepool/fs01
sourcepool/fs01@snapshot01 19K - 34K -
sourcepool/fs01@snapshot02 0 - 34K -
[ root@unknown:/ ] $
data intact
[ root@unknown:/ ] $ cd /sourcepool/fs01/
[ root@unknown:/sourcepool/fs01 ] $ ls -al
total 32
drwxr-xr-x 2 root root 28 Nov 17 19:45 .
drwxr-xr-x 3 root root 3 Nov 17 19:49 ..
-rw-r--r-- 1 root root 0 Nov 17 19:45 a
-rw-r--r-- 1 root root 0 Nov 17 19:45 b
-rw-r--r-- 1 root root 0 Nov 17 19:45 c
-rw-r--r-- 1 root root 0 Nov 17 19:45 d
-rw-r--r-- 1 root root 0 Nov 17 19:45 e
-rw-r--r-- 1 root root 0 Nov 17 19:45 f
-rw-r--r-- 1 root root 0 Nov 17 19:45 g
-rw-r--r-- 1 root root 0 Nov 17 19:45 h
-rw-r--r-- 1 root root 0 Nov 17 19:45 i
-rw-r--r-- 1 root root 0 Nov 17 19:45 j
-rw-r--r-- 1 root root 0 Nov 17 19:45 k
-rw-r--r-- 1 root root 0 Nov 17 19:45 l
-rw-r--r-- 1 root root 0 Nov 17 19:45 m
-rw-r--r-- 1 root root 0 Nov 17 19:45 n
-rw-r--r-- 1 root root 0 Nov 17 19:45 o
-rw-r--r-- 1 root root 0 Nov 17 19:45 p
-rw-r--r-- 1 root root 0 Nov 17 19:45 q
-rw-r--r-- 1 root root 0 Nov 17 19:45 r
-rw-r--r-- 1 root root 0 Nov 17 19:45 s
-rw-r--r-- 1 root root 0 Nov 17 19:45 t
-rw-r--r-- 1 root root 0 Nov 17 19:45 u
-rw-r--r-- 1 root root 0 Nov 17 19:45 v
-rw-r--r-- 1 root root 0 Nov 17 19:45 w
-rw-r--r-- 1 root root 0 Nov 17 19:45 x
-rw-r--r-- 1 root root 0 Nov 17 19:45 y
-rw-r--r-- 1 root root 0 Nov 17 19:45 z
[ root@unknown:/sourcepool/fs01 ] $
Should now be able to restart applications.
Last step is to cleanup and remove old zpool when confident that all is good.
Snapshot cleanup
[ root@unknown:/sourcepool/fs01 ] $ zfs list -t snapshot
NAME USED AVAIL REFER MOUNTPOINT
old_sourcepool/fs01@snapshot01 28.4K - 49.4K -
old_sourcepool/fs01@snapshot02 0 - 49.4K -
sourcepool/fs01@snapshot01 19K - 34K -
sourcepool/fs01@snapshot02 0 - 34K -
[ root@unknown:/sourcepool/fs01 ] $ zfs destroy old_sourcepool/fs01@snapshot01
[ root@unknown:/sourcepool/fs01 ] $ zfs destroy old_sourcepool/fs01@snapshot02
[ root@unknown:/sourcepool/fs01 ] $ zfs destroy sourcepool/fs01@snapshot01
[ root@unknown:/sourcepool/fs01 ] $ zfs destroy sourcepool/fs01@snapshot02
[ root@unknown:/sourcepool/fs01 ] $ zfs list -t snapshot
no datasets available
[ root@unknown:/sourcepool/fs01 ] $
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment