Last active
August 15, 2017 19:29
-
-
Save rbocchinfuso/58a786ce1964cf3d512b5de91f5cdf62 to your computer and use it in GitHub Desktop.
zfs migration process
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Create source and target zfs pools for testing | |
sourcepool is a raidz1 stripe comprised of 4x1GB volumes and targetpool is a single 5GB device | |
[ root@unknown:/ ] $ zpool create sourcepool raidz1 c1t0d0 c1t1d0 c1t2d0 c1t3d0 | |
[ root@unknown:/ ] $ zpool create targetpool c1t4d0 | |
Check pools after creation | |
[ root@unknown:/ ] $ zpool status -v sourcepool | |
pool: sourcepool | |
state: ONLINE | |
scan: none requested | |
config: | |
NAME STATE READ WRITE CKSUM | |
sourcepool ONLINE 0 0 0 | |
raidz1-0 ONLINE 0 0 0 | |
c1t0d0 ONLINE 0 0 0 | |
c1t1d0 ONLINE 0 0 0 | |
c1t2d0 ONLINE 0 0 0 | |
c1t3d0 ONLINE 0 0 0 | |
errors: No known data errors | |
[ root@unknown:/ ] $ zpool status -v targetpool | |
pool: targetpool | |
state: ONLINE | |
scan: none requested | |
config: | |
NAME STATE READ WRITE CKSUM | |
targetpool ONLINE 0 0 0 | |
c1t4d0 ONLINE 0 0 0 | |
errors: No known data errors | |
[ root@unknown:/ ] $ zpool status -x | |
all pools are healthy | |
[ root@unknown:/ ] $ zpool list | |
NAME SIZE ALLOC FREE CAP HEALTH ALTROOT | |
rpool 31.8G 6.11G 25.6G 19% ONLINE - | |
sourcepool 3.94G 130K 3.94G 0% ONLINE - | |
targetpool 4.97G 77K 4.97G 0% ONLINE - | |
[ root@unknown:/ ] $ df -h | |
Filesystem size used avail capacity Mounted on | |
rpool/ROOT/s10_1-13 31G 4.1G 25G 15% / | |
/devices 0K 0K 0K 0% /devices | |
ctfs 0K 0K 0K 0% /system/contract | |
proc 0K 0K 0K 0% /proc | |
mnttab 0K 0K 0K 0% /etc/mnttab | |
swap 1.4G 356K 1.4G 1% /etc/svc/volatile | |
objfs 0K 0K 0K 0% /system/object | |
sharefs 0K 0K 0K 0% /etc/dfs/sharetab | |
/usr/lib/libc/libc_hwcap2.so.1 | |
29G 4.1G 25G 15% /lib/libc.so.1 | |
fd 0K 0K 0K 0% /dev/fd | |
swap 1.4G 72K 1.4G 1% /tmp | |
swap 1.4G 28K 1.4G 1% /var/run | |
rpool/export 31G 32K 25G 1% /export | |
rpool/export/home 31G 32K 25G 1% /export/home | |
rpool 31G 42K 25G 1% /rpool | |
sourcepool 2.9G 45K 2.9G 1% /sourcepool | |
targetpool 4.9G 31K 4.9G 1% /targetpool | |
create source filesystem on sourcepool | |
[ root@unknown:/ ] $ zfs create sourcepool/fs01 | |
[ root@unknown:/ ] $ zfs list | |
NAME USED AVAIL REFER MOUNTPOINT | |
rpool 6.23G 25.0G 42.5K /rpool | |
rpool/ROOT 4.16G 25.0G 31K legacy | |
rpool/ROOT/s10_1-13 4.16G 25.0G 4.16G / | |
rpool/dump 1.00G 25.0G 1.00G - | |
rpool/export 64K 25.0G 32K /export | |
rpool/export/home 32K 25.0G 32K /export/home | |
rpool/swap 1.06G 25.1G 1.00G - | |
sourcepool 254K 2.90G 44.9K /sourcepool | |
sourcepool/fs01 44.9K 2.90G 44.9K /sourcepool/fs01 | |
targetpool 97K 4.89G 31K /targetpool | |
[ root@unknown:/ ] $ df -h | |
Filesystem size used avail capacity Mounted on | |
rpool/ROOT/s10_1-13 31G 4.2G 25G 15% / | |
/devices 0K 0K 0K 0% /devices | |
ctfs 0K 0K 0K 0% /system/contract | |
proc 0K 0K 0K 0% /proc | |
mnttab 0K 0K 0K 0% /etc/mnttab | |
swap 1.0G 356K 1.0G 1% /etc/svc/volatile | |
objfs 0K 0K 0K 0% /system/object | |
sharefs 0K 0K 0K 0% /etc/dfs/sharetab | |
/usr/lib/libc/libc_hwcap2.so.1 | |
29G 4.2G 25G 15% /lib/libc.so.1 | |
fd 0K 0K 0K 0% /dev/fd | |
swap 1.0G 72K 1.0G 1% /tmp | |
swap 1.0G 28K 1.0G 1% /var/run | |
rpool/export 31G 32K 25G 1% /export | |
rpool/export/home 31G 32K 25G 1% /export/home | |
rpool 31G 42K 25G 1% /rpool | |
sourcepool 2.9G 45K 2.9G 1% /sourcepool | |
targetpool 4.9G 31K 4.9G 1% /targetpool | |
sourcepool/fs01 2.9G 45K 2.9G 1% /sourcepool/fs01 | |
[ root@unknown:/ ] $ | |
create some dummy data on fs01 | |
[ root@unknown:/ ] $ cd /sourcepool/fs01/ | |
[ root@unknown:/sourcepool/fs01 ] $ ls -al | |
total 8 | |
drwxr-xr-x 2 root root 2 Nov 17 19:44 . | |
drwxr-xr-x 3 root root 3 Nov 17 19:44 .. | |
[ root@unknown:/sourcepool/fs01 ] $ touch {a..z} | |
[ root@unknown:/sourcepool/fs01 ] $ ls -al | |
total 34 | |
drwxr-xr-x 2 root root 28 Nov 17 19:45 . | |
drwxr-xr-x 3 root root 3 Nov 17 19:44 .. | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 a | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 b | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 c | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 d | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 e | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 f | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 g | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 h | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 i | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 j | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 k | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 l | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 m | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 n | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 o | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 p | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 q | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 r | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 s | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 t | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 u | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 v | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 w | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 x | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 y | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 z | |
[ root@unknown:/sourcepool/fs01 ] $ | |
Install pipe view so you can monitor transfer status | |
pkgadd -d http://get.opencsw.org/now | |
/opt/csw/bin/pkgutil -U | |
/opt/csw/bin/pkgutil -y -i pv | |
/usr/sbin/pkgchk -L CSWpv # list files | |
/opt/csw/bin/pv | |
Migration Process | |
This process allows for incrmental syncronoization. Applications can stay up until the final snapshot and sync. | |
Inital Sync | |
Take snashot | |
[ root@unknown:/ ] $ zfs snapshot sourcepool/fs01@snapshot01 | |
[ root@unknown:/ ] $ zfs list -t snapshot | |
NAME USED AVAIL REFER MOUNTPOINT | |
sourcepool/sourcefs01@snapshot01 0 - 49.4K - | |
Sync | |
[ root@unknown:/ ] $ zfs send sourcepool/fs01@snapshot01 | /opt/csw/bin/pv | zfs receive targetpool/fs01 | |
76.1KiB 0:00:00 [ 1.5MiB/s] [ <=> ] | |
[ root@unknown:/ ] $ | |
After the sync you will see the fs01 appear in the targetpool | |
[ root@unknown:/ ] $ df -h | |
Filesystem size used avail capacity Mounted on | |
rpool/ROOT/s10_1-13 31G 4.2G 25G 15% / | |
/devices 0K 0K 0K 0% /devices | |
ctfs 0K 0K 0K 0% /system/contract | |
proc 0K 0K 0K 0% /proc | |
mnttab 0K 0K 0K 0% /etc/mnttab | |
swap 1.0G 356K 1.0G 1% /etc/svc/volatile | |
objfs 0K 0K 0K 0% /system/object | |
sharefs 0K 0K 0K 0% /etc/dfs/sharetab | |
/usr/lib/libc/libc_hwcap2.so.1 | |
29G 4.2G 25G 15% /lib/libc.so.1 | |
fd 0K 0K 0K 0% /dev/fd | |
swap 1.0G 72K 1.0G 1% /tmp | |
swap 1.0G 28K 1.0G 1% /var/run | |
rpool/export 31G 32K 25G 1% /export | |
rpool/export/home 31G 32K 25G 1% /export/home | |
rpool 31G 42K 25G 1% /rpool | |
sourcepool 2.9G 49K 2.9G 1% /sourcepool | |
targetpool 4.9G 32K 4.9G 1% /targetpool | |
sourcepool/fs01 2.9G 49K 2.9G 1% /sourcepool/fs01 | |
targetpool/fs01 4.9G 34K 4.9G 1% /targetpool/fs01 | |
[ root@unknown:/ ] $ zfs list | |
NAME USED AVAIL REFER MOUNTPOINT | |
rpool 6.23G 25.0G 42.5K /rpool | |
rpool/ROOT 4.16G 25.0G 31K legacy | |
rpool/ROOT/s10_1-13 4.16G 25.0G 4.16G / | |
rpool/dump 1.00G 25.0G 1.00G - | |
rpool/export 64K 25.0G 32K /export | |
rpool/export/home 32K 25.0G 32K /export/home | |
rpool/swap 1.06G 25.1G 1.00G - | |
sourcepool 163K 2.90G 49.4K /sourcepool | |
sourcepool/fs01 49.4K 2.90G 49.4K /sourcepool/fs01 | |
sourcepool/fs01@snapshot01 0 - 49.4K - | |
targetpool 140K 4.89G 32K /targetpool | |
targetpool/fs01 34K 4.89G 34K /targetpool/fs01 | |
targetpool/fs01@snapshot01 0 - 34K - | |
[ root@unknown:/ ] $ | |
Cutover | |
Shutdown all applications | |
Unmount source filesystem(s) | |
Note: Be sure you are not in the filesystem you are trying to unmount | |
[ root@unknown:/ ] $ zfs umount sourcepool/fs01 | |
[ root@unknown:/ ] $ df -h | |
Filesystem size used avail capacity Mounted on | |
rpool/ROOT/s10_1-13 31G 4.2G 25G 15% / | |
/devices 0K 0K 0K 0% /devices | |
ctfs 0K 0K 0K 0% /system/contract | |
proc 0K 0K 0K 0% /proc | |
mnttab 0K 0K 0K 0% /etc/mnttab | |
swap 1017M 356K 1017M 1% /etc/svc/volatile | |
objfs 0K 0K 0K 0% /system/object | |
sharefs 0K 0K 0K 0% /etc/dfs/sharetab | |
/usr/lib/libc/libc_hwcap2.so.1 | |
29G 4.2G 25G 15% /lib/libc.so.1 | |
fd 0K 0K 0K 0% /dev/fd | |
swap 1017M 72K 1017M 1% /tmp | |
swap 1017M 28K 1017M 1% /var/run | |
rpool/export 31G 32K 25G 1% /export | |
rpool/export/home 31G 32K 25G 1% /export/home | |
rpool 31G 42K 25G 1% /rpool | |
sourcepool 2.9G 49K 2.9G 1% /sourcepool | |
targetpool 4.9G 32K 4.9G 1% /targetpool | |
targetpool/fs01 4.9G 34K 4.9G 1% /targetpool/fs01 | |
[ root@unknown:/ ] $ | |
take new snapshot | |
[ root@unknown:/ ] $ zfs snapshot sourcepool/fs01@snapshot02 | |
[ root@unknown:/ ] $ zfs list -t snapshot | |
NAME USED AVAIL REFER MOUNTPOINT | |
sourcepool/fs01@snapshot01 28.4K - 49.4K - | |
sourcepool/fs01@snapshot02 0 - 49.4K - | |
targetpool/fs01@snapshot01 0 - 34K - | |
[ root@unknown:/ ] $ | |
Sync incremental differences between two snapshots | |
[ root@unknown:/ ] $ zfs send -i sourcepool/fs01@snapshot01 sourcepool/fs01@snapshot02 | /opt/csw/bin/pv | zfs receive targetpool/fs01 | |
33.2KiB 0:00:00 [ 679KiB/s] [ <=> ] | |
[ root@unknown:/ ] $ | |
Make targetpool the sourcepool | |
Export sourepool and targetpool | |
[ root@unknown:/ ] $ zpool export sourcepool | |
[ root@unknown:/ ] $ zpool status -v | |
pool: rpool | |
state: ONLINE | |
scan: none requested | |
config: | |
NAME STATE READ WRITE CKSUM | |
rpool ONLINE 0 0 0 | |
c0d0s0 ONLINE 0 0 0 | |
errors: No known data errors | |
pool: targetpool | |
state: ONLINE | |
scan: none requested | |
config: | |
NAME STATE READ WRITE CKSUM | |
targetpool ONLINE 0 0 0 | |
c1t4d0 ONLINE 0 0 0 | |
errors: No known data errors | |
[ root@unknown:/ ] $ zpool export targetpool | |
[ root@unknown:/ ] $ zpool status -v | |
pool: rpool | |
state: ONLINE | |
scan: none requested | |
config: | |
NAME STATE READ WRITE CKSUM | |
rpool ONLINE 0 0 0 | |
c0d0s0 ONLINE 0 0 0 | |
errors: No known data errors | |
[ root@unknown:/ ] $ | |
List pools available for import | |
[ root@unknown:/ ] $ zpool import | |
pool: targetpool | |
id: 6974299001134861399 | |
state: ONLINE | |
action: The pool can be imported using its name or numeric identifier. | |
config: | |
targetpool ONLINE | |
c1t4d0 ONLINE | |
pool: sourcepool | |
id: 16188486505158652390 | |
state: ONLINE | |
action: The pool can be imported using its name or numeric identifier. | |
config: | |
sourcepool ONLINE | |
raidz1-0 ONLINE | |
c1t0d0 ONLINE | |
c1t1d0 ONLINE | |
c1t2d0 ONLINE | |
c1t3d0 ONLINE | |
import original sourcepool as old_sourcepool | |
[ root@unknown:/ ] $ zpool import sourcepool old_sourcepool | |
import targetpool as sourcepool | |
[ root@unknown:/ ] $ zpool import targetpool sourcepool | |
validate pool swap | |
[ root@unknown:/ ] $ zpool status -v | |
pool: old_sourcepool | |
state: ONLINE | |
scan: none requested | |
config: | |
NAME STATE READ WRITE CKSUM | |
old_sourcepool ONLINE 0 0 0 | |
raidz1-0 ONLINE 0 0 0 | |
c1t0d0 ONLINE 0 0 0 | |
c1t1d0 ONLINE 0 0 0 | |
c1t2d0 ONLINE 0 0 0 | |
c1t3d0 ONLINE 0 0 0 | |
errors: No known data errors | |
pool: rpool | |
state: ONLINE | |
scan: none requested | |
config: | |
NAME STATE READ WRITE CKSUM | |
rpool ONLINE 0 0 0 | |
c0d0s0 ONLINE 0 0 0 | |
errors: No known data errors | |
pool: sourcepool | |
state: ONLINE | |
scan: none requested | |
config: | |
NAME STATE READ WRITE CKSUM | |
sourcepool ONLINE 0 0 0 | |
c1t4d0 ONLINE 0 0 0 | |
errors: No known data errors | |
[ root@unknown:/ ] $ | |
snapshots intact | |
[ root@unknown:/ ] $ zfs list -t snapshot | |
NAME USED AVAIL REFER MOUNTPOINT | |
old_sourcepool/fs01@snapshot01 28.4K - 49.4K - | |
old_sourcepool/fs01@snapshot02 0 - 49.4K - | |
sourcepool/fs01@snapshot01 19K - 34K - | |
sourcepool/fs01@snapshot02 0 - 34K - | |
[ root@unknown:/ ] $ | |
devices mounted | |
[ root@unknown:/ ] $ df -h | |
Filesystem size used avail capacity Mounted on | |
rpool/ROOT/s10_1-13 31G 4.2G 25G 15% / | |
/devices 0K 0K 0K 0% /devices | |
ctfs 0K 0K 0K 0% /system/contract | |
proc 0K 0K 0K 0% /proc | |
mnttab 0K 0K 0K 0% /etc/mnttab | |
swap 1014M 356K 1013M 1% /etc/svc/volatile | |
objfs 0K 0K 0K 0% /system/object | |
sharefs 0K 0K 0K 0% /etc/dfs/sharetab | |
/usr/lib/libc/libc_hwcap2.so.1 | |
29G 4.2G 25G 15% /lib/libc.so.1 | |
fd 0K 0K 0K 0% /dev/fd | |
swap 1013M 72K 1013M 1% /tmp | |
swap 1013M 28K 1013M 1% /var/run | |
rpool/export 31G 32K 25G 1% /export | |
rpool/export/home 31G 32K 25G 1% /export/home | |
rpool 31G 42K 25G 1% /rpool | |
old_sourcepool 2.9G 49K 2.9G 1% /old_sourcepool | |
old_sourcepool/fs01 2.9G 49K 2.9G 1% /old_sourcepool/fs01 | |
sourcepool 4.9G 32K 4.9G 1% /sourcepool | |
sourcepool/fs01 4.9G 34K 4.9G 1% /sourcepool/fs01 | |
[ root@unknown:/ ] $ zfs list | |
NAME USED AVAIL REFER MOUNTPOINT | |
old_sourcepool 211K 2.90G 49.4K /old_sourcepool | |
old_sourcepool/fs01 77.8K 2.90G 49.4K /old_sourcepool/fs01 | |
old_sourcepool/fs01@snapshot01 28.4K - 49.4K - | |
old_sourcepool/fs01@snapshot02 0 - 49.4K - | |
rpool 6.24G 25.0G 42.5K /rpool | |
rpool/ROOT 4.17G 25.0G 31K legacy | |
rpool/ROOT/s10_1-13 4.17G 25.0G 4.17G / | |
rpool/dump 1.00G 25.0G 1.00G - | |
rpool/export 64K 25.0G 32K /export | |
rpool/export/home 32K 25.0G 32K /export/home | |
rpool/swap 1.06G 25.1G 1.00G - | |
sourcepool 187K 4.89G 32K /sourcepool | |
sourcepool/fs01 53K 4.89G 34K /sourcepool/fs01 | |
sourcepool/fs01@snapshot01 19K - 34K - | |
sourcepool/fs01@snapshot02 0 - 34K - | |
[ root@unknown:/ ] $ | |
data intact | |
[ root@unknown:/ ] $ cd /sourcepool/fs01/ | |
[ root@unknown:/sourcepool/fs01 ] $ ls -al | |
total 32 | |
drwxr-xr-x 2 root root 28 Nov 17 19:45 . | |
drwxr-xr-x 3 root root 3 Nov 17 19:49 .. | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 a | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 b | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 c | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 d | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 e | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 f | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 g | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 h | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 i | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 j | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 k | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 l | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 m | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 n | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 o | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 p | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 q | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 r | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 s | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 t | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 u | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 v | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 w | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 x | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 y | |
-rw-r--r-- 1 root root 0 Nov 17 19:45 z | |
[ root@unknown:/sourcepool/fs01 ] $ | |
Should now be able to restart applications. | |
Last step is to cleanup and remove old zpool when confident that all is good. | |
Snapshot cleanup | |
[ root@unknown:/sourcepool/fs01 ] $ zfs list -t snapshot | |
NAME USED AVAIL REFER MOUNTPOINT | |
old_sourcepool/fs01@snapshot01 28.4K - 49.4K - | |
old_sourcepool/fs01@snapshot02 0 - 49.4K - | |
sourcepool/fs01@snapshot01 19K - 34K - | |
sourcepool/fs01@snapshot02 0 - 34K - | |
[ root@unknown:/sourcepool/fs01 ] $ zfs destroy old_sourcepool/fs01@snapshot01 | |
[ root@unknown:/sourcepool/fs01 ] $ zfs destroy old_sourcepool/fs01@snapshot02 | |
[ root@unknown:/sourcepool/fs01 ] $ zfs destroy sourcepool/fs01@snapshot01 | |
[ root@unknown:/sourcepool/fs01 ] $ zfs destroy sourcepool/fs01@snapshot02 | |
[ root@unknown:/sourcepool/fs01 ] $ zfs list -t snapshot | |
no datasets available | |
[ root@unknown:/sourcepool/fs01 ] $ | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment