Skip to content

Instantly share code, notes, and snippets.

@dlangille
Last active February 23, 2016 20:28
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dlangille/f08e68d70af05b00ee1f to your computer and use it in GitHub Desktop.
Save dlangille/f08e68d70af05b00ee1f to your computer and use it in GitHub Desktop.
and now we wait
root@mfsbsd:~ # zpool status
pool: system
state: DEGRADED
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Mon Feb 22 19:14:30 2016
39.6G scanned out of 20.0T at 72.3M/s, 80h14m to go
3.73G resilvered, 0.19% done
config:
NAME STATE READ WRITE CKSUM
system DEGRADED 0 0 0
raidz2-0 DEGRADED 0 0 0
gptid/1f65b3e4-efc8-11e2-92b7-00259082215a ONLINE 0 0 0
gptid/21f2788d-efc8-11e2-92b7-00259082215a ONLINE 0 0 0
gptid/2484e4ef-efc8-11e2-92b7-00259082215a ONLINE 0 0 0
replacing-3 OFFLINE 0 0 0
1601580527160658707 OFFLINE 0 0 0 was /dev/da3p3
gptid/b0737f6b-d996-11e5-9b1d-00259082215a ONLINE 0 0 0 (resilvering)
gptid/29b08a29-efc8-11e2-92b7-00259082215a ONLINE 0 0 0
gptid/2c5b024e-efc8-11e2-92b7-00259082215a ONLINE 0 0 0
gptid/cf46c81a-d6ba-11e5-8298-00259082215a ONLINE 0 0 0
gptid/31e93100-efc8-11e2-92b7-00259082215a ONLINE 0 0 0
gptid/329ad7d7-efc8-11e2-92b7-00259082215a ONLINE 0 0 0
gptid/3412b7e6-efc8-11e2-92b7-00259082215a ONLINE 0 0 112 (resilvering)
errors: No known data errors
root@mfsbsd:~ #
root@mfsbsd:~ # zpool status
pool: system
state: DEGRADED
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Mon Feb 22 19:14:30 2016
689G scanned out of 20.0T at 298M/s, 18h50m to go
65.2G resilvered, 3.37% done
config:
NAME STATE READ WRITE CKSUM
system DEGRADED 0 0 0
raidz2-0 DEGRADED 0 0 0
gptid/1f65b3e4-efc8-11e2-92b7-00259082215a ONLINE 0 0 0
gptid/21f2788d-efc8-11e2-92b7-00259082215a ONLINE 0 0 0
gptid/2484e4ef-efc8-11e2-92b7-00259082215a ONLINE 0 0 0
replacing-3 OFFLINE 0 0 0
1601580527160658707 OFFLINE 0 0 0 was /dev/da3p3
gptid/b0737f6b-d996-11e5-9b1d-00259082215a ONLINE 0 0 0 (resilvering)
gptid/29b08a29-efc8-11e2-92b7-00259082215a ONLINE 0 0 0
gptid/2c5b024e-efc8-11e2-92b7-00259082215a ONLINE 0 0 0
gptid/cf46c81a-d6ba-11e5-8298-00259082215a ONLINE 0 0 0
gptid/31e93100-efc8-11e2-92b7-00259082215a ONLINE 0 0 0
gptid/329ad7d7-efc8-11e2-92b7-00259082215a ONLINE 0 0 0
gptid/3412b7e6-efc8-11e2-92b7-00259082215a ONLINE 0 0 112 (resilvering)
errors: No known data errors
root@mfsbsd:~ #
scan: resilver in progress since Mon Feb 22 19:14:30 2016
1.13T scanned out of 20.0T at 319M/s, 17h11m to go
110G resilvered, 5.65% done
@dlangille
Copy link
Author

  scan: resilver in progress since Mon Feb 22 19:14:30 2016
        17.5T scanned out of 20.0T at 272M/s, 2h40m to go
        1.67T resilvered, 87.50% done

@dlangille
Copy link
Author

root@mfsbsd:~ # zpool status
  pool: system
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
    attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
    using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://illumos.org/msg/ZFS-8000-9P
  scan: resilvered 1.90T in 23h17m with 0 errors on Tue Feb 23 18:32:00 2016
config:

    NAME                                            STATE     READ WRITE CKSUM
    system                                          ONLINE       0     0     0
      raidz2-0                                      ONLINE       0     0     0
        gptid/1f65b3e4-efc8-11e2-92b7-00259082215a  ONLINE       0     0     0
        gptid/21f2788d-efc8-11e2-92b7-00259082215a  ONLINE       0     0     0
        gptid/2484e4ef-efc8-11e2-92b7-00259082215a  ONLINE       0     0     0
        gptid/b0737f6b-d996-11e5-9b1d-00259082215a  ONLINE       0     0     0
        gptid/29b08a29-efc8-11e2-92b7-00259082215a  ONLINE       0     0     0
        gptid/2c5b024e-efc8-11e2-92b7-00259082215a  ONLINE       0     0     0
        gptid/cf46c81a-d6ba-11e5-8298-00259082215a  ONLINE       0     0     0
        gptid/31e93100-efc8-11e2-92b7-00259082215a  ONLINE       0     0     0
        gptid/329ad7d7-efc8-11e2-92b7-00259082215a  ONLINE       0     0     0
        gptid/3412b7e6-efc8-11e2-92b7-00259082215a  ONLINE       0     0  355K

errors: No known data errors
root@mfsbsd:~ # 

@dlangille
Copy link
Author

Comparing what's on mfsbsd to what's on FreeBSD 10.2 (with freebsd-update applied):

[dan@slocum:~] $ md5 /boot/pmbr /boot/gptzfsboot
MD5 (/boot/pmbr) = cc396177e5d0766ffe6a26de95366860
MD5 (/boot/gptzfsboot) = 699bdb5118877777ee7537eb05f8e58f
[dan@slocum:~] $ 

vs:

root@mfsbsd:~ # md5 /boot/pmbr /boot/gptzfsboot
MD5 (/boot/pmbr) = cc396177e5d0766ffe6a26de95366860
MD5 (/boot/gptzfsboot) = 699bdb5118877777ee7537eb05f8e58f
root@mfsbsd:~ # gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da3
bootcode written to da3
root@mfsbsd:~ # gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da7
bootcode written to da7
root@mfsbsd:~ # 

@dlangille
Copy link
Author

does not boot. See https://twitter.com/DLangille/status/702204608932945920

ZFS: i/o error - all block copies unavailable
ZFS: can't read MOS object directory
ZFS: can't find root filesystem
gptzfsboot: failed to mount default pool system

FreeBBSD/x86 boot
ZFS: i/o error - all block copies unavailable
ZFS: can't find dataset u
Default: system/<0x0>:
boot:
root@mfsbsd:~ # glabel status
                                                  Name  Status  Components
diskid/DISK-%20%20%20%20%20%20%20%20%20%20%20Z2T5TSSAS     N/A  da0
diskid/DISK-%20%20%20%20%20%20%20%20%20%20%20Z2U658MGS     N/A  da1
                                    gpt/boot_Z2T5TSSAS     N/A  da0p1
            gptid/1d13990f-efc8-11e2-92b7-00259082215a     N/A  da0p1
                                    gpt/swap_Z2T5TSSAS     N/A  da0p2
            gptid/1eb9d9c9-efc8-11e2-92b7-00259082215a     N/A  da0p2
                                    gpt/data_Z2T5TSSAS     N/A  da0p3
            gptid/1f65b3e4-efc8-11e2-92b7-00259082215a     N/A  da0p3
diskid/DISK-%20%20%20%20%20%20%20%20%20%20%20Z2U4W1XGS     N/A  da2
diskid/DISK-%20%20%20%20%20%20%20%20%20%20%20Z2U658VGS     N/A  da3
diskid/DISK-%20%20%20%20%20%20%20%20%20%20%20Z2U65DKGS     N/A  da4
diskid/DISK-%20%20%20%20%20%20%20%20%20%20%20Z2T5P7JAS     N/A  da5
diskid/DISK-%20%20%20%20%20%20%20%20%20%20%20Z2U62E0GS     N/A  da6
diskid/DISK-%20%20%20%20%20%20%20%20%20%20%2013Q8PNBYS     N/A  da7
                                    gpt/boot_Z2U658MGS     N/A  da1p1
            gptid/1fbe22bc-efc8-11e2-92b7-00259082215a     N/A  da1p1
                                    gpt/swap_Z2U658MGS     N/A  da1p2
            gptid/2145cde3-efc8-11e2-92b7-00259082215a     N/A  da1p2
                                    gpt/data_Z2U658MGS     N/A  da1p3
            gptid/21f2788d-efc8-11e2-92b7-00259082215a     N/A  da1p3
                          diskid/DISK-AAYZX3EPKWXG9P19     N/A  da8
                                 diskid/DISK-Z2T3BAXAS     N/A  ada0
                                 diskid/DISK-13Q8PMAYS     N/A  ada1
                                    gpt/boot_Z2U4W1XGS     N/A  da2p1
            gptid/225138c7-efc8-11e2-92b7-00259082215a     N/A  da2p1
                                    gpt/swap_Z2U4W1XGS     N/A  da2p2
            gptid/23d8e509-efc8-11e2-92b7-00259082215a     N/A  da2p2
                                    gpt/data_Z2U4W1XGS     N/A  da2p3
            gptid/2484e4ef-efc8-11e2-92b7-00259082215a     N/A  da2p3
            gptid/9f068c0a-d996-11e5-9b1d-00259082215a     N/A  da3p1
            gptid/a7b6f578-d996-11e5-9b1d-00259082215a     N/A  da3p2
            gptid/b0737f6b-d996-11e5-9b1d-00259082215a     N/A  da3p3
                                    gpt/boot_Z2U65DKGS     N/A  da4p1
            gptid/278253b1-efc8-11e2-92b7-00259082215a     N/A  da4p1
                                    gpt/swap_Z2U65DKGS     N/A  da4p2
            gptid/290de883-efc8-11e2-92b7-00259082215a     N/A  da4p2
                                    gpt/data_Z2U65DKGS     N/A  da4p3
            gptid/29b08a29-efc8-11e2-92b7-00259082215a     N/A  da4p3
                                    gpt/boot_Z2T5P7JAS     N/A  da5p1
            gptid/2a092031-efc8-11e2-92b7-00259082215a     N/A  da5p1
                                    gpt/swap_Z2T5P7JAS     N/A  da5p2
            gptid/2bb0b344-efc8-11e2-92b7-00259082215a     N/A  da5p2
                                    gpt/data_Z2T5P7JAS     N/A  da5p3
            gptid/2c5b024e-efc8-11e2-92b7-00259082215a     N/A  da5p3
                                    gpt/boot_Z2U62E0GS     N/A  da6p1
            gptid/2fb43dba-efc8-11e2-92b7-00259082215a     N/A  da6p1
                                    gpt/swap_Z2U62E0GS     N/A  da6p2
            gptid/313fd0a8-efc8-11e2-92b7-00259082215a     N/A  da6p2
                                    gpt/data_Z2U62E0GS     N/A  da6p3
            gptid/31e93100-efc8-11e2-92b7-00259082215a     N/A  da6p3
                                    gpt/boot_13Q8PNBYS     N/A  da7p1
            gptid/33bf85bd-efc8-11e2-92b7-00259082215a     N/A  da7p1
                                    gpt/swap_13Q8PNBYS     N/A  da7p2
            gptid/33dbfcc3-efc8-11e2-92b7-00259082215a     N/A  da7p2
                                    gpt/data_13Q8PNBYS     N/A  da7p3
            gptid/3412b7e6-efc8-11e2-92b7-00259082215a     N/A  da7p3
                                              gpt/boot     N/A  da8p1
            gptid/4f33d1a9-4b11-11e5-9b48-3085a9a4510d     N/A  da8p1
                                            gpt/rootfs     N/A  da8p2
            gptid/4fe4a8ab-4b11-11e5-9b48-3085a9a4510d     N/A  da8p2
            gptid/cf1125bc-d6ba-11e5-8298-00259082215a     N/A  ada0p1
            gptid/cf259555-d6ba-11e5-8298-00259082215a     N/A  ada0p2
            gptid/cf46c81a-d6ba-11e5-8298-00259082215a     N/A  ada0p3
                                gpt/bootcode_13Q8PMAYS     N/A  ada1p1
            gptid/3243a7eb-efc8-11e2-92b7-00259082215a     N/A  ada1p1
                                    gpt/swap_13Q8PMAYS     N/A  ada1p2
            gptid/32642099-efc8-11e2-92b7-00259082215a     N/A  ada1p2
                                    gpt/disk_13Q8PMAYS     N/A  ada1p3
            gptid/329ad7d7-efc8-11e2-92b7-00259082215a     N/A  ada1p3
root@mfsbsd:~ # 

NOTE: da8 is the mfsBSD thumb drive.

@dlangille
Copy link
Author

root@mfsbsd:~ # zpool import -R /mnt system
root@mfsbsd:~ # mkdir /mnt/dev
root@mfsbsd:~ # mount -t zfs system/rootfs /mnt
root@mfsbsd:~ # ls /mnt/dev/
root@mfsbsd:~ # mount -t devfs devfs /mnt/dev
root@mfsbsd:~ # ls /mnt/dev/zfs
/mnt/dev/zfs
root@mfsbsd:~ # cp -R /mnt/boot /mnt/boot.orig
root@mfsbsd:~ # rm -rf /mnt/boot
root@mfsbsd:~ # mv /mnt/boot.orig /mnt/boot
root@mfsbsd:~ # ls /mnt/boot
beastie.4th     brand-fbsd.4th      device.hints~       loader          logo-beastiebw.4th  menusets.4th        version.4th
boot            brand.4th       dtb         loader.4th      logo-fbsdbw.4th     modules         zfs
boot0           cdboot          firmware        loader.conf     logo-orb.4th        pmbr            zfsboot
boot0sio        check-password.4th  frames.4th      loader.conf~        logo-orbbw.4th      pxeboot         zfsloader
boot1           color.4th       gptboot         loader.efi      mbr         screen.4th
boot1.efi       defaults        gptzfsboot      loader.help     menu-commands.4th   shortcuts.4th
boot1.efifat        delay.4th       kernel          loader.rc       menu.4th        support.4th
boot2           device.hints        kernel.old      logo-beastie.4th    menu.rc         userboot.so
root@mfsbsd:~ # 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment