Skip to content

Instantly share code, notes, and snippets.

@dch

dch/resumable.md Secret

Last active November 20, 2019 12:18
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dch/26f2c660fee145c24ffc79126e9cbb21 to your computer and use it in GitHub Desktop.
Save dch/26f2c660fee145c24ffc79126e9cbb21 to your computer and use it in GitHub Desktop.
zfs resumable send over zerotier & mbuffer

send-and-forget

  • use shells/fish
  • pkg install -yr FreeBSD mosh hpenc tmux htop rsync mbuffer
# destination
# hpenc psk
Random key: h7gp...
# set PSK=h7gp...
# mbuffer -4 -s 128k -m 1G -I 9000 \
  | hpenc -d -b 16M -k $PSK  \
  | zfs recv -Fuvs zroot
# source
# zfs snapshot -r zroot@migrate
# zfs send -LevcpDR zroot@migrate \
  | hpenc -b 16M -k $PSK \
  | mbuffer -4 -s 128k -m 1G -O 123.45.67.89:9000

with resumable streams

NB this is a bit messy if you have a failure during a -R as you need to clean up the descendant filesystems manually. Too messy for my liking.

on sink side

while true
    mbuffer -4 -s 128k -m 1G -I 9000 \
        | zfs recv -Fuvs zroot/shared/got 
end

on source side

while true
    sudo zfs send -LevcpDR zroot@snap \
        (ssh -p 2200 root@10.144.157.90 zfs get -Hpo value receive_resume_token zroot/shared/got) \
        | mbuffer -4 -s 128k -m 1G -O 10.144.157.90:9000  
end
@dch
Copy link
Author

dch commented Jun 29, 2018

next time remember to use security/hpenc as well and the new "keep it compressed in transit" flags

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment