Skip to content

Instantly share code, notes, and snippets.

@mohakshah
Last active July 15, 2023 04:58
Show Gist options
  • Save mohakshah/b203d33a235307c40065bdc43e287547 to your computer and use it in GitHub Desktop.
Save mohakshah/b203d33a235307c40065bdc43e287547 to your computer and use it in GitHub Desktop.
Building ZFS on Raspberry Pi 3 running Rasbpian

Introduction

This is a tutorial for building and installing the latest release version (0.7.3 as of writing) of "ZFS on Linux" on a Raspberry Pi 3 running Raspbian Stretch. Specifically, we'll be building the dkms version of ZoL, which saves you the hassle of re-compiling the kernel modules after every kernel update. Even though ZoL added support for building dkms packages for debian in version 0.7.3, the build process on a Raspberry Pi 3 is not quite straight-forward. Hopefully, these instructions will make it easier.

Steps

  1. Install the build dependencies.
$ sudo apt-get update
$ sudo apt-get install build-essential autoconf libtool gawk alien fakeroot
$ sudo apt-get install dkms zlib1g-dev uuid-dev libattr1-dev libblkid-dev libselinux-dev libudev-dev libssl-dev parted lsscsi wget ksh

You'll also need the kernel headers. Running apt-get upgrade will take care of both installation and upgradation.

$ sudo apt-get upgrade raspberrypi-kernel raspberrypi-kernel-headers
  1. Download the source.

Download the latest release tarballs for zfs and spl from this page and extract them.

$ wget https://github.com/zfsonlinux/zfs/releases/download/zfs-0.7.3/spl-0.7.3.tar.gz && tar -xzf spl-0.7.3.tar.gz
$ wget https://github.com/zfsonlinux/zfs/releases/download/zfs-0.7.3/zfs-0.7.3.tar.gz && tar -xzf zfs-0.7.3.tar.gz
  1. Compile and install spl.
$ cd spl-0.7.3/
$ autoreconf --install --force
$ ./configure

You need to make a small edit to the Makefile generated by configure before you proceed.

$ sed -E 's/(^RPMBUILD = rpmbuild.*)/\1 --target=armhf/' -i Makefile
$ make pkg-utils deb-dkms
$ for deb in *.deb; do sudo dpkg -i "$deb"; done

The final step will take some time as dkms builds spl modules for each kernel version installed in your system.

  1. Compile and install zfs.

The build process for zfs is the same as spl, albeit with an extra option --with-config=srpm in ./configure

$ cd ../zfs-0.7.3/
$ autoreconf --install --force
$ ./configure --with-config=srpm
$ sed -E 's/(^RPMBUILD = rpmbuild.*)/\1 --target=armhf/' -i Makefile
$ make pkg-utils deb-dkms
$ for deb in *.deb; do sudo dpkg -i "$deb"; done

The zfs source is much bigger than spl's and hence the build time will also be much longer.

  1. Reboot and Enjoy. :)

Sources

@slyphon
Copy link

slyphon commented Apr 13, 2018

hey, thank you so much for this. small suggestion, might want to add "dkms" to the list of apt-get install packages. My raspbian install didn't have it by default :)

@bueli
Copy link

bueli commented May 9, 2018

+1 for adding the "dkms" package

@beren12
Copy link

beren12 commented May 13, 2018

I made a tutorial a little while ago to use the debian packages should be every step needed, included tuning for low men usage. Make sure you don't use swap on a normal zpool you need to use a partition or a special zvol.

@mohakshah
Copy link
Author

I have added dkms to the set of dependencies.

@mohakshah
Copy link
Author

I should also note that since last few 0.7.x versions of ZoL, while building on this 64-bit debian distribution for pi 3, I haven't had the need to edit the Makefile. Haven't checked on official Raspbian distro yet.

@bobk48
Copy link

bobk48 commented Nov 7, 2018

My Raspbian on a Raspberry Pi 3, after using wget ZFS 0.7.11, with the spl for 0.7.11, after reboot reports that the ZFS module is not loaded, and mod probe can’t find it. I followed the above instructions and they all executed properly, but no ZFS on the Raspberry pi.

@drdebian
Copy link

drdebian commented Nov 12, 2018

I installed using this guide, but with 0.7.11... When trying to use any of the utilities I get this

root@rp01:/home/pi# zpool status
The ZFS modules are not loaded.
Try running '/sbin/modprobe zfs' as root to load them.

Running modprobe gives me this:

root@rp01:/home/pi# /sbin/modprobe zfs
modprobe: FATAL: Module zfs not found in directory /lib/modules/4.14.71-v7+

What am I missing? Any hints greatly appreciated!

[edit]
Looks like the kernel modules for 4.14.71-v7+ weren't built... :(

root@rp01:/home/pi# dpkg-reconfigure zfs-dkms
Removing old zfs-0.7.11 DKMS files...

------------------------------
Deleting module version: 0.7.11
completely from the DKMS tree.
------------------------------
Done.
Loading new zfs-0.7.11 DKMS files...
It is likely that 4.14.71-v7+ belongs to a chroot's host
Building for 3.6-trunk-rpi, 4.14.71+ and 4.14.71-v7+
Building initial module for 3.6-trunk-rpi
configure: error: in `/var/lib/dkms/zfs/0.7.11/build':
configure: error: SHUT 'ER DOWN CLANCY, SHE'S PUMPIN' MUD!
See `config.log' for more details
Error! Bad return status for module build on kernel: 3.6-trunk-rpi (armv7l)
Consult /var/lib/dkms/zfs/0.7.11/build/make.log for more information.

[edit #2]
Managed to compile the kernel module after getting rid of the old 3.6-trunk-rpi! Hooray!

@Bbertatum
Copy link

Bbertatum commented Feb 10, 2019

thx for the guide, lil sidenote, need some other packages to compile (version 0.8) :
apt install python3-dev python3-cffi python3-setuptools

@jerlich
Copy link

jerlich commented May 27, 2019

how is the performance?

@RobertSkawinski
Copy link

Can you provide an image please? - That would be great! :)

@kapcom01
Copy link

kapcom01 commented Jun 12, 2019

Thanks for this guide.
I am on Raspbian GNU/Linux 9.9 (stretch).
I am sharing my experience to help others save time. Here is what worked for me and what not:

  • [FAILED] installation from stable repository
  • [FAILED] installation from testing repository (dkms builts but zfsutils is missing from the repo)
  • [FAILED] this guide for zfs 0.7.3
  • [FAILED] this guide for zfs 0.8.0
  • [WORKED] this guide for zfs 0.7.13

NOTE: before you begin with this guide you have to do sudo apt remove 3.6-trunk-rpi (thanks to @drdebian)

@stefanhinker
Copy link

I'm having issues with the "make pkg-utils" part of this guide when running on raspbian 4.19.65-v7l+.
I've tried it with various versions of zfs, but it's always the same.
The error is:
`make install-data-hook
make[6]: Entering directory '/tmp/zfs-build-pi-MEDFVMSR/BUILD/zfs-0.8.1'
make[6]: Nothing to be done for 'install-data-hook'.
make[6]: Leaving directory '/tmp/zfs-build-pi-MEDFVMSR/BUILD/zfs-0.8.1'
make[5]: Leaving directory '/tmp/zfs-build-pi-MEDFVMSR/BUILD/zfs-0.8.1'
make[4]: Leaving directory '/tmp/zfs-build-pi-MEDFVMSR/BUILD/zfs-0.8.1'
make[3]: Leaving directory '/tmp/zfs-build-pi-MEDFVMSR/BUILD/zfs-0.8.1'
make[2]: Leaving directory '/tmp/zfs-build-pi-MEDFVMSR/BUILD/zfs-0.8.1'

  • find /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/lib/arm-linux-gnueabihf -name *.la -exec rm -f {} ;
  • exit 0
    Processing files: zfs-0.8.1-1.armv7hnl
    error: File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/sbin/*
    error: File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/usr/share/man/man1/*
    error: File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/usr/share/man/man5/*
    error: File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/usr/share/man/man8/*
    error: File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/lib/udev/rules.d/*
    error: File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/etc/init.d/*
    error: File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/etc/sudoers.d/*

RPM build errors:
File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/sbin/*
File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/usr/share/man/man1/*
File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/usr/share/man/man5/*
File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/usr/share/man/man8/*
File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/lib/udev/rules.d/*
File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/etc/init.d/*
File not found: /tmp/zfs-build-pi-MEDFVMSR/BUILDROOT/zfs-0.8.1-1.%{_arch}/etc/sudoers.d/*
make[1]: *** [Makefile:1234: rpm-common] Error 1
make[1]: Leaving directory '/home/pi/zfs/zfs-0.8.1'
make: *** [Makefile:1193: rpm-utils] Error 2
`
When I try the same on an x86 box, the path turns out to be
/tmp/zfs-build-ubuntu-89OPc67R/BUILDROOT/zfs-0.8.1-1.x86_64

It looks as if the system architecture isn't recognized. But I'm not familiar enough with make and package building to figure this out on my own. Any help appreciated.

@colettace
Copy link

Having the same problem as @stefanhinker ... clearly the %{_arch} substitution is not being made in the execution, but this may be a red herring in that the fles that are "not-found" are in fact there:

pi@raspberrypi:~ $ file /tmp/spl-build-pi-ySnPem5U/BUILDROOT/spl-0.7.13-1.%{_arch}/usr/bin
/tmp/spl-build-pi-ySnPem5U/BUILDROOT/spl-0.7.13-1.%{_arch}/usr/bin: directory
pi@raspberrypi:~ $ ls /tmp/spl-build-pi-ySnPem5U/BUILDROOT/spl-0.7.13-1.%{_arch}/usr/bin
splslab.py
pi@raspberrypi:~ $ file /tmp/spl-build-pi-ySnPem5U/BUILDROOT/spl-0.7.13-1.%{_arch}/usr/share/man/man5/
/tmp/spl-build-pi-ySnPem5U/BUILDROOT/spl-0.7.13-1.%{_arch}/usr/share/man/man5/: directory
pi@raspberrypi:~ $ ls /tmp/spl-build-pi-ySnPem5U/BUILDROOT/spl-0.7.13-1.%{_arch}/usr/share/man/man5/
spl-module-parameters.5

@Alexey-Tsarev
Copy link

Alexey-Tsarev commented Oct 2, 2019

zfs-0.8.2 failed to build on
Linux laser-gw 4.19.66-v7+ #1253 SMP Thu Aug 15 11:49:46 BST 2019 armv7l GNU/Linux

  CC       strlcpy.lo
  CC       timestamp.lo
  CC       zone.lo
  CPPAS    atomic.lo
  CCLD     libspl.la
ar: `u' modifier ignored since `D' is the default (see `U')
Making all in libtpool
  CC       thread_pool.lo
  CCLD     libtpool.la
ar: `u' modifier ignored since `D' is the default (see `U')
Making all in libzutil
  CC       zutil_device_path.lo
  CC       zutil_import.lo
  CC       zutil_nicenum.lo
  CC       zutil_pool.lo
  CCLD     libzutil.la
ar: `u' modifier ignored since `D' is the default (see `U')
Making all in libunicode
  CC       u8_textprep.lo
  CC       uconv.lo
  CCLD     libunicode.la
ar: `u' modifier ignored since `D' is the default (see `U')
Making all in libuutil
  CC       uu_avl.lo
  CC       uu_alloc.lo
  CC       uu_dprintf.lo
  CC       uu_ident.lo
  CC       uu_misc.lo
  CC       uu_list.lo
  CC       uu_open.lo
  CC       uu_pname.lo
  CC       uu_string.lo
  CCLD     libuutil.la
ar: `u' modifier ignored since `D' is the default (see `U')
Making all in libnvpair
  CC       libnvpair.lo
  CC       libnvpair_json.lo
  CC       nvpair_alloc_system.lo
  CC       nvpair_alloc_fixed.lo
  CC       nvpair.lo
  CC       fnvpair.lo
  CCLD     libnvpair.la
ar: `u' modifier ignored since `D' is the default (see `U')
Making all in libzpool
  CC       kernel.lo
  CC       taskq.lo
make[3]: *** No rule to make target '../../module/zfs/zfeature_common.c', needed by 'zfeature_common.lo'.  Stop.
make[3]: *** Waiting for unfinished jobs....
  CC       util.lo
Makefile:653: recipe for target 'all-recursive' failed
make[2]: *** [all-recursive] Error 1
Makefile:840: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
Makefile:709: recipe for target 'all' failed
make: *** [all] Error 2

pi@laser-gw:~/dev/zfs$ uname -a
Linux laser-gw 4.19.66-v7+ #1253 SMP Thu Aug 15 11:49:46 BST 2019 armv7l GNU/Linux

At the same time, the "zfs-0.7.13" built successfully:

make[2]: Leaving directory '/usr/src/linux-headers-4.19.66-v7+'
kmoddir=/lib/modules/4.19.66-v7+; \
if [ -n "" ]; then \
        find $kmoddir -name 'modules.*' | xargs rm -f; \
fi
sysmap=/boot/System.map-4.19.66-v7+; \
if [ -f $sysmap ]; then \
        depmod -ae -F $sysmap 4.19.66-v7+; \
fi
make[1]: Leaving directory '/home/pi/dev/zfs/module'
make[1]: Entering directory '/home/pi/dev/zfs'
make[2]: Entering directory '/home/pi/dev/zfs'
make[2]: Nothing to be done for 'install-exec-am'.
 /bin/mkdir -p '/usr/local/src/zfs-0.7.13'
 /usr/bin/install -c -m 644 zfs.release.in zfs_config.h.in '/usr/local/src/zfs-0.7.13'
 /bin/mkdir -p '/usr/local/src/zfs-0.7.13/4.19.66-v7+'
 /usr/bin/install -c -m 644 zfs.release zfs_config.h module/Module.symvers '/usr/local/src/zfs-0.7.13/4.19.66-v7+'
make[2]: Leaving directory '/home/pi/dev/zfs'
make[1]: Leaving directory '/home/pi/dev/zfs'
+ sudo ldconfig
+ sudo depmod -a
+ sudo modprobe zfs
+ sudo zpool import -a

If someone has a solution, pls share!

@Alexey-Tsarev
Copy link

Alexey-Tsarev commented Oct 8, 2019

@drdebian
Copy link

drdebian commented Oct 9, 2019

@eudocimus
Copy link

eudocimus commented Nov 8, 2019

@MAFLO321
Copy link

MAFLO321 commented Dec 15, 2019

@stefanhinker @colettace
The problem can be fixed with the following to build zfs 0.8.3 on a Raspberry Pi 4:

./autogen.sh
./configure --disable-pyzfs
# https://github.com/zfsonlinux/zfs/issues/9504
sed -E 's/(^RPMBUILD = rpmbuild.*)/\1 --target=armv7hnl-linux-gnueabihf/' -i Makefile

make deb-dkms deb-utils

see openzfs/zfs#9504 for more information

@peterska
Copy link

There is no need to use sed to modify the Makefile. rpmbuild errrors are caused by the rpmbuild package on raspbian not storing the architecture files in an appropriately named directory. You can fix this permanently by running the following:

cd /usr/lib/rpm/platform; sudo ln -s armv7hnl-linux-gnueabihf armv7hnl-linux

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment