Skip to content

Instantly share code, notes, and snippets.

@csgordon
Last active August 9, 2024 13:38
Show Gist options
  • Save csgordon/74658096f7838382b40bd64e11f6983e to your computer and use it in GitHub Desktop.
Save csgordon/74658096f7838382b40bd64e11f6983e to your computer and use it in GitHub Desktop.

Installing OpenBSD 7.3-current on a VisionFive2

Here are some concise instructions for getting OpenBSD 7.3-current running on a StarFive VisionFive 2 (v1.3B, though from other reports referenced below, it sounds like the v1.2 boards can also be made to work).

You will need:

  • An SD card of at least 1GB
  • An eMMC module attached to the board (though see below for a variation that should work to install directly to sd card)
  • Internet access
  • A USB TTL serial adapter that supports 3.3V (some of these support multiple voltages, but if yours only supports other voltages and not 3.3 it will either not work or damage your board!)
  • A second machine to interact with the VF2 over serial

Before you start

  • Verify that the board boots StarFive's Debian or Fedora fork. For 1.2 boards you might need to update firmware, for my 1.3 board the June 2023 Debian image worked perfectly when the board was set to boot from SD.
  • Verify that the eMMC is inserted correctly. I, and others on the StarFive boards, have had trouble getting this seated correctly. If you can see it from Linux you're good to go. If you jumped ahead and nuked the sd card with your Linux image already, you can try to verify this from the u-boot prompt (see below to get there) with mmc dev 0, and if it's actually formatted you could try ls mmc 0:1 / depending on the filesystem.

Personally verified instructions

  1. Burn install73.img from -current snapshots for riscv64 to an sd card
  2. Mount the EFI partition and drop the dtb from this mailing list post in there, in the root of the partition: https://marc.info/?l=openbsd-misc&m=169046816826966&w=2. This is a file specific to board version 1.3B, though it sounds like it might have been used successfully on a v1.2 board. YMMV, I only have a 1.3B.
  3. Eject the SD card from where you wrote it, stick it in the VF2
  4. Attach the serial adapter to the appropriate GPIO pins on the VF2, as on page 35 of the quick start guide: https://doc-en.rvspace.org/VisionFive2/PDF/VisionFive2_QSG.pdf (see Figure 4-1)
  5. Attach the USB end to whatever machine you're using.
  6. Kick off a serial connection from the other machine. You might have to dig around for examples, but this step is the same as for Raspberry Pis, which makes instructions easier to come by.
  • On FreeBSD, I run sudo screen /dev/cuaU0 115200
  • On MacOS, I run sudo screen /dev/cu.URT0 115200
  1. Boot the board (v1.3b) to the u-boot prompt from the internal u-boot (switches to 0,0), hit a key (in the terminal connection on the other computer!) to interrupt (in theory you could wait for it to time out), and run the commands from that post:
load mmc 1:1 ${fdt_addr_r} jh7110-starfive-visionfive-2-v1.3b.dtb
load mmc 1:1 ${kernel_addr_r} efi/boot/bootriscv64.efi
bootefi ${kernel_addr_r} ${fdt_addr_r}
  • If either of the first two commands fail, check that the card is inserted correctly, and that you inserted before powering on the VF2. (Technically you can re-probe the sd card with mmc rescan and mmc device 1, but this is sometimes flaky for me.)
  1. Follow the install process as you typically would! The eMMC device should show up as sd0, the sd card is sd1. Choose the eMMC as the installation target, and when it gets to installing sets, choose disk, and follow the steps to mount sd1.
  2. Reboot, this time using 0:1 (first partition of mmc device 0, the eMMC) in the second command that loads the (but not the first, since the dtb file is still only on the sd card!)
  3. Log in, mount the EFI partition on your eMMC, and the sd card, and copy the same dtb file from the EFI partition on the sd card to the EFI partition on the eMMC
  • Unless you did some serious customization to your install, this should be the only FAT partition that shows up when you ask disklabel to show the eMMC's partitions.
  • Technically you should be able to mount the eMMC card's EFI partition and copy this over at the end of your install process if you want to, but I did not think to do this
  1. You can now boot without the sd card, using 0:1 in both commands. The adventurous could set this as the default boot sequence in u-boot.

Speculative: Installing to SD Card or NVMe

The instructions above assume you have an eMMC module installed, as I did. But in theory installing to an SD card or NVMe is possible.

Installing to SD Card

If you only have an SD card, there's a way to boot the install process off the same media you're going to install to, as long as you have internet access during the install.

Grab miniroot73.img instead of install73. Boot as instructed, but when you get to installing sets you'll need an internet connection. Type ! to temporarily escape to a shell very early in the process, before configuring network, and use the date command to set the date, or SSL won't work. Then exit the shell back to the install program, and carry on, selecting the sd card as the destination. You can then choose http as the source for disk sets. This will, however, overwrite your install media, so if something goes wrong be prepared to re-image miniroot73.img.

I did try this once but was having difficulty when specifying cdn.openbsd.org as the server (verifications were failing, the connection seemed flaky). I assumed at the time that the issue was my home internet (which is sometimes flaky) or the wifi dongle, but I've since discovered that this sometimes happens with the cdn server (https://dev.to/nabbisen/openbsd-pkgadd-didnt-work-due-to-ocsp-verify-failed-53b2). Instead, use ftp.openbsd.org, which is what I've since used to follow this approach on a Raspberry Pi 4B.

I haven't tried it, but have seen reports (LINK) that the wired ethernet works in at least one port, in which case you can use that. Or, if you have a wifi dongle compatible with OpenBSD, it should be recognized and usable during installation (I did configure my realtek dongle during one boot from the install media).

Another option, if you have a USB CD drive lying around: yes, that's right, you could download the sets (for the right architecture!) and burn them to a CD. I happen to have one plugged into the USB hub I used with my VF2, and it's detected during setup.

Installing to NVMe

I don't have an NVMe, but it has been done using edk2: https://marc.info/?l=openbsd-misc&m=169048799607277&w=2

Note, however, that if you stick a high-power NVMe drive in the VF2, it may draw too much power to be stable: https://forum.rvspace.org/t/unlocking-new-possibilities-starfive-visionfive-2-sbc-now-supports-tianocore-edk-ii-uefi/2779/42

Speculative: Cleaning up the boot sequence

Manually typing the load commands into u-boot every time is not pleasant. In principle, it should be possible to extend the onboard u-boot's settings with an option to try those manually (in addition to the things it tries by default).

Speculative: Getting a real TTY

The OpenBSD ARM install file notes that the OpenBSD boot loader --- which is what runs immediately after the bootefi command, and briefly gives you a prompt before booting the kernel --- can be directed to use a framebuffer for the serial device by typing set tty fb0 before the prompt times out. Presumably there's a way to set that permanently (on FreeBSD it would be a setting in /etc/boot.conf) but I haven't figured that out yet, or tried manually. I can see from dmesg output that my USB keyboard is detected.

What Works, What Doesn't

All critically depending on this one DTB...

Working

  • Storage on SD card and eMMC
  • PCIe and USB
    • Notably, USB peripherals that are supported on other platforms work, such as a Realtek wifi dongle using the urtwn driver

Notable not working / unimplemented

Mostly graphics support

Untested / Unexamined

Framebuffer support, GPIO pins (beyond serial), various extra board connectors (CSI, DSI), cpu temp, cpu frequency control, wired ethernet (though this is reported to work)

What's Happening in Step 8

I'll fill this in later with more detail, but basically we're telling u-boot to load the specific DTB from the SD card, and pass its contents to the OpenBSD loader, so the loader can in turn pass a reasonably-accurate device tree to the kernel. Without this, the kernel won't be able to locate various onboard devices and/or won't know the right settings to communicate with the devices.

Resources I used to get here

I originally set out to figure out if I could run FreeBSD on this device, not OpenBSD, because I am actually an active FreeBSD user. That search led me to a forum post:

Which in turn linked to:

  • A toolkit for booting the FreeBSD kernel plus a ramdisk, but without any driver support (even storage): https://github.com/robn/freebsd-vf2
  • A number of links to OpenBSD source changes for the VF2 (more precisely, the JH7110), which got me interested in seeing how OpenBSD worked on the board

After much, much more digging, I eventually found a very useful Reddit thread:

Which linked to other interesting resources, but critically to the mailing list thread where someone had posted an actual DTB used for some of the OpenBSD support work:

Earlier I also found the record of someone trying to walk the same path as me before, of getting FreeBSD up and running on this to do some driver development: https://adventurist.me/posts/00315. Since that post, the documentation situation has gotten somewhat (slightly) better; not only did the OpenBSD developers find some useful info somewhere, it turns out the same chipset is used in the Pine64 Star64, and they have found much more informative datasheets than anything I could find on StarFive's wiki: https://wiki.pine64.org/wiki/STAR64#Datasheets_for_Components_and_Peripherals.

Now that I've confirmed the OpenBSD drivers work, I'm going to (a) use OpenBSD a bit(!), and (b) try porting some drivers to FreeBSD, since compared to the last person to try this, I have the benefit of working OpenBSD drivers to go off of, and seemingly slightly better technical documentation (not sure how much better). I've already got a full cross-build of world and kernel for riscv64 from an amd64 machine (I'll write up those directions at some point, it took a bit of digging and cross-referencing), so we'll see how that goes. Some of the OpenBSD driver changes are actually minor adjustments to existing drivers to just recognize additional devices, but I'm not yet sure of the dependency order among drivers. I just hope the storage isn't connected via PCIe, I don't want that to be the first driver I have to port! If I actually get somewhere, maybe I'll pick up a Star64 too...

@mengzhuo
Copy link

mengzhuo commented Nov 11, 2023

Thanks you for this detail instruction!
I've available boot and install OpenBSD 7.4 on MilkV Mars (StarFive CPU) !

I have one question, how to set perf to 100%?

mars-1# sysctl hw.setperf=100
sysctl: hw.setperf: Operation not permitted

updates using sysctl hw.perfpolicy=high works!

@csgordon
Copy link
Author

csgordon commented Nov 11, 2023 via email

@christgau
Copy link

christgau commented Jan 5, 2024

@csgordon Thanks for this guide. Based on it, I've been able to install OpenBSD 7.4 on a VisionFive2 rev1.3 with an NVMe drive. The drive I installed is an Samsung SSD 980 PRO 500GB. Here's what I've done in addition to your guide using stuff from somewhere else. Maybe someone finds it beneficial to be bundled altogether:

  1. Updated u-boot and SPL with the blobs provided by VisionFive2 software v5.10.3. Don't know if is required, but I just upgraded to avoid issues. Update was flawless following the official board's quick start guide.
  2. Followed the guide above and installed OpenBSD to the NVMe.
  3. Followed the instructions from here to manually boot into the fresh installation using the DTB on the SD card and the kernel from the fresh NVMe installation:
    • Manually enter NVMe installation
      load mmc 1:1 ${fdt_addr_r} jh7110-starfive-visionfive-2-v1.3b.dtb
      load nvme 0:1 ${kernel_addr_r} efi/boot/bootriscv64.efi
      bootefi ${kernel_addr_r} ${fdt_addr_r}
      
    • Copy the DTB from the SD into a new directory vendor on tne NVMe's EFI partition (/dev/sd0i in my case)
    • Then I am able to manually boot the NVMe installation just as stated here (essentially, just telling u-boot to load everything from NVMe now):
      load nvme 0:1 ${fdt_addr_r} vendor/jh7110-starfive-visionfive-2-v1.3b.dtb
      load nvme 0:1 ${kernel_addr_r} efi/boot/bootriscv64.efi
      bootefi ${kernel_addr_r} ${fdt_addr_r}
      
    • This can then be persisted to uboot
      env default -a -f
      setenv distro_boot_env "echo Booting OpenBSD from NVME; {command from above separated by ';'}
      saveenv
      
  4. Say hello to OpenBSD on the VisionFive:
    $ uname -a
    OpenBSD fv2.localdomain 7.4 GENERIC.MP#427 riscv64
    $ fv2# dmesg | grep sd0
    sd0 at scsibus0 targ 1 lun 0: <NVMe, Samsung SSD 980, 5B2Q>
    [...]
    root on sd0a (7faa6d333279bc97.a) swap on sd0b dump on sd0b
    

@csgordon
Copy link
Author

csgordon commented Jan 5, 2024 via email

@nbosley
Copy link

nbosley commented Jan 20, 2024

This may vary based on the firmware revision, but "distro_boot_env" did absolutely nothing for me.
If you have trouble autobooting OpenBSD because the boot process isn't loading the dtb file, check your 'printenv' output, especially this line:
efi_dtb_prefixes=/ /dtb/ /dtb/current/
and this one:
fdtfile=starfive/starfive_visionfive2.dtb
Based on those vars, I moved "jh7110-starfive-visionfive-2-v1.3b.dtb" to "starfive/starfive_visionfive2.dtb" in my EFI filesystem, and my vf2 is now able to boot OpenBSD from my eMMC automagically now after power-on.
Caveat: for whatever reason, it still wants to check eMMC 0:3 first, but once that times out, it starts scanning other partitions, finds mmc 0:1, and loads the dtb and efi binary, and boots correctly with them.

UPDATE:
My issue was just firmware-related.
My vf2 arrived with the firmware from the v2.11.5 release.
I upgraded to v5.10.3 (using tftpd from FreeBSD's inetd, which worked fine).
Even after that, autoboot didn't work at first.
Turns out the 'env default -f -a' and 'reset' wasn't good enough for me.
I had to do an 'env erase' and then a 'reset'.
Only after that did 'printenv' show the newer vars I was expecting.
The update also allows me to boot NVMe now, which works fine so far.
You can save yourself a few minutes, by just choosing the shell option after the OS installation, then mount the sd card's EFI FS and the NVMe's EFI FS, and copy the dtb over right then.

And here were are:

vf2# uname -a
OpenBSD vf2.censored 7.4 GENERIC.MP#463 riscv64
vf2# sysctl | grep disk
hw.disknames=sd0:1c17d8548a33200a,sd1:21bc47b88aa96f39,sd2:cb41d0135d7e009a
hw.diskcount=3
vf2# 
vf2# df
Filesystem  512-blocks      Used     Avail Capacity  Mounted on
/dev/sd0a      2018844    118880   1799024     7%    /
/dev/sd0l     10313144        72   9797416     1%    /home
/dev/sd0d      8114940        12   7709184     1%    /tmp
/dev/sd0f     60934744   2145528  55742480     4%    /usr
/dev/sd0h     40614428       288  38583420     1%    /usr/local
/dev/sd0k     12179004         4  11570052     1%    /usr/obj
/dev/sd0j      6082908         4   5778760     1%    /usr/src
/dev/sd0e     24371196      6940  23145700     1%    /var
vf2# dmesg | grep sd0
sd0 at scsibus0 targ 1 lun 0: <NVMe, CT1000P3SSD8, P9CR>
sd0: 953869MB, 512 bytes/sector, 1953525168 sectors
root on sd0a (1c17d8548a33200a.a) swap on sd0b dump on sd0b
vf2#

Fun!

Didn't try the ethernet ports yet from within the OS, but I can see at least that the dwqe driver bound to both ports.

@gyptazy
Copy link

gyptazy commented Jan 31, 2024

Didn't try the ethernet ports yet from within the OS, but I can see at least that the dwqe driver bound to both ports.

This sounds promising! Need to give it a try on the weekend

@christgau
Copy link

Ethernet just works out of the box.

@gyptazy
Copy link

gyptazy commented Feb 1, 2024

@christgau Thanks for the info! Awesome!

@barracuda156
Copy link

@csgordon Do you know if this gonna work for PineTab-V? It seems to use the same or similar hardware.

@csgordon
Copy link
Author

csgordon commented Aug 9, 2024 via email

@barracuda156
Copy link

Note that there's no graphics support.

@csgordon Got it, thanks. Not of much use for me in this case; I would just preferred to use some BSD version over Linux, but I need some sensible GUI working (Linux got KDE Plasma BTW).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment