FreeBSD provides optional binary compatibility with Linux, allowing users to install and run unmodified Linux binaries. The Linux binaries can be started in the same way native FreeBSD binaries can. They behave almost exactly like native processes and can be traced and debugged the usual way. The "Linuxulator" is available for the i386, amd64, and arm64 architectures. This project aimed to add preliminary Linuxulator support for FreeBSD/powerpc64. Signal handling and vdso are supported, and many system calls for powerpc64 are supported.
A big thanks to my mentors. Without their help, this would have not been possible. They helped me a lot when I got stuck and patiently taught me when I didn't understand something.
- Justin Hibbits
- Dmitry Chagin
Most of the results of my project are contained in this directory, although there are some files that required some modification in the macro and preprocessor directives. https://github.com/Koichi98/freebsd-src/tree/dev/koichi/sys/powerpc/linux
The work I did was basically to initialize the members of struct sysentvec, which is the structure describes an ABI to kernel. This structure has the pointer to the function that set the syscall arguments to the registers, that setup the stack for the user process, etc. I worked to make the ELF-Linux version of this structure.
64-bit PowerPC ELF Application Binary Interface has two versions. ELF v1 is supported in this project since the test code was compiled in ELF v1 in my debian/powerpc64 machine. Specifically, I've mainly implemented linux_exec_setregs(), linux_exec_setregs_funcdesc(), linux_set_syscall_retval(), and linux_fetch_syscall_args(). I did the test by executing the code that just returns which was statically linked. As a side note, copyout(), which copies memory from the kernel to user space used by functions such as linux_copyout_strings(), does not link correctly when built as a module, and this non-trivial bug annoyed me greatly. Building it with the whole kernel solved the problem, and now, as a stopgap measure, it seems to work correctly if I modify sys/kern/kern_cpuset.c as described here. https://reviews.freebsd.org/D36234
I needed to pass the address of the user stack correctly since the dynamically linked binaries use it to get argc, argv, and envp. Those were also initialized in the registers and the problem didn't emerge since the statically linked binaries use those directly. Linux for powerpc64 uses different values or macros for ioctl(2) so I also needed to override some of the macros variables defined in sys/compat/linux/linux.h.
I read the corresponding part of Linux source code to implement these. linux_rt_sendsig(), linux_rt_sigreturn(), and linux_vdso_sigcode() are coded by referring to /arch/powerpc/kernel/signal_64.c and /arch/powerpc/kernel/signal.c mainly. Previously, the signal trampoline code was pushed on the user stack and it calls the signal handler function in the user space whenever the user has registered the specific handling function with sigaction(2). However, it is now implemented as a vdso for security reasons. Therefore, I also needed to refer to the linker script for vdso in Linux (/arch/powerpc/kernel/vdso/vdso64.ld.S) to prepare the linker script for vdso in Linuxulator (sys/powerpc/linux/linux_vdso.lds.s).
I was finally able to run the LTP test of syscalls by supporting the signal handling. The results of the test for each syscalls were shown in this table. https://docs.google.com/spreadsheets/d/1gSGMNTimvkVdrzj2WcpPmMa1SiVm7wrGJAvmIhrvvy8/edit?usp=sharing
I haven't merge the deliverable of my project since I have to make the code clean and neat. Since it is a good stopping point I would like to commit by following the committer's guide. https://docs.freebsd.org/en/articles/committers-guide/#git-primer
As the result of the LTP test shows, there are lots of things to do to support the syscalls completely.
- Fixing values of macros There must be many macros that have different value for powerpc64 as they were for ioctl(2).
- Implementing timer related syscalls I added the support of vdso but it only has signal trampoline code inside. Timer related syscall such as clock_gettime(2) is one of the common use cases of vdso.
- Implementing futex related syscalls Linuxulator on powerpc64 doesn't support futex related functions such as futex_xchgl(). Also, DWARF annotations seem to be needed in signal trampoline for threading related program.
- Implementing ptrace(2) The implementation of ptrace related functions such as linux_ptrace_getregs_machdep() are needed.
As mentioned in the previous section, there are two ELF versions for powerpc64 and it only supports ELFv1 now. I would also like to support ELFv2 which is often used for powerpc64le.