I'd like to use a simple Pi Zero which has but one USB interface. That's why my preference is connecting a RS485 transceiver to the GPIO header. Here is a useful picture from stackexcange (that also indicates that it's advisable to use a 3.3V compatible transceiver):
Kontron provides a good introduction to configuring the Linux kernel for using RS485 on the Raspberri Pi, especially regarding disabling kernel console output in the kernel configuration level. The docs link to the official RasPi UART configuration docs, which refers to the BCM2835 peripherals documentation.
On most RasPis only UART0 can be used which is of the PL011 type (basically a 16550). There is an internal TX BUSY state but there is no way to make it control a GPIO in hardware. The directional GPIO has thus to be controlled in "user land".
This also means that any program that uses RS485 will have to manage the GPIO configuration. I'm certain that there are good home automation solutions out there but learning to do the configuration and selecting features might be more effort than creating a simple solution.
The Kontron page already contains a recipe for using RS485 using PySerial. The Python minimalmodbus package provides access to Modbus-RTU servers. The minimalmodbus docs indicate that the author was aware of controlling the RS485 communication direction with assistance from the kernel. The details for the Raspberry Pi will have to be worked out.
An alternative is using Liblightmodbus, a C99 library with a concept similar to stm8ef-modbus (it can even be used for AVR µCs).