Disclaimer: I don't have a deep understanding of this topic, so take everything I have written up here with a grain of salt. This also is all from the perspective of 8 bit SDR video. HDR video is probably a whole other can of worms.
This describes my observations about Converting from RGB to video, attempting to match colors when encoding using libx264.