rez: macOS multi-arch packages

README.md

Tiggered from https://academysoftwarefdn.slack.com/archives/C0321B828FM/p1661300813516219 and https://academysoftwarefdn.slack.com/archives/C0321B828FM/p1680211147200339.

Facts

• Pre-existing packages are probably x86_64

• Libraries may be fat, ie there’s a single dylib that may support both x86_64 and arm64 (though not exclusive to that)

• arm64 macs can run both x86_64 and arm64 packages, x86_64 macs can only run x86_64 packages

• An app that’s a specific arch cannot load a binary that’s another arch on any of the platforms (some exceptions, but I don’t care to consider them). So an arm64 app cannot load a x86_64 library adn vise-versa.

• An app that’s a specific arch can load a fat library if it has a matching arch within it.

• It could become a thing on Linux or Windows too. But it's hard to know and predict how it would look like.

• CPU architectures in rez is currently desinged using a package called arch that has versions named x86_64, arm64, etc. Which means rez is not aware of what an architecture is. The arch package is set as an implicit package by default.

Possible solutions

1. Create a arch-universal version and use rezconfig.py to set it as an implicit if rez is run on an M1 CPU. A big downside is that all packages would now need to be re-released and rebuilt as universals... Which means it's all "universal" or nothing. In practice, a package could say it's universal but could be either x86_64 or arm64...

2. So perhaps it makes sense to define arch packages as versions-of-that-arch but not exclusively to every other arch. arch_arm64-1 and arch_x86_64-1, suddenly solve together, and it's really up to you to define if that's something you want or not. Of course, that's not exactly the kind of change you could anticipate enough in advance to not have a huge package ecosystem that would be virtually impossible to relabel/build/variant/key, so I don't know that the suggestion practically helps all that much. (https://academysoftwarefdn.slack.com/archives/C0321B828FM/p1661442932166329?thread_ts=1661300813.516219&cid=C0321B828FM)

3. Something like this:

   • apple_arch-x86_64 has two variants: arch-x86_64 and arch-arm64 (the two arches it can run on)
   • apple_arch-arm64 has a dependency on arch-arm64
   • apple_universal-2 has a variant each for the two arches above (apple_arch-x86_64 and apple_arch-arm64).
   • If a package only supports one arch, we add one of these to the requires (apple_arch-x86_64 or apple_arch-arm64)
   • If it supports both, then we add apple_universal-2 and then let Rez just pick the underlying arch accordingly

   Then we’re going to patch our rez-env to look to see which apple_arch got resolved, and then prefix arch -$REZ_APPLE_ARCH_MAJOR_VERSION to the shell that gets launched by Rez.

   That way the shell is in the right arch mode for the packages within it, and anything launched within it will default to that arch.

   We still need to try this out but I think it should do the trick. The one big downside is having to go back and patch up all our released package.py files with the right arch, but this should in theory avoid.

   However other than that, having these arch packages let us version them independently in case we need to add more logic to the system

   https://academysoftwarefdn.slack.com/archives/C0321B828FM/p1661909570171429?thread_ts=1661300813.516219&cid=C0321B828FM

4. Native solution built-in to rez. Rez would know what a CPU architecture is.