- Proposal: SE-NNNN
- Authors: Zac Bowling (Google)
- Review Manager: TBD
- Status: Awaiting implementation
During the review process, add the following fields as needed:
- Implementation: apple/swift#NNNNN
- Decision Notes: Rationale, Additional Commentary
- Bugs: SR-NNNN, SR-MMMM
- Previous Revision: 1
- Previous Proposal: SE-XXXX
There are number of inconsistencies and issues with how you access libc across each platform within Swift.
libc is currently accessible either by the native Darwin module on Darwin
platforms or the Glibc module for all other platforms. These modules provide
nearly the same API across each platform but are exposed under different names.
This situation thus requires you to gate importing each module on some type of
platform check which. More often #if os(Linux) is used which not inclusive of
all possible platforms the code will likely run on. This check is also
insufficient for checking the specific libc you are targeting has the features
you may want at compile time.
The Glibc swift module is currently compiled against the libc on the machine
that built the toolchain which may not necessarily match the libc on the machine
where the toolchain runs or the the target sysroot/SDK you point your swiftc at
compile time. In contrast, the Darwin module is always up-to-date with the
target SDK since the Darwin modulemap is included inside the SDK folder.
A side effect of the Glibc module being compiled ahead of time is that it bakes
in absolute paths of the system to the system headers which breaks when
cross-compiling against relocatable SDK paths.
In some libc implementations, some features provided are hidden behind feature macros that have to be defined before before you include the libc headers when you use them. Some of these macros can even change function signatures and behaviors of libc functions. (See feature_test_macros(7) on Glibc).
The Glibc module name is also unfortunately named for users on FreeBSD,
Android, Haiku, and Fuchsia where the libc is not specifically GNU's glibc but
some other libc. The module name is also incorrect on Linux when using another
libc like musl, dietlibc, or uClibc instead of GNU's libc.
Swift-evolution thread: Discussion thread topic for that proposal
We want to smooth over some of these inconsistencies that make Swift code less
portable when consuming libc in Swift. We want to leave some backward
compatibility with code that makes use of the Darwin and Glibc modules today
but encourage developers to migrate code to the solution proposed here.
We create a PlatformLibC module that replaces the Darwin and Glibc. This
module would simply re-export the symbols from both of these modules. Developers
should be able to include this module to get all portable libc methods they
need without platform checks.
Instead of compiling the Glibc module as part of the stdlib, we move the module to compile on-demand for each target SDK passed into Swiftc. We can also update the paths used by the importer as the SDK base for the module map. There is some additional glue code, outside of the just the Glibc.modulemap, that would have to be compiled and cached for each non-Darwin target.
Since we are JITing the module, we allow developer to be able to set feature flags on the libc module that the clang importer will set when compiling against the target's libc headers. By default we can implicitly set sensible defaults based on the target's triple (like _GNU_SOURCE on Linux libcs).
TODO: We write some code that does the things
Proposed design:
- Create a PlatformLibC module that re-exports Darwin or Glibc for each arch.
- Move the glibc.modulemap and any extra glibc swift glue code from the platform stdlib package into the lib/swift folder so that it can be compiled for the target SDK on demand.
- Update the compiler frontend to compile and cache the libc module for us
for each new -sdk passed in
- swiftc should apply the same kind of transforms we do in gyb (maybe we can use the standard C++ processor on the file) on the glibc.modulemap
- compile the rest of the glibc module with the extra glue code and store /tmp like the module cache.
- add the cached new glibc module from /tmp to the linker options for specific toolchains jitting this module.
- Add args to the swiftc frontend for defining libc define options for the SDK
Darwin should be unaffected.
Unfortunately the current glibc module on Linux (or anything else) is
already inherently brittle given it's being based on the libc of
whomever built the Glibc module into the toolchain. The glibc module
should remain compatibile for non-Darwin developers that have previously
built their own toolchain for their own system. Developers that rely on
the glibc module to behave like it did from a foreign system's libc
different enough than their own (without obtaining and passing in that same
foreign system's sysroot into the -sdk arg) may experience some source
breakage. This might be people who obtained a pre-built toolchain from
Swift.org for example and ran it on a different OS than it was compiled on.
Any breakage suffered is likely just exposing undetected errors in the target's
build.
Each libc implementation has it's own ABI stability guarantees and this proposal doesn't change that.
The ABI stability of any targeted libc is entirely the responsibility of the developer consuming libc on their target platform to check and ensure that the methods and features they are using from their libc is ABI stable for all the targets they want to deploy their binaries on.
This adds a new public API (the PlatformLibC module) and deprecates usage of
the Darwin and Glibc modules. This modules API is entirely determined by the
target platform's libc and any additional code that is added to the glue code
we have added to build the module.
Instead of JITing the Glibc module, implementing a VFS layer for the clang importer was considered so that we could relocate the target sysroot for SDK for the glibc module map. This would require changes in clang and would still not solve being able to update the glibc module to match the target's libc features.