Serge breakdown

gistfile1.txt

So I've made enough progress and I currently have enough understanding of what's
happening to share it with you. I can't invest more time on this than what I've
already be doing, hopefully this is going to put you on track.

= Why do we have this -isystem /usr/include?

emcmake passes a toolchain file (cmake/Modules/Platform/Emscripten.cmake) to
cmake that informs it about the toolchain. In the process it sets
CMAKE_CROSSCOMPILING=ON.

During llvm build process, if CMAKE_CROSSCOMPILING=ON, then a sub build of llvm
happens, a "native" build, to build the native tools needed to bootstrap the
compiler. Mostly llvm-tblgen and clang-tblgen, but they are not the only ones.

the configure step for this "native" build is confused by all the environment
variables set by emcmake, which leads to invalid decision. In particular it does
detect some zlib.h and zstd.h headers in the system (in /usr/include) and tries
to use them. But has it still using em++, or maybe because of some env variable,
this native build keeps on trying to generate wasm (!).

So one way to get rid of these extra header is to force the  "native build" not
to look for zlib or zstd. The following patch does so (it also replaces em++ by
clang++ to prevent some of the side effects mentionned above):

patch -p1 << EOF

--- ./llvm/cmake/modules/CrossCompile.cmake    2023-12-14 10:22:34.750923364
0100
+++ ./llvm/cmake/modules/CrossCompile.cmake2   2023-12-14 10:59:55.217196320
0100
@@ -70,8 +70,10 @@
   add_custom_command(OUTPUT
   \${\${project_name}_\${target_name}_BUILD}/CMakeCache.txt
     COMMAND \${CMAKE_COMMAND} -G "\${CMAKE_GENERATOR}"
         -DCMAKE_MAKE_PROGRAM="\${CMAKE_MAKE_PROGRAM}"
-        -DCMAKE_C_COMPILER_LAUNCHER="\${CMAKE_C_COMPILER_LAUNCHER}"
-        -DCMAKE_CXX_COMPILER_LAUNCHER="\${CMAKE_CXX_COMPILER_LAUNCHER}"
+        -DCMAKE_C_COMPILER="clang"
+        -DCMAKE_CXX_COMPILER="clang++"
+        -DLLVM_ENABLE_ZLIB="\${LLVM_ENABLE_ZLIB}"
+        -DLLVM_ENABLE_ZSTD="\${LLVM_ENABLE_ZSTD}"
         \${CROSS_TOOLCHAIN_FLAGS_\${target_name}} \${CMAKE_CURRENT_SOURCE_DIR}
         \${CROSS_TOOLCHAIN_FLAGS_\${project_name}_\${target_name}}
EOF

while passing

    -DLLVM_ENABLE_ZSTD=OFF                          \
    -DLLVM_ENABLE_ZLIB=OFF                          \

If I understand correctly, forcing the compiler to be clang should be enough, no
need to hack LLVM_ENABLE_ZSTD, I'm just sharing the different steps I've been
exploring.

It is however, not enough, but certainly a track to follow.

Another approach that led me to a final clang.js (\o/) is to skip the "native
build" step. This can be done either through modification of the
cmake/Modules/Platform/Emscripten.cmake file or (and that's what I did) a small
patch:

patch -p1 << EOF

--- llvm/CMakeLists.txt 2023-12-14 14:32:34.608021529 +0100
+++ llvm/CMakeLists.txt2        2023-12-14 14:32:25.722915506 +0100
@@ -762,6 +762,7 @@
   message(FATAL_ERROR "Cannot enable BUILD_SHARED_LIBS with LLVM_LINK_LLVM_DYLIB.  We recommend disabling BUILD_SHARED_LIBS.")
 endif()

+set(CMAKE_CROSSCOMPILING OFF)
 option(LLVM_OPTIMIZED_TABLEGEN "Force TableGen to be built with optimization" OFF)
 if(CMAKE_CROSSCOMPILING OR (LLVM_OPTIMIZED_TABLEGEN AND (LLVM_ENABLE_ASSERTIONS OR CMAKE_CONFIGURATION_TYPES)))
   set(LLVM_USE_HOST_TOOLS ON)
EOF

if you do so, you need to provide your own version of the bootstrapped tool. I
did this by recompiling llvm locally to get llvm-tblgen and clang-tblgen, then
passed

    -DLLVM_TABLEGEN=$HOME/sources/llvm-project/_build/bin/llvm-tblgen \
    -DCLANG_TABLEGEN=$HOME/sources/llvm-project/_build/bin/clang-tblgen

with these changes I've been able to build clang.js, but not the install step,
as clang-ast-dump is another missing tool that needs to be compiled natively and
passed to LLVM, but there's no variable to do so. It would be easy to craft a
patch to do so though.

Again, sorry for not "finishing" the job, but I've used more than the time
credit I wanted to allocate to this. Hopefully this long wall of text will help you.

Serge