Skip to content

Instantly share code, notes, and snippets.

@georgemarrows
Last active October 5, 2016 19:32
Show Gist options
  • Save georgemarrows/a4fb838114eae0504abe3fd6d8fd5cc3 to your computer and use it in GitHub Desktop.
Save georgemarrows/a4fb838114eae0504abe3fd6d8fd5cc3 to your computer and use it in GitHub Desktop.
Attempt at providing an `llvm39` homebrew formula for homebrew-versions

This is currently just a light SHA update of the current llvm formula, which is at 3.8.1. Not surprisingly it ends up looking a lot like Godin's update of llvm to 3.9.0 here: Homebrew/homebrew-core#4579 . LLVM switched to CMake-based builds after 3.8. This is why I haven't adapted the current llvm38 formula because it (probably) requires a good bit of changing to move it over from autoconfig.

Current status

  • brew install llvm39 builds to completion - see _latest_build.txt
  • creates /usr/local/opt/llvm39 directory as a symlink to /usr/local/Cellar/llvm39/3.9.0/
  • brew test llvm39 fails on the first C++ build with problems finding stdarg.h - see _test_failure.txt
    • This seems to be because I have XCode 8 but Command Line Tools 7.3 - am fixing this
    • brew test llvm fails in the same way on my Mac

Questions

  • do we want to base the llvm39 build off the proposed homebrew-core llvm 3.9 update?
    • the big advantage is consistency between the llvm and llvm39 versions
    • the disadvantage is the potential change for LLVM users between llvm38 and llvm39
  • llvm38 seems to suffix all binaries with -38. Should llvm39 do the same? If yes, is there a standard Homebrew trick for this, or is it specific to the LLVM build scripts?
  • DONE I'm not v knowledgeable about either LLVM or Homebrew (perhaps not a good basis for this work!) so am not sure what llvm39 should look like when installed. Any pointers?
    • the adapted llvm formula seems to do the right thing.

Proposed tests

For each set of options...

  • --with-all-targets --with-graphviz --with-lldb --with-ocaml --with-python --with-test --with-toolchain
  • --with-shared-libs

... run the following tests

  • run formula tests
  • build Pony - DONE - self test and hello world work
  • build others that use LLVM 3.9
  • manually check things are getting installed in the correct location - DONE /usr/local/opt/llvm39 links to Cellar
6:51 $ brew install --keep-tmp llvm39
==> Downloading https://homebrew.bintray.com/bottles/llvm39-3.9.0.el_capitan.bottle.tar.gz
curl: (22) The requested URL returned error: 404 Not Found
Error: Failed to download resource "llvm39"
Download failed: https://homebrew.bintray.com/bottles/llvm39-3.9.0.el_capitan.bottle.tar.gz
Warning: Bottle installation failed: building from source.
==> Using the sandbox
==> Downloading http://llvm.org/releases/3.9.0/llvm-3.9.0.src.tar.xz
Already downloaded: /Users/georgem/Library/Caches/Homebrew/llvm39-3.9.0.tar.xz
==> Downloading http://llvm.org/releases/3.9.0/cfe-3.9.0.src.tar.xz
Already downloaded: /Users/georgem/Library/Caches/Homebrew/llvm39--clang-3.9.0.tar.xz
==> Downloading http://llvm.org/releases/3.9.0/clang-tools-extra-3.9.0.src.tar.xz
Already downloaded: /Users/georgem/Library/Caches/Homebrew/llvm39--clang-tools-extra-3.9.0.tar.xz
==> Downloading http://llvm.org/releases/3.9.0/openmp-3.9.0.src.tar.xz
Already downloaded: /Users/georgem/Library/Caches/Homebrew/llvm39--openmp-3.9.0.tar.xz
==> Downloading http://llvm.org/releases/3.9.0/libcxx-3.9.0.src.tar.xz
Already downloaded: /Users/georgem/Library/Caches/Homebrew/llvm39--libcxx-3.9.0.tar.xz
==> Downloading http://llvm.org/releases/3.9.0/libunwind-3.9.0.src.tar.xz
Already downloaded: /Users/georgem/Library/Caches/Homebrew/llvm39--libunwind-3.9.0.tar.xz
==> Downloading http://llvm.org/releases/3.9.0/lld-3.9.0.src.tar.xz
Already downloaded: /Users/georgem/Library/Caches/Homebrew/llvm39--lld-3.9.0.tar.xz
==> Downloading http://llvm.org/releases/3.9.0/polly-3.9.0.src.tar.xz
Already downloaded: /Users/georgem/Library/Caches/Homebrew/llvm39--polly-3.9.0.tar.xz
==> Downloading http://llvm.org/releases/3.9.0/compiler-rt-3.9.0.src.tar.xz
Already downloaded: /Users/georgem/Library/Caches/Homebrew/llvm39--compiler-rt-3.9.0.tar.xz
==> cmake -G Unix Makefiles /private/tmp/llvm39-20161003-21563-1nq8pf2/llvm-3.9.0.src -DCMAKE_C_FLAGS_RELEASE=-DNDEBUG -DCMAKE_CXX_FLAGS_RELEASE=-DNDEBUG -DCMAKE_INSTALL_PREFIX=/usr/local/Cellar/llvm39/3.9.0 -DCMAKE_BUILD_TYPE=Release -DC
==> make
==> make install
==> Kept temporary files
Temporary files retained at /tmp/llvm39-20161003-21563-1nq8pf2
==> Caveats
LLVM executables are installed in /usr/local/opt/llvm39/bin.
Extra tools are installed in /usr/local/opt/llvm39/share/llvm39.
To use the bundled libc++ please add the following LDFLAGS:
LDFLAGS="-L/usr/local/opt/llvm39/lib -Wl,-rpath,/usr/local/opt/llvm39/lib"
This formula is keg-only, which means it was not symlinked into /usr/local.
macOS already provides this software and installing another version in
parallel can cause all kinds of trouble.
Generally there are no consequences of this for you. If you build your
own software and it requires this formula, you'll need to add to your
build variables:
LDFLAGS: -L/usr/local/opt/llvm39/lib
CPPFLAGS: -I/usr/local/opt/llvm39/include
If you need Python to find bindings for this keg-only formula, run:
echo /usr/local/opt/llvm39/lib/python2.7/site-packages >> /usr/local/lib/python2.7/site-packages/llvm39.pth
7:29 $ brew test llvm39
Testing llvm39
==> Using the sandbox
==> /usr/local/Cellar/llvm39/3.9.0/bin/llvm-config --prefix
==> /usr/local/Cellar/llvm39/3.9.0/bin/clang -L/usr/local/Cellar/llvm39/3.9.0/lib -fopenmp -nobuiltininc -I/usr/local/Cellar/llvm39/3.9.0/lib/clang/3.9.0/include omptest.c -o omptest
==> ./omptest
==> /usr/local/Cellar/llvm39/3.9.0/bin/clang++ -v -nostdinc -I/Library/Developer/CommandLineTools/usr/include/c++/v1 -I/include -I/usr/include test.cpp -o testCLT++
Last 15 lines from /Users/georgem/Library/Logs/Homebrew/llvm39/test.02.clang++:
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
/Library/Developer/CommandLineTools/usr/include/c++/v1
/usr/include
/usr/local/include
End of search list.
In file included from test.cpp:1:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/iostream:38:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/ios:215:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/iosfwd:90:
/usr/include/wchar.h:89:10: fatal error: 'stdarg.h' file not found
#include <stdarg.h>
^
1 error generated.
Error: llvm39: failed
Failed executing: /usr/local/Cellar/llvm39/3.9.0/bin/clang++ -v -nostdinc -I/Library/Developer/CommandLineTools/usr/include/c++/v1 -I/include -I/usr/include test.cpp -o testCLT++
/usr/local/Homebrew/Library/Homebrew/formula.rb:1733:in `block in system'
/usr/local/Homebrew/Library/Homebrew/formula.rb:1671:in `open'
/usr/local/Homebrew/Library/Homebrew/formula.rb:1671:in `system'
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/llvm39.rb:352:in `block in <class:Llvm39>'
/usr/local/Homebrew/Library/Homebrew/formula.rb:1571:in `block (2 levels) in run_test'
/usr/local/Homebrew/Library/Homebrew/formula.rb:832:in `with_logging'
/usr/local/Homebrew/Library/Homebrew/formula.rb:1570:in `block in run_test'
/usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:14:in `block in mktemp'
/usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:74:in `block in run'
/usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:74:in `chdir'
/usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:74:in `run'
/usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:13:in `mktemp'
/usr/local/Homebrew/Library/Homebrew/formula.rb:1564:in `run_test'
/usr/local/Homebrew/Library/Homebrew/test.rb:28:in `block in <main>'
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/timeout.rb:66:in `timeout'
/usr/local/Homebrew/Library/Homebrew/test.rb:27:in `<main>'
class CodesignRequirement < Requirement
include FileUtils
fatal true
satisfy(:build_env => false) do
mktemp do
cp "/usr/bin/false", "llvm_check"
quiet_system "/usr/bin/codesign", "-f", "-s", "lldb_codesign", "--dryrun", "llvm_check"
end
end
def message
<<-EOS.undent
lldb_codesign identity must be available to build with LLDB.
See: https://llvm.org/svn/llvm-project/lldb/trunk/docs/code-signing.txt
EOS
end
end
class Llvm39 < Formula
desc "Next-gen compiler infrastructure"
homepage "http://llvm.org/"
stable do
url "http://llvm.org/releases/3.9.0/llvm-3.9.0.src.tar.xz"
sha256 "66c73179da42cee1386371641241f79ded250e117a79f571bbd69e56daa48948"
resource "clang" do
url "http://llvm.org/releases/3.9.0/cfe-3.9.0.src.tar.xz"
sha256 "7596a7c7d9376d0c89e60028fe1ceb4d3e535e8ea8b89e0eb094e0dcb3183d28"
end
resource "clang-tools-extra" do
url "http://llvm.org/releases/3.9.0/clang-tools-extra-3.9.0.src.tar.xz"
sha256 "5b7aec46ec8e999ec683c87ad744082e1133781ee4b01905b4bdae5d20785f14"
end
resource "compiler-rt" do
url "http://llvm.org/releases/3.9.0/compiler-rt-3.9.0.src.tar.xz"
sha256 "e0e5224fcd5740b61e416c549dd3dcda92f10c524216c1edb5e979e42078a59a"
end
# Only required to build & run Compiler-RT tests on OS X, optional otherwise.
# http://clang.llvm.org/get_started.html
resource "libcxx" do
url "http://llvm.org/releases/3.9.0/libcxx-3.9.0.src.tar.xz"
sha256 "d0b38d51365c6322f5666a2a8105785f2e114430858de4c25a86b49f227f5b06"
end
resource "libunwind" do
url "http://llvm.org/releases/3.9.0/libunwind-3.9.0.src.tar.xz"
sha256 "66675ddec5ba0d36689757da6008cb2596ee1a9067f4f598d89ce5a3b43f4c2b"
end
resource "lld" do
url "http://llvm.org/releases/3.9.0/lld-3.9.0.src.tar.xz"
sha256 "986e8150ec5f457469a20666628bf634a5ca992a53e157f3b69dbc35056b32d9"
end
resource "lldb" do
url "http://llvm.org/releases/3.9.0/lldb-3.9.0.src.tar.xz"
sha256 "61280e07411e3f2b4cca0067412b39c16b0a9edd19d304d3fc90249899d12384"
end
resource "openmp" do
url "http://llvm.org/releases/3.9.0/openmp-3.9.0.src.tar.xz"
sha256 "df88f90d7e5b5e9525a35fa2e2b93cbbb83c4882f91df494e87ee3ceddacac91"
end
resource "polly" do
url "http://llvm.org/releases/3.9.0/polly-3.9.0.src.tar.xz"
sha256 "ef0dd25010099baad84597cf150b543c84feac2574d055d6780463d5de8cd97e"
end
end
bottle do
# FIXME
sha256 "ce2d461a9aff5e4b91bdb76f855be312e0b2f3e8a92950ff8628a465ea9f71cb" => :sierra
sha256 "3198e307a4360b428c953cc00408e5b80bd07b4946b96db8bbbfb03f81cb4aa2" => :el_capitan
sha256 "1957948bbbbee09eac331b2b0bad399b04f94d8c79271ea0aee4ac99ab7e6100" => :yosemite
sha256 "dba4556fbb273ef07dcc135a32eabb4b2eafb874ceecb9732b2ccaeded965a80" => :mavericks
end
# FIXME add 3.9 branches
head do
url "http://llvm.org/git/llvm.git"
resource "clang" do
url "http://llvm.org/git/clang.git"
end
resource "clang-tools-extra" do
url "http://llvm.org/git/clang-tools-extra.git"
end
resource "compiler-rt" do
url "http://llvm.org/git/compiler-rt.git"
end
resource "libcxx" do
url "http://llvm.org/git/libcxx.git"
end
resource "libunwind" do
url "http://llvm.org/git/libunwind.git"
end
resource "lld" do
url "http://llvm.org/git/lld.git"
end
resource "lldb" do
url "http://llvm.org/git/lldb.git"
end
resource "openmp" do
url "http://llvm.org/git/openmp.git"
end
resource "polly" do
url "http://llvm.org/git/polly.git"
end
end
keg_only :provided_by_osx
option :universal
option "without-compiler-rt", "Do not build Clang runtime support libraries for code sanitizers, builtins, and profiling"
option "without-libcxx", "Do not build libc++ standard library"
option "with-toolchain", "Build with Toolchain to facilitate overriding system compiler"
option "without-libunwind", "Do not build libunwind library"
option "without-lld", "Do not build LLD linker"
option "with-lldb", "Build LLDB debugger"
option "with-python", "Build bindings against custom Python"
option "without-rtti", "Build without C++ RTTI or exception handling"
option "without-utils", "Do not install utility binaries"
option "without-polly", "Do not build Polly optimizer"
option "with-test", "Build LLVM unit tests"
option "with-shared-libs", "Build shared instead of static libraries"
option "without-libffi", "Do not use libffi to call external functions"
option "with-all-targets", "Build all targets. Default targets: AMDGPU, ARM, NVPTX, and X86"
depends_on "libffi" => :recommended # http://llvm.org/docs/GettingStarted.html#requirement
depends_on "graphviz" => :optional # for the 'dot' tool (lldb)
depends_on "ocaml" => :optional
if MacOS.version <= :snow_leopard
depends_on :python
else
depends_on :python => :optional
end
depends_on "cmake" => :build
if build.with? "lldb"
depends_on "swig" if MacOS.version >= :lion
depends_on CodesignRequirement
end
# Apple's libstdc++ is too old to build LLVM
fails_with :llvm
# According to the official llvm readme, GCC 4.7+ is required
fails_with :gcc_4_0
fails_with :gcc
("4.3".."4.6").each do |n|
fails_with :gcc => n
end
def build_libcxx?
build.with?("libcxx") || !MacOS::CLT.installed?
end
def install
# Apple's libstdc++ is too old to build LLVM
ENV.libcxx if ENV.compiler == :clang
(buildpath/"tools/clang").install resource("clang")
(buildpath/"tools/clang/tools/extra").install resource("clang-tools-extra")
(buildpath/"projects/openmp").install resource("openmp")
(buildpath/"projects/libcxx").install resource("libcxx") if build_libcxx?
(buildpath/"projects/libunwind").install resource("libunwind") if build.with? "libunwind"
(buildpath/"tools/lld").install resource("lld") if build.with? "lld"
(buildpath/"tools/polly").install resource("polly") if build.with? "polly"
if build.with? "lldb"
if build.with? "python"
pyhome = `python-config --prefix`.chomp
ENV["PYTHONHOME"] = pyhome
pylib = "#{pyhome}/lib/libpython2.7.dylib"
pyinclude = "#{pyhome}/include/python2.7"
end
(buildpath/"tools/lldb").install resource("lldb")
# Building lldb requires a code signing certificate.
# The instructions provided by llvm creates this certificate in the
# user's login keychain. Unfortunately, the login keychain is not in
# the search path in a superenv build. The following three lines add
# the login keychain to ~/Library/Preferences/com.apple.security.plist,
# which adds it to the superenv keychain search path.
mkdir_p "#{ENV["HOME"]}/Library/Preferences"
username = ENV["USER"]
system "security", "list-keychains", "-d", "user", "-s", "/Users/#{username}/Library/Keychains/login.keychain"
end
if build.with? "compiler-rt"
(buildpath/"projects/compiler-rt").install resource("compiler-rt")
# compiler-rt has some iOS simulator features that require i386 symbols
# I'm assuming the rest of clang needs support too for 32-bit compilation
# to work correctly, but if not, perhaps universal binaries could be
# limited to compiler-rt. llvm makes this somewhat easier because compiler-rt
# can almost be treated as an entirely different build from llvm.
ENV.permit_arch_flags
end
args = %w[
-DLLVM_OPTIMIZED_TABLEGEN=ON
]
args << "-DLLVM_TARGETS_TO_BUILD=#{build.with?("all-targets") ? "all" : "AMDGPU;ARM;NVPTX;X86"}"
args << "-DLIBOMP_ARCH=x86_64"
args << "-DLLVM_BUILD_EXTERNAL_COMPILER_RT=ON" if build.with? "compiler-rt"
args << "-DLLVM_CREATE_XCODE_TOOLCHAIN=ON" if build.with? "toolchain"
if build.with? "shared-libs"
args << "-DBUILD_SHARED_LIBS=ON"
args << "-DLIBOMP_ENABLE_SHARED=ON"
else
args << "-DLLVM_BUILD_LLVM_DYLIB=ON"
end
if build.with? "test"
args << "-DLLVM_BUILD_TESTS=ON"
args << "-DLLVM_ABI_BREAKING_CHECKS=WITH_ASSERTS"
end
if build.with? "rtti"
args << "-DLLVM_ENABLE_RTTI=ON"
args << "-DLLVM_ENABLE_EH=ON"
end
args << "-DLLVM_INSTALL_UTILS=ON" if build.with? "utils"
args << "-DLLVM_ENABLE_LIBCXX=ON" if build_libcxx?
if build.with?("lldb") && build.with?("python")
args << "-DLLDB_RELOCATABLE_PYTHON=ON"
args << "-DPYTHON_LIBRARY=#{pylib}"
args << "-DPYTHON_INCLUDE_DIR=#{pyinclude}"
end
if build.with? "libffi"
args << "-DLLVM_ENABLE_FFI=ON"
args << "-DFFI_INCLUDE_DIR=#{Formula["libffi"].opt_lib}/libffi-#{Formula["libffi"].version}/include"
args << "-DFFI_LIBRARY_DIR=#{Formula["libffi"].opt_lib}"
end
if build.universal?
ENV.permit_arch_flags
args << "-DCMAKE_OSX_ARCHITECTURES=#{Hardware::CPU.universal_archs.as_cmake_arch_flags}"
end
if build.with? "polly"
args << "-DWITH_POLLY=ON"
args << "-DLINK_POLLY_INTO_TOOLS=ON"
end
mktemp do
system "cmake", "-G", "Unix Makefiles", buildpath, *(std_cmake_args + args)
system "make"
system "make", "install"
system "make", "install-xcode-toolchain" if build.with? "toolchain"
end
(share/"clang/tools").install Dir["tools/clang/tools/scan-{build,view}"]
inreplace "#{share}/clang/tools/scan-build/bin/scan-build", "$RealBin/bin/clang", "#{bin}/clang"
bin.install_symlink share/"clang/tools/scan-build/bin/scan-build", share/"clang/tools/scan-view/bin/scan-view"
man1.install_symlink share/"clang/tools/scan-build/man/scan-build.1"
# install llvm python bindings
(lib/"python2.7/site-packages").install buildpath/"bindings/python/llvm"
(lib/"python2.7/site-packages").install buildpath/"tools/clang/bindings/python/clang"
end
def caveats
s = <<-EOS.undent
LLVM executables are installed in #{opt_bin}.
Extra tools are installed in #{opt_pkgshare}.
EOS
if build_libcxx?
s += <<-EOS.undent
To use the bundled libc++ please add the following LDFLAGS:
LDFLAGS="-L#{opt_lib} -Wl,-rpath,#{opt_lib}"
EOS
end
s
end
test do
assert_equal prefix.to_s, shell_output("#{bin}/llvm-config --prefix").chomp
(testpath/"omptest.c").write <<-EOS.undent
#include <stdlib.h>
#include <stdio.h>
#include <omp.h>
int main() {
#pragma omp parallel num_threads(4)
{
printf("Hello from thread %d, nthreads %d\\n", omp_get_thread_num(), omp_get_num_threads());
}
return EXIT_SUCCESS;
}
EOS
system "#{bin}/clang", "-L#{lib}", "-fopenmp", "-nobuiltininc",
"-I#{lib}/clang/#{version}/include",
"omptest.c", "-o", "omptest"
testresult = shell_output("./omptest")
sorted_testresult = testresult.split("\n").sort.join("\n")
expected_result = <<-EOS.undent
Hello from thread 0, nthreads 4
Hello from thread 1, nthreads 4
Hello from thread 2, nthreads 4
Hello from thread 3, nthreads 4
EOS
assert_equal expected_result.strip, sorted_testresult.strip
(testpath/"test.c").write <<-EOS.undent
#include <stdio.h>
int main()
{
printf("Hello World!\\n");
return 0;
}
EOS
(testpath/"test.cpp").write <<-EOS.undent
#include <iostream>
int main()
{
std::cout << "Hello World!" << std::endl;
return 0;
}
EOS
# Testing Command Line Tools
if MacOS::CLT.installed?
libclangclt = Dir["/Library/Developer/CommandLineTools/usr/lib/clang/#{DevelopmentTools.clang_version}*"].last { |f| File.directory? f }
system "#{bin}/clang++", "-v", "-nostdinc",
"-I/Library/Developer/CommandLineTools/usr/include/c++/v1",
"-I#{libclangclt}/include",
"-I/usr/include", # need it because /Library/.../usr/include/c++/v1/iosfwd refers to <wchar.h>, which CLT installs to /usr/include
"test.cpp", "-o", "testCLT++"
assert_match "/usr/lib/libc++.1.dylib", shell_output("otool -L ./testCLT++").chomp
assert_equal "Hello World!", shell_output("./testCLT++").chomp
system "#{bin}/clang", "-v", "-nostdinc",
"-I/usr/include", # this is where CLT installs stdio.h
"test.c", "-o", "testCLT"
assert_equal "Hello World!", shell_output("./testCLT").chomp
end
# Testing Xcode
if MacOS::Xcode.installed?
libclangxc = Dir["#{MacOS::Xcode.toolchain_path}/usr/lib/clang/#{DevelopmentTools.clang_version}*"].last { |f| File.directory? f }
system "#{bin}/clang++", "-v", "-nostdinc",
"-I#{MacOS::Xcode.toolchain_path}/usr/include/c++/v1",
"-I#{libclangxc}/include",
"-I#{MacOS.sdk_path}/usr/include",
"test.cpp", "-o", "testXC++"
assert_match "/usr/lib/libc++.1.dylib", shell_output("otool -L ./testXC++").chomp
assert_equal "Hello World!", shell_output("./testXC++").chomp
system "#{bin}/clang", "-v", "-nostdinc",
"-I#{MacOS.sdk_path}/usr/include",
"test.c", "-o", "testXC"
assert_equal "Hello World!", shell_output("./testXC").chomp
end
# link against installed libc++
# related to https://github.com/Homebrew/legacy-homebrew/issues/47149
if build_libcxx?
system "#{bin}/clang++", "-v", "-nostdinc",
"-std=c++11", "-stdlib=libc++",
"-I#{MacOS::Xcode.toolchain_path}/usr/include/c++/v1",
"-I#{libclangxc}/include",
"-I#{MacOS.sdk_path}/usr/include",
"-L#{lib}",
"-Wl,-rpath,#{lib}", "test.cpp", "-o", "test"
assert_match "#{opt_lib}/libc++.1.dylib", shell_output("otool -L ./test").chomp
assert_equal "Hello World!", shell_output("./test").chomp
end
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment