Running libc++'s test suite.

Instructions.md

1. Checkout the LLVM sources somewhere persistent, for example /opt/llvm-src.

git clone https://llvm.org/git/llvm.git /opt/llvm-src

2. Create an permanent alias to the LIT test driver. This alias will be how you invoke the tests.

   # In ~/.bashrc or similar
   alias lit="python /opt/llvm-src/utils/lit/lit.py"

3. Checkout the libc++ sources to where you want to test from:

git clone https://llvm.org/git/libcxx.git ~/workspace/libcxx

4. Copy the attached lit.site.cfg file to ~/workspace/libcxx/test/lit.site.cfg.

5. Edit the lit.site.cfg to your liking. The example i've provided is setup to test a gcc installation under /opt/gcc-tot, so you'll need to change that as appropriate.

6. Run the tests! To do this simply invoke lit passing the test directory or file you want to run. For example:

lit -sv ~/workspace/libcxx/test/std/utilities

As a side note a couple of common LIT options you'll probably want to use:

• --param=cxx_under_test=<compiler-to-test>

  Change the compiler your testing against.

• --param=std=<std_version>

  Change the dialect used during testing, and disable/enable the approprate tests for that dialect. <std_version> is one of c++03, c++11, c++14, or c++1z.

• --param=compile_flags=<flags>

  Add a list of additional compile flags when running the tests. Hopefully you won't need this.

• --param=link_flags=<flags>

  Add a list of additional link flags when running the tests. Hopefully you won't need this.

lit.site.cfg

# This file belogs in libcxx/test/lit.site.cfg

# Configure the testsuite for libstdc++.
config.cxx_stdlib_under_test    = 'libstdc++'

# The compiler used to test. Set this to your prefered default. You
# can use --param=cxx_under_test=<comp> to override it on the command line.
config.cxx_under_test           = "/opt/gcc-tot/bin/g++"

# The directory containing libstdc++.so. This will add '-L<path>'
# and '-rpath <path>'
config.cxx_library_root         = "/opt/gcc-tot/lib64"

# Enable/Disable testing certain features. These should be the right defaults
# for ToT libstdc++
config.enable_exceptions        = "True"
config.enable_experimental      = "True"
config.enable_filesystem        = "True"
config.enable_rtti              = "True"
config.enable_shared            = "True"
config.enable_32bit             = "False"

# Misc GCC configuration options
config.target_triple            = ""
config.sysroot                  = ""

# This path will be used when generating temporary files when running the tests.
config.project_obj_root         = '/tmp/libcxx-testsuite-output'
config.libcxx_obj_root          = '/tmp/libcxx-testsuite-output'

# Let the main config do the real work.
import os
cfg_path = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lit.cfg')
config.loaded_site_config = True
lit_config.load_config(config, cfg_path)