oneapi_base rez package build script

README.md

This is a work in progress.

N.B., the install payload is extremely large (20+Gb)

package.py

name = 'oneapi_base'

version = '2021.1.6.1113'

authors = ['intel']

description = """
    The Intel® oneAPI Base Toolkit is a core set of tools and libraries for building and deploying high-performance, data-centric applications across diverse architectures.

    It features the Data Parallel C++ (DPC++) language, an evolution of C++ that:
     * Allows code reuse across hardware targets—CPUs, GPUs, and FPGAs1
     * Permits custom tuning for individual accelerators
"""


@early()
def uuid():
    import uuid
    return str(uuid.uuid5(uuid.NAMESPACE_DNS, name))


@early()
def variants():
    from rez.package_py_utils import expand_requires
    requires = ["platform-**", "arch-**"]
    return [expand_requires(*requires)]


def pre_build_commands():
    env.LDFLAGS = '-Wl,--enable-new-dtags $LDFLAGS'
    for c in ["CFLAGS", "CPPFLAGS", "CXXFLAGS"]:
        setenv(c, "$%s -fPIC" % c)
    oneapi_ver = '.'.join(str(this.version).split('.')[0:4])
    url_base = 'https://registrationcenter-download.intel.com/akdlm/irc_nas/16642'
    filename = 'l_BaseKit_b_{oneapi_ver}_offline.tar.gz'
    env.REZ_BUILD_SOURCE_URL = "{url_base}/{filename}"
    env.REZ_BUILD_SOURCE_FILENAME = filename

    if undefined('REZ_REPO_PAYLOAD_DIR'):
        env.REZ_REPO_PAYLOAD_DIR = "{build.install_path}/src"

    def get_component(module):
        modules = {
            'tbb': 'tbb-devel',
            'gdb': 'dpcpp-debugger',
            'ccl': 'ccl',
            'advisor': 'advisor',
            'compat': 'dpcpp-ct',
            'python': 'python',
            'ipp': 'ipp-devel',
            'vtune': 'vtune',
            'dpcpp': 'dpcpp-compiler',
            'libdp': 'libdpstd',
            'daal': 'daal-devel',
            'dnnl': 'dnnl',
            'mkldnn': 'dnnl',
            'mkl_dnn': 'dnnl',
            'mkl': 'mkl-devel',
            'onevpl': 'onevpl'
        }
        return 'intel-oneapi-{0}-for-installer__x86_64'.format(modules[module])

    def get_components(*args):
        components = ';'.join([get_component(c) for c in args])

    env.COMPONENT_STRING = 'COMPONENTS=%s' % get_components(*this.components)


@early()
def components():
    return [
        # toggle the oneapi components to  
        'tbb',
        #'gdb',
        #'ccl',
        #'advisor',
        #'compat',
        #'python',
        #'ipp',
        #'vtune',
        #'dpcpp',
        #'libdp',
        #'daal',
        'dnnl',
        'mkl',
        #'onevpl'
    ]

build_command = """
    wget $REZ_BUILD_SOURCE_URL
    tar -xvf $REZ_BUILD_SOURCE_FILENAME --strip 1

    sed -i 's/ACCEPT_EULA=decline/ACCEPT_EULA=accept/g' silent.cfg
    echo $REZ_BUILD_INSTALL_PATH | xargs -I '{}' sed -i 's#/opt/intel#{}#g' silent.cfg
    sed -i 's/#INTEL_SW_IMPROVEMENT_PROGRAM_CONSENT=no/INTEL_SW_IMPROVEMENT_PROGRAM_CONSENT=no/g' silent.cfg

    echo $COMPONENT_STRING | xargs -I '{}' sed -i 's/#COMPONENTS=/{}/g' silent.cfg
    # echo $REZ_BUILD_SOURCE_PATH/key.lic | xargs -I '{}' sed -i 's@#ACTIVATION_LICENSE_FILE=@ACTIVATION_LICENSE_FILE={}@g' silent.cfg
    # sed -i 's/no_license/license_file/g' silent.cfg

    ./setup.sh --cli-mode --silent ./silent.cfg
    rm -rf ~/intel/isep
"""


def commands():
    env.PATH.append('{root}/inteloneapi/bin')

    # Not quite sure why sourcing setvars.sh here requires appending /usr/bin:/bin to $PATH here.
    # Sourcing setvars.sh in pre_commands() doesn't; post_commands() does!
    # Would be good to properly set up the environment directly in rez, instead of sourcing setvars.sh.
    #env.PATH.append('/usr/bin:/bin')
    #source('{root}/inteloneapi/setvars.sh')
    
    pathver = '{this.version.major}.{this.version.minor}-beta%02d' % this.version.patch
    lib = "lib"
    iarch = 'intel64'
    lp64_ilp64 = 'lp64'
    platform = 'linux'  #todo: darwin, windows
    intel_root = '{root}/inteloneapi'

    mkl_root = '{intel_root}/mkl/{pathver}'
    mkl_lib = '{mkl_root}/lib/{iarch}'
    mkl_include = '{mkl_root}/include'
    mkl_include_mod = '{mkl_root}/include/{iarch}/{lp64_ilp64}'
    mkl_pkgconfig = '{mkl_root}/tools/pkgconfig'
    mkl_nlspath = '{mkl_lib}/locale/\%l_\%t/\%N'
    mkl_bin = '{mkl_root}/bin'

    compiler_root = '{intel_root}/compiler/{pathver}/{platform}'
    compiler_lib = '{compiler_root}/lib'
    compiler_lib_x64 = '{compiler_lib}/x64'
    compiler_include = '{compiler_root}/include'
    compiler_icc_lib = '{compiler_root}/compiler/lib/{iarch}'
    compiler_icc_include = '{compiler_root}/compiler/include'
    env.OCL_ICD_FILENAMES.append('{compiler_lib_x64}/libintelocl.so')

    tbb_root = '{intel_root}/tbb/{pathver}'
    tbb_lib = '{tbb_root}/lib/{iarch}/gcc4.8'
    tbb_include = '{tbb_root}/include'

    mpi_root = '{intel_root}/mpi/{pathver}'
    mpi_lib = '{mpi_root}/lib'
    mpi_lib_release = '{mpi_lib}/release'
    mpi_lib_release_mt = '{mpi_lib}/release_mt'
    mpi_lib_debug = '{mpi_lib}/debug'
    mpi_lib_debug_mt = '{mpi_lib}/debug_mt'
    mpi_include = '{mpi_root}/include'

    daal_root = '{intel_root}/daal/{pathver}'
    daal_lib = '{daal_root}/lib/{iarch}'
    daal_include = '{daal_root}/include'

    ipp_root = '{intel_root}/ipp/{pathver}'
    ipp_lib = '{ipp_root}/lib/{iarch}'
    ipp_include = '{ipp_root}/include'

    dpct_root = '{intel_root}/dpccpp-ct/{pathver}'
    dpct_include = '{dpct_root}/include'

    ccl_root = '{intel_root}/ccl/{pathver}'
    ccl_lib_dpcpp = '{ccl_root}/lib/cpu_gpu_dpcpp'
    ccl_lib_cpu = '{ccl_root}/lib/cpu_icc'
    ccl_include = '{ccl_root}/include'

    # OneDNN aka mkl_dnn
    # Runtime specific dependencies:
    #
    # | Runtime configuration | Requirements                                         |
    # | --------------------- | ---------------------------------------------------- |
    # | `cpu_gomp`            | No additional requirements                           |
    # | `cpu_iomp`            | Intel OpenMP runtime                                 |
    # | `cpu_tbb`             | Threading Building Blocks 2017 or later              |
    # | `cpu_dpcpp_gpu_dpcpp` | Intel DPC++ Compiler Beta                            |
    dnnl_conf = 'cpu_gomp'
    dnnl_root = '{intel_root}/OneDNN/{pathver}/{dnnl_conf}'
    dnnl_lib = '{dnnl_root}/lib'
    dnnl_include = '{dnnl_root}/include'

    # todo: compiler, debugger, vpl, vtune, intelpython, advisor, dev-utilities, fpga

    includes = [
        mkl_include,
        mkl_include_mod,
        tbb_include,
        mpi_include,
        daal_include,
        ipp_include,
        dpct_include,
        ccl_include,
        dnnl_include,  #
    ]

    library_paths = [
        mkl_lib,
        tbb_lib,
        mpi_lib,
        mpi_lib_release_mt,
        daal_lib,
        ipp_lib,
        ccl_lib_dpcpp,  #
        dnnl_lib,
    ]

    paths = [
        mkl_bin,
    ]

    pkgconfigs = [
        mkl_pkgconfig,
    ]

    for p in paths:
        env.PATH.append(p)

    env.MKLROOT = mkl_root
    env.TBBROOT = tbb_root
    env.I_MPI_ROOT = mpi_root
    env.DAALROOT = daal_root
    env.IPPROOT = ipp_root
    env.DPCT_BUNDLE_ROOT = dpct_root
    env.CCL_ROOT = ccl_root
    env.DPCPP_ROOT = intel_root
    env.ONEAPI_ROOT = intel_root
    env.DNNLROOT = dnnl_root
    env.MKLDNN_ROOT = dnnl_root
    #_ = [env.LD_LIBRARY_PATH.append(path) for path in library_paths]
    if building:
        ld_flags = ''
        for lib in library_paths:
            ld_flags += '-L{0} -Wl,-rpath,{0} '.format(lib)
            env.LD_RUN_PATH.append(lib)
            env.LIBRARY_PATH.append(lib)
            env.LD_LIBRARY_PATH.append(lib)
            _ = [env.LD_RUN_PATH.append(path) for path in library_paths]
        env.LDFLAGS = '{ld_flags} $LDFLAGS'

        cflags = ''
        for inc in includes:
            env.CPATH.prepend(inc)
            cflags += '-I%s ' % inc
        for c in ["CFLAGS", "CPPFLAGS", "CXXFLAGS"]:
            setenv(c, '{cflags} $%s' % c)

Not quite sure why sourcing setvars.sh here requires appending /usr/bin:/bin to $PATH here.

You've stumbled onto a known issue here.

The preceding env.PATH.append will overwrite PATH - the first append/prepend on any env-var in rez always does, otherwise you'd have the rez env polluted with bin paths etc from the parent env, and stuff would break. Unfortunately this means that system stuff (ie stuff in /bin etc) is lost - the system paths don't get added back to PATH until the end.

The fix for this is better management of env vars in general. For example, a useful default mode for path-like vars would probably behave like so:

• remember the pre-rez value of PATH
• for every PATH append/prepend in commands, the pre-rez PATH should always appear appended/prepended to that.

In other words, if commands for two packages both appended A and B to PATH (in that order), and the pre-rez PATH was X, then the end value of PATH should be A:B:X. However, immediately after A is appended, PATH should be A:X also. Currently, the values of PATH over time would be A, A:B, A:B:X - they should be A:X, A:B:X (or X:A, X:A:B, depending on how env var management is configured to behave).

Hth
A

Ah! Thanks dude. Appreciate your insight.

So, that's interesting -- I guess I was able to source that script in the pre_commands section without first appending system stuff to PATH because the resolve didn't happen to include any earlier evaluating pre_commands that tinkered with PATH. It's like they say -- live by the pre_command, die by the pre_command.

I think I'm following you, with the better env-var management behavior w.r.t. pre-rez values... so, to use your examples of how PATH mutates over time, wouldn't reconciling env-var values with pre-rez values X for every package 'dirty' the state for the next package processed in the resolve (with X)...? or is that the desired behavior? Would that affect the behavior of a statement like env.LIBS = "-L{root}/libs $LIBS"?

[Also -- I should probably be clearer about this -- I don't think I'd necessarily endorse handling the entire OneAPI base platform like this (installed and managed as a single package). It's been a couple of months since I put this together, and having built a few things against this, I much prefer having individual meta-packages for different components (referencing oneapi_base, if need be)]