bazel.log

python  build/build.py --enable_cuda True  --cuda_path $EBROOTCUDA --cudnn_path $EBROOTCUDNN --cuda_compute_capabilities "7.0,7.5,8.0" 

     _   _  __  __
    | | / \ \ \/ /
 _  | |/ _ \ \  /
| |_| / ___ \/  \
 \___/_/   \/_/\_\


Starting local Bazel server and connecting to it...
Bazel binary path: ./bazel-2.0.0-linux-x86_64
Python binary path: /p/software/jusuf/stages/Devel-2020/software/Python/3.8.5-GCCcore-9.3.0/bin/python
Python version: 3.8
MKL-DNN enabled: yes
-march=native: no
CUDA enabled: yes
CUDA toolkit path: /p/software/jusuf/stages/Devel-2020/software/CUDA/11.0
CUDNN library path: /p/software/jusuf/stages/Devel-2020/software/cuDNN/8.0.2.39-CUDA-11.0
CUDA compute capabilities: 7.0,7.5,8.0

Building XLA and installing it in the jaxlib source tree...
./bazel-2.0.0-linux-x86_64 run --verbose_failures=true --config=short_logs --config=mkl_open_source_only --config=cuda --define=xla_python_enable_gpu=true :install_xla_in_source_tree /p/project/ccstao/cstao05/jax/jax/build
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=0 --terminal_columns=80
INFO: Reading rc options for 'run' from /p/project/ccstao/cstao05/jax/jax/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'run' from /p/project/ccstao/cstao05/jax/jax/.bazelrc:
  Inherited 'build' options: --repo_env PYTHON_BIN_PATH=/p/software/jusuf/stages/Devel-2020/software/Python/3.8.5-GCCcore-9.3.0/bin/python --python_path=/p/software/jusuf/stages/Devel-2020/software/Python/3.8.5-GCCcore-9.3.0/bin/python --repo_env TF_NEED_CUDA=1 --action_env TF_CUDA_COMPUTE_CAPABILITIES=7.0,7.5,8.0 --distinct_host_configuration=false --copt=-Wno-sign-compare -c opt --apple_platform_type=macos --macos_minimum_os=10.9 --announce_rc --define open_source_build=true --define=no_aws_support=true --define=no_gcp_support=true --define=no_hdfs_support=true --define=no_kafka_support=true --define=no_ignite_support=true --define=grpc_no_ares=true --spawn_strategy=standalone --strategy=Genrule=standalone --cxxopt=-std=c++14 --host_cxxopt=-std=c++14 --action_env CUDA_TOOLKIT_PATH=/p/software/jusuf/stages/Devel-2020/software/CUDA/11.0 --action_env CUDNN_INSTALL_PATH=/p/software/jusuf/stages/Devel-2020/software/cuDNN/8.0.2.39-CUDA-11.0
INFO: Found applicable config definition build:short_logs in file /p/project/ccstao/cstao05/jax/jax/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:mkl_open_source_only in file /p/project/ccstao/cstao05/jax/jax/.bazelrc: --define=tensorflow_mkldnn_contraction_kernel=1
INFO: Found applicable config definition build:cuda in file /p/project/ccstao/cstao05/jax/jax/.bazelrc: --crosstool_top=@local_config_cuda//crosstool:toolchain --define=using_cuda=true --define=using_cuda_nvcc=true
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
INFO: Call stack for the definition of repository 'local_config_cuda' which is a cuda_configure (rule definition at /p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/third_party/gpus/cuda_configure.bzl:1430:18):
 - /p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/tensorflow/workspace.bzl:96:5
 - /p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/tensorflow/workspace.bzl:75:5
 - /p/project/ccstao/cstao05/jax/jax/WORKSPACE:46:1
ERROR: An error occurred during the fetch of repository 'local_config_cuda':
   Traceback (most recent call last):
	File "/p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/third_party/gpus/cuda_configure.bzl", line 1400
		_create_local_cuda_repository(<1 more arguments>)
	File "/p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/third_party/gpus/cuda_configure.bzl", line 977, in _create_local_cuda_repository
		_get_cuda_config(repository_ctx, <1 more arguments>)
	File "/p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/third_party/gpus/cuda_configure.bzl", line 666, in _get_cuda_config
		find_cuda_config(repository_ctx, <2 more arguments>)
	File "/p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/third_party/gpus/cuda_configure.bzl", line 643, in find_cuda_config
		_exec_find_cuda_config(<3 more arguments>)
	File "/p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/third_party/gpus/cuda_configure.bzl", line 637, in _exec_find_cuda_config
		execute(repository_ctx, <1 more arguments>)
	File "/p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/third_party/remote_config/common.bzl", line 217, in execute
		fail(<1 more arguments>)
Repository command failed
Could not find any cublas_api.h matching version '' in any subdirectory:
        ''
        'include'
        'include/cuda'
        'include/*-linux-gnu'
        'extras/CUPTI/include'
        'include/cuda/CUPTI'
of:
        '/lib'
        '/opt/ddn/ime//lib'
        '/usr'
        '/usr/lib64'
        '/usr/lib64//bind9-export'
        '/usr/lib64/dyninst'
        '/usr/lib64/mysql'
        '/usr/lib64/qt-3.3/lib'
        '/usr/lib64/slurm'
ERROR: Skipping ':install_xla_in_source_tree': no such package '@local_config_cuda//cuda': Traceback (most recent call last):
	File "/p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/third_party/gpus/cuda_configure.bzl", line 1400
		_create_local_cuda_repository(<1 more arguments>)
	File "/p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/third_party/gpus/cuda_configure.bzl", line 977, in _create_local_cuda_repository
		_get_cuda_config(repository_ctx, <1 more arguments>)
	File "/p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/third_party/gpus/cuda_configure.bzl", line 666, in _get_cuda_config
		find_cuda_config(repository_ctx, <2 more arguments>)
	File "/p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/third_party/gpus/cuda_configure.bzl", line 643, in find_cuda_config
		_exec_find_cuda_config(<3 more arguments>)
	File "/p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/third_party/gpus/cuda_configure.bzl", line 637, in _exec_find_cuda_config
		execute(repository_ctx, <1 more arguments>)
	File "/p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/third_party/remote_config/common.bzl", line 217, in execute
		fail(<1 more arguments>)
Repository command failed
Could not find any cublas_api.h matching version '' in any subdirectory:
        ''
        'include'
        'include/cuda'
        'include/*-linux-gnu'
        'extras/CUPTI/include'
        'include/cuda/CUPTI'
of:
        '/lib'
        '/opt/ddn/ime//lib'
        '/usr'
        '/usr/lib64'
        '/usr/lib64//bind9-export'
        '/usr/lib64/dyninst'
        '/usr/lib64/mysql'
        '/usr/lib64/qt-3.3/lib'
        '/usr/lib64/slurm'
WARNING: Target pattern parsing failed.
ERROR: no such package '@local_config_cuda//cuda': Traceback (most recent call last):
	File "/p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/third_party/gpus/cuda_configure.bzl", line 1400
		_create_local_cuda_repository(<1 more arguments>)
	File "/p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/third_party/gpus/cuda_configure.bzl", line 977, in _create_local_cuda_repository
		_get_cuda_config(repository_ctx, <1 more arguments>)
	File "/p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/third_party/gpus/cuda_configure.bzl", line 666, in _get_cuda_config
		find_cuda_config(repository_ctx, <2 more arguments>)
	File "/p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/third_party/gpus/cuda_configure.bzl", line 643, in find_cuda_config
		_exec_find_cuda_config(<3 more arguments>)
	File "/p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/third_party/gpus/cuda_configure.bzl", line 637, in _exec_find_cuda_config
		execute(repository_ctx, <1 more arguments>)
	File "/p/project/ccstao/cstao05/.cache/bazel/_bazel_strube1/3720009e78cd59f073890b18a1ce60fb/external/org_tensorflow/third_party/remote_config/common.bzl", line 217, in execute
		fail(<1 more arguments>)
Repository command failed
Could not find any cublas_api.h matching version '' in any subdirectory:
        ''
        'include'
        'include/cuda'
        'include/*-linux-gnu'
        'extras/CUPTI/include'
        'include/cuda/CUPTI'
of:
        '/lib'
        '/opt/ddn/ime//lib'
        '/usr'
        '/usr/lib64'
        '/usr/lib64//bind9-export'
        '/usr/lib64/dyninst'
        '/usr/lib64/mysql'
        '/usr/lib64/qt-3.3/lib'
        '/usr/lib64/slurm'
INFO: Elapsed time: 202.327s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
ERROR: Build failed. Not running target
FAILED: Build did NOT complete successfully (0 packages loaded)
Traceback (most recent call last):
  File "build/build.py", line 380, in <module>
    main()
  File "build/build.py", line 375, in main
    shell(command)
  File "build/build.py", line 47, in shell
    output = subprocess.check_output(cmd)
  File "/p/software/jusuf/stages/Devel-2020/software/Python/3.8.5-GCCcore-9.3.0/lib/python3.8/subprocess.py", line 411, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/p/software/jusuf/stages/Devel-2020/software/Python/3.8.5-GCCcore-9.3.0/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['./bazel-2.0.0-linux-x86_64', 'run', '--verbose_failures=true', '--config=short_logs', '--config=mkl_open_source_only', '--config=cuda', '--define=xla_python_enable_gpu=true', ':install_xla_in_source_tree', '/p/project/ccstao/cstao05/jax/jax/build']' returned non-zero exit status 1.