Created
February 2, 2021 05:25
-
-
Save feifzhou/152d5c6e15e3485befa78e69cd340c32 to your computer and use it in GitHub Desktop.
Build log for JAX on linux ppc64le
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ python build/build.py --cuda_path /usr/tce/packages/cuda/cuda-10.1.243 --noenable_mkl_dnn --enable_cuda --cudnn_path /usr/tce/packages/cudnn/cudnn-7.0-cuda-9.0/ --cudnn_version 7.0 --bazel_path $HOME/bin/bazel | |
| _ _ __ __ | |
| | | / \ \ \/ / | |
| _ | |/ _ \ \ / | |
| | |_| / ___ \/ \ | |
| \___/_/ \/_/\_\ | |
| WARNING: Output base '/usr/WS2/zhou6/.cache/bazel/_bazel_zhou6/45ac699d106f4b2f92bdd647a162ed50' is on NFS. This may lead to surprising failures and undetermined behavior. | |
| Extracting Bazel installation... | |
| Starting local Bazel server and connecting to it... | |
| Bazel binary path: /g/g90/zhou6/bin/bazel | |
| Python binary path: /g/g90/zhou6/lassen-space/conda-ibm/envs/powerai/bin/python | |
| Python version: 3.6 | |
| MKL-DNN enabled: no | |
| -march=native: no | |
| CUDA enabled: yes | |
| CUDA toolkit path: /usr/tce/packages/cuda/cuda-10.1.243 | |
| CUDNN library path: /usr/tce/packages/cudnn/cudnn-7.0-cuda-9.0/ | |
| CUDA compute capabilities: 3.5,5.2,6.0,6.1,7.0 | |
| CUDNN version: 7.0 | |
| ROCm enabled: no | |
| Building XLA and installing it in the jaxlib source tree... | |
| /g/g90/zhou6/bin/bazel run --verbose_failures=true --config=short_logs --config=cuda --define=xla_python_enable_gpu=true :build_wheel -- --output_path=/usr/WS2/zhou6/jax/dist | |
| WARNING: Output base '/usr/WS2/zhou6/.cache/bazel/_bazel_zhou6/45ac699d106f4b2f92bdd647a162ed50' is on NFS. This may lead to surprising failures and undetermined behavior. | |
| INFO: Options provided by the client: | |
| Inherited 'common' options: --isatty=0 --terminal_columns=80 | |
| INFO: Reading rc options for 'run' from /usr/WS2/zhou6/jax/.bazelrc: | |
| Inherited 'common' options: --experimental_repo_remote_exec | |
| INFO: Reading rc options for 'run' from /usr/WS2/zhou6/jax/.bazelrc: | |
| Inherited 'build' options: --repo_env PYTHON_BIN_PATH=/g/g90/zhou6/lassen-space/conda-ibm/envs/powerai/bin/python --action_env=PYENV_ROOT --python_path=/g/g90/zhou6/lassen-space/conda-ibm/envs/powerai/bin/python --repo_env TF_NEED_CUDA=1 --action_env TF_CUDA_COMPUTE_CAPABILITIES=3.5,5.2,6.0,6.1,7.0 --repo_env TF_NEED_ROCM=0 --action_env TF_ROCM_AMDGPU_TARGETS=gfx803,gfx900,gfx906,gfx1010 --distinct_host_configuration=false -c opt --apple_platform_type=macos --macos_minimum_os=10.9 --announce_rc --define open_source_build=true --define=no_kafka_support=true --define=no_ignite_support=true --define=grpc_no_ares=true --spawn_strategy=standalone --strategy=Genrule=standalone --enable_platform_specific_config --action_env CUDA_TOOLKIT_PATH=/usr/tce/packages/cuda/cuda-10.1.243 --action_env CUDNN_INSTALL_PATH=/usr/tce/packages/cudnn/cudnn-7.0-cuda-9.0/ --action_env TF_CUDNN_VERSION=7.0 | |
| INFO: Found applicable config definition build:short_logs in file /usr/WS2/zhou6/jax/.bazelrc: --output_filter=DONT_MATCH_ANYTHING | |
| INFO: Found applicable config definition build:cuda in file /usr/WS2/zhou6/jax/.bazelrc: --crosstool_top=@local_config_cuda//crosstool:toolchain --define=using_cuda=true --define=using_cuda_nvcc=true | |
| INFO: Found applicable config definition build:linux in file /usr/WS2/zhou6/jax/.bazelrc: --copt=-Wno-sign-compare --define=no_aws_support=true --define=no_gcp_support=true --define=no_hdfs_support=true --cxxopt=-std=c++14 --host_cxxopt=-std=c++14 --copt=-Wno-stringop-truncation | |
| Loading: | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Loading: 0 packages loaded | |
| Analyzing: target //build:build_wheel (1 packages loaded, 0 targets configured) | |
| DEBUG: Rule 'io_bazel_rules_docker' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1556410077 -0400" | |
| DEBUG: Repository io_bazel_rules_docker instantiated at: | |
| /usr/WS2/zhou6/jax/WORKSPACE:34:10: in <toplevel> | |
| /usr/WS2/zhou6/.cache/bazel/_bazel_zhou6/45ac699d106f4b2f92bdd647a162ed50/external/org_tensorflow/tensorflow/workspace0.bzl:65:34: in workspace | |
| /usr/WS2/zhou6/.cache/bazel/_bazel_zhou6/45ac699d106f4b2f92bdd647a162ed50/external/bazel_toolchains/repositories/repositories.bzl:37:23: in repositories | |
| Repository rule git_repository defined at: | |
| /usr/WS2/zhou6/.cache/bazel/_bazel_zhou6/45ac699d106f4b2f92bdd647a162ed50/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in <toplevel> | |
| Analyzing: target //build:build_wheel (14 packages loaded, 12 targets configured) | |
| Analyzing: target //build:build_wheel (14 packages loaded, 12 targets configured) | |
| Analyzing: target //build:build_wheel (14 packages loaded, 12 targets configured) | |
| Analyzing: target //build:build_wheel (159 packages loaded, 6231 targets configured) | |
| Analyzing: target //build:build_wheel (160 packages loaded, 6740 targets configured) | |
| WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/6abbba3fca9fdf8d31f74800a7ddb40b103ae6e3.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found | |
| INFO: Analyzed target //build:build_wheel (175 packages loaded, 13603 targets configured). | |
| INFO: Found 1 target... | |
| [0 / 53] [Prepa] BazelWorkspaceStatusAction stable-status.txt ... (11 actions, 0 running) | |
| ERROR: /usr/WS2/zhou6/.cache/bazel/_bazel_zhou6/45ac699d106f4b2f92bdd647a162ed50/external/com_google_absl/absl/base/BUILD.bazel:116:11: Compiling absl/base/dynamic_annotations.cc failed: undeclared inclusion(s) in rule '@com_google_absl//absl/base:dynamic_annotations': | |
| this rule is missing dependency declarations for the following files included by 'absl/base/dynamic_annotations.cc': | |
| '/usr/tce/packages/gcc/gcc-8.3.1/rh/include/c++/8/stdlib.h' | |
| '/usr/tce/packages/gcc/gcc-8.3.1/rh/include/c++/8/cstdlib' | |
| '/usr/tce/packages/gcc/gcc-8.3.1/rh/include/c++/8/ppc64le-redhat-linux/bits/c++config.h' | |
| '/usr/tce/packages/gcc/gcc-8.3.1/rh/include/c++/8/ppc64le-redhat-linux/bits/os_defines.h' | |
| '/usr/tce/packages/gcc/gcc-8.3.1/rh/include/c++/8/ppc64le-redhat-linux/bits/cpu_defines.h' | |
| '/usr/tce/packages/gcc/gcc-8.3.1/rh/lib/gcc/ppc64le-redhat-linux/8/include/stddef.h' | |
| '/usr/tce/packages/gcc/gcc-8.3.1/rh/include/c++/8/bits/std_abs.h' | |
| Target //build:build_wheel failed to build | |
| ERROR: /usr/WS2/zhou6/.cache/bazel/_bazel_zhou6/45ac699d106f4b2f92bdd647a162ed50/external/org_tensorflow/tensorflow/compiler/xla/python/BUILD:492:17 Middleman _middlemen/@org_Utensorflow_S_Stensorflow_Scompiler_Sxla_Spython_Cxla_Uextension.so-BazelCppSemantics_build_arch_ppc-opt failed: undeclared inclusion(s) in rule '@com_google_absl//absl/base:dynamic_annotations': | |
| this rule is missing dependency declarations for the following files included by 'absl/base/dynamic_annotations.cc': | |
| '/usr/tce/packages/gcc/gcc-8.3.1/rh/include/c++/8/stdlib.h' | |
| '/usr/tce/packages/gcc/gcc-8.3.1/rh/include/c++/8/cstdlib' | |
| '/usr/tce/packages/gcc/gcc-8.3.1/rh/include/c++/8/ppc64le-redhat-linux/bits/c++config.h' | |
| '/usr/tce/packages/gcc/gcc-8.3.1/rh/include/c++/8/ppc64le-redhat-linux/bits/os_defines.h' | |
| '/usr/tce/packages/gcc/gcc-8.3.1/rh/include/c++/8/ppc64le-redhat-linux/bits/cpu_defines.h' | |
| '/usr/tce/packages/gcc/gcc-8.3.1/rh/lib/gcc/ppc64le-redhat-linux/8/include/stddef.h' | |
| '/usr/tce/packages/gcc/gcc-8.3.1/rh/include/c++/8/bits/std_abs.h' | |
| INFO: Elapsed time: 1096.168s, Critical Path: 1.25s | |
| INFO: 162 processes: 147 internal, 15 local. | |
| FAILED: Build did NOT complete successfully | |
| ERROR: Build failed. Not running target | |
| FAILED: Build did NOT complete successfully | |
| Traceback (most recent call last): | |
| File "build/build.py", line 506, in <module> | |
| main() | |
| File "build/build.py", line 501, in main | |
| shell(command) | |
| File "build/build.py", line 51, in shell | |
| output = subprocess.check_output(cmd) | |
| File "/g/g90/zhou6/lassen-space/conda-ibm/envs/powerai/lib/python3.6/subprocess.py", line 356, in check_output | |
| **kwargs).stdout | |
| File "/g/g90/zhou6/lassen-space/conda-ibm/envs/powerai/lib/python3.6/subprocess.py", line 438, in run | |
| output=stdout, stderr=stderr) | |
| subprocess.CalledProcessError: Command '['/g/g90/zhou6/bin/bazel', 'run', '--verbose_failures=true', '--config=short_logs', '--config=cuda', '--define=xla_python_enable_gpu=true', ':build_wheel', '--', '--output_path=/usr/WS2/zhou6/jax/dist']' returned non-zero exit status 1. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment