Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save AndreasMadsen/94f5100aff697cdf5ff6c26f90a6dad7 to your computer and use it in GitHub Desktop.
Save AndreasMadsen/94f5100aff697cdf5ff6c26f90a6dad7 to your computer and use it in GitHub Desktop.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/common_runtime/bfc_allocator.cc:274] ****************************************************************************************************
W tensorflow/core/common_runtime/bfc_allocator.cc:275] Ran out of memory trying to allocate 148.09MiB. See logs for memory state.
W tensorflow/core/framework/op_kernel.cc:993] Resource exhausted: OOM when allocating tensor with shape[2560,892,1,17]
Traceback (most recent call last):
File "/zhome/ff/2/77654/stdpy3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1022, in _do_call
return fn(*args)
File "/zhome/ff/2/77654/stdpy3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1004, in _run_fn
status, run_metadata)
File "/appl/python/3.6.0/lib/python3.6/contextlib.py", line 89, in __exit__
next(self.gen)
File "/zhome/ff/2/77654/stdpy3/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 469, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.ResourceExhaustedError: OOM when allocating tensor with shape[2560,892,1,17]
[[Node: aconv1d_32/Conv2D = Conv2D[T=DT_FLOAT, data_format="NHWC", padding="VALID", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/gpu:0"](aconv1d_32/ExpandDims, aconv1d_32/ExpandDims_1)]]
[[Node: Mean/_29 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_128_Mean", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "atorus.py", line 54, in <module>
data: np.random.randint(0, 128, size=(160, 200))
File "/zhome/ff/2/77654/stdpy3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 767, in run
run_metadata_ptr)
File "/zhome/ff/2/77654/stdpy3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 965, in _run
feed_dict_string, options, run_metadata)
File "/zhome/ff/2/77654/stdpy3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1015, in _do_run
target_list, options, run_metadata)
File "/zhome/ff/2/77654/stdpy3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1035, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.ResourceExhaustedError: OOM when allocating tensor with shape[2560,892,1,17]
[[Node: aconv1d_32/Conv2D = Conv2D[T=DT_FLOAT, data_format="NHWC", padding="VALID", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/gpu:0"](aconv1d_32/ExpandDims, aconv1d_32/ExpandDims_1)]]
[[Node: Mean/_29 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_128_Mean", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
Caused by op 'aconv1d_32/Conv2D', defined at:
File "atorus.py", line 32, in <module>
name='aconv1d')
File "/zhome/ff/2/77654/stdpy3/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 639, in convolution
op=op)
File "/zhome/ff/2/77654/stdpy3/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 421, in with_space_to_batch
result = op(input_converted, num_spatial_dims, "VALID")
File "/zhome/ff/2/77654/stdpy3/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 631, in op
name=name)
File "/zhome/ff/2/77654/stdpy3/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 114, in _non_atrous_convolution
name=scope)
File "/zhome/ff/2/77654/stdpy3/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 2034, in conv1d
data_format=data_format)
File "/zhome/ff/2/77654/stdpy3/lib/python3.6/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 396, in conv2d
data_format=data_format, name=name)
File "/zhome/ff/2/77654/stdpy3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
op_def=op_def)
File "/zhome/ff/2/77654/stdpy3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2395, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/zhome/ff/2/77654/stdpy3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1264, in __init__
self._traceback = _extract_stack()
ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[2560,892,1,17]
[[Node: aconv1d_32/Conv2D = Conv2D[T=DT_FLOAT, data_format="NHWC", padding="VALID", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/gpu:0"](aconv1d_32/ExpandDims, aconv1d_32/ExpandDims_1)]]
[[Node: Mean/_29 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_128_Mean", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
(
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment