log

2020-02-16 05:04:45.108469: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer.so.6
2020-02-16 05:04:45.110358: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer_plugin.so.6
2020-02-16 05:04:46.099043: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-02-16 05:04:46.121264: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:
pciBusID: 0000:03:00.0 name: Quadro GP100 computeCapability: 6.0
coreClock: 1.4425GHz coreCount: 56 deviceMemorySize: 15.90GiB deviceMemoryBandwidth: 681.88GiB/s
2020-02-16 05:04:46.121315: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-02-16 05:04:46.121350: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-02-16 05:04:46.123657: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-02-16 05:04:46.123998: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-02-16 05:04:46.126524: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-02-16 05:04:46.127760: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-02-16 05:04:46.127799: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-02-16 05:04:46.129805: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
2020-02-16 05:04:46.210919: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-02-16 05:04:46.246086: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2594180000 Hz
2020-02-16 05:04:46.253659: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x556f59fa3150 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-02-16 05:04:46.253694: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-02-16 05:04:46.481808: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x556f59ccb940 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-02-16 05:04:46.481852: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Quadro GP100, Compute Capability 6.0
2020-02-16 05:04:46.484417: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:
pciBusID: 0000:03:00.0 name: Quadro GP100 computeCapability: 6.0
coreClock: 1.4425GHz coreCount: 56 deviceMemorySize: 15.90GiB deviceMemoryBandwidth: 681.88GiB/s
2020-02-16 05:04:46.484487: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-02-16 05:04:46.484506: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-02-16 05:04:46.484550: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-02-16 05:04:46.484569: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-02-16 05:04:46.484586: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-02-16 05:04:46.484614: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-02-16 05:04:46.484629: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-02-16 05:04:46.486591: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
2020-02-16 05:04:46.486635: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-02-16 05:04:46.837719: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1096] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-02-16 05:04:46.837808: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102]      0
2020-02-16 05:04:46.837844: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] 0:   N
2020-02-16 05:04:46.841272: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1241] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 15163 MB memory) -> physical GPU (device: 0, name: Quadro GP100, pci bus id: 0000:03:00.0, compute capability: 6.0)
Epoch 1/10
2020-02-16 05:04:54.035565: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-02-16 05:04:54.795808: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
    250/Unknown - 31s 125ms/step - loss: 0.5060 - accuracy: 0.73962020-02-16 05:05:20.382006: W tensorflow/core/common_runtime/base_collective_executor.cc:217] BaseCollectiveExecutor::StartAbort Out of range: End of sequence
         [[{{node IteratorGetNext}}]]
         [[Fill/_28]]
2020-02-16 05:05:20.382094: W tensorflow/core/common_runtime/base_collective_executor.cc:217] BaseCollectiveExecutor::StartAbort Out of range: End of sequence
         [[{{node IteratorGetNext}}]]
250/250 [==============================] - 34s 136ms/step - loss: 0.5060 - accuracy: 0.7396 - val_loss: 0.4043 - val_accuracy: 0.8165
Epoch 2/10
250/250 [==============================] - 25s 101ms/step - loss: 0.3106 - accuracy: 0.8752 - val_loss: 0.3517 - val_accuracy: 0.8540
Epoch 3/10
250/250 [==============================] - 26s 103ms/step - loss: 0.2411 - accuracy: 0.9077 - val_loss: 0.3313 - val_accuracy: 0.8660
Epoch 4/10
250/250 [==============================] - 26s 103ms/step - loss: 0.2002 - accuracy: 0.9268 - val_loss: 0.3661 - val_accuracy: 0.8510
Epoch 5/10
250/250 [==============================] - 26s 103ms/step - loss: 0.1960 - accuracy: 0.9260 - val_loss: 0.3559 - val_accuracy: 0.8590
Epoch 6/10
250/250 [==============================] - 26s 103ms/step - loss: 0.1797 - accuracy: 0.9345 - val_loss: 0.4077 - val_accuracy: 0.8475
Epoch 7/10
250/250 [==============================] - 26s 103ms/step - loss: 0.1460 - accuracy: 0.9481 - val_loss: 0.4638 - val_accuracy: 0.8365
Epoch 8/10
250/250 [==============================] - 26s 104ms/step - loss: 0.1211 - accuracy: 0.9590 - val_loss: 0.4925 - val_accuracy: 0.8405
Epoch 9/10
250/250 [==============================] - 26s 104ms/step - loss: 0.0952 - accuracy: 0.9701 - val_loss: 0.4847 - val_accuracy: 0.8555
Epoch 10/10
250/250 [==============================] - 26s 103ms/step - loss: 0.0863 - accuracy: 0.9731 - val_loss: 0.4940 - val_accuracy: 0.8445