You may want to verify GPU is working with Tensorflow in runtime. Here are some useful python code snippets:
## show all devices available ##
from tensorflow.python.client import device_lib
print(device_lib.list_local_devices())
To make sure GPU works in computation:
import tensorflow as tf