How to restore CUDNNLSTM of TensorFlow at CPU device? So that it could be used in GPU, CPU or Mobile

check_model_at_cpy.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_layers = 1\n",
    "num_units = 128\n",
    "direction = \"bidirectional\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from cudnn_cpu_bi_trans/model_1\n"
     ]
    }
   ],
   "source": [
    "inputs    = tf.placeholder(tf.float32, [None, None, 32], name=\"inputs\")\n",
    "with tf.variable_scope(\"cudnn_lstm\"):\n",
    "    single_cell = lambda: tf.contrib.cudnn_rnn.CudnnCompatibleLSTMCell(num_units)\n",
    "    cells_fw = [single_cell() for _ in range(num_layers)]\n",
    "    cells_bw = [single_cell() for _ in range(num_layers)]\n",
    "    # Leave the scope arg unset.\n",
    "    (outputs, output_state_fw,\n",
    "    output_state_bw) = tf.contrib.rnn.stack_bidirectional_dynamic_rnn(cells_fw, cells_bw, inputs,dtype=tf.float32)\n",
    "\n",
    "    saver = tf.train.Saver()\n",
    "# Create session\n",
    "sess = tf.Session()\n",
    "# Restores\n",
    "saver.restore(sess, 'cudnn_cpu_bi_trans/model_1')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "out = sess.run(outputs, {inputs: np.zeros([10,100,32])})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10, 100, 256)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out.su"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python (tenosflow)",
   "language": "python",
   "name": "tensorflow"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}

restore_cudnn.ipynb

restore_cudnn.ipynb show how to save the model which use cudnn and then is restored using CudnnCompatibleLSTMCell (but not only GPU) cell.

Thank you for your tutorial!
However, after restoring cudnn_LSTM from checkpoint, I cant restore other variables from meta graph using something like this:

saver = tf.train.import_meta_graph("{}.meta".format(checkpoint))
saver.restore(sess, checkpoint)

That's to say, if I restore cudnn_LSTM in your way, then I cant restore other variables from meta graph.
And I need to get some variables like placeholder, accuracy for inference.

Can you tell me how to save and restore properly in this situation?
BTW, how can I restore cudnn variables to GPU directly? After googling a lot, I found that people just care about restoring cudnn variables to CPU, but I cant even restore it to GPU. Am I missing something? THANKS!

Hi, I'm running into troubles when restore into CudnnCompatibleLSTMCell from CudnnLSTM, I've run the code above on my machine, and it raised an error, so I'm wandering if I'm using the version that do not support this feature. Would you please provide your tensorflow version that can run the code above? I'm currently run on Tensorflow 1.5.

@SysuJayce I solved it, you should rebuild model and restore ckpt instead of using import_meta_graph

for anyone who had the same issue I did, remember to set time_major=True in stack_bidirectional_dynamic_rnn().

The CudnnLSTM() function is time major, as opposed to batch major. So remember to transpose your inputs and outputs (swap dimensions 0 and 1)