Implementing a CNN for Text Classification.ipynb

implementing-a-cnn-for-text-classification.ipynb

This is the source code used to compare word2vec initialization versus not.

In the training section, comment or not the following block of code to have word2vec initialization or not:

      # 
      # Use of pre-trained word2vec
      #
      initW = None
      print("Load word2vec at {}".format(FLAGS.pre_trained_w2v))
      initW = load_pre_tained_word2vec(voc_dict, FLAGS.pre_trained_w2v)
      sess.run(cnn.W.assign(initW))

The experiments show that using word2vec initialization or not gives the same result.

The following shows accuracy over epochs on 3 runs. One with random initialization of embedding, another with word2vec initialization and another with word2vec initialization but without further learning of embedding parameters:

The one which grows less fast is with the random initialization.

Here is the loss and corresponding colors: