#What is Word Embedding I say Word2Vec Word Embedding converts natural language into a vector that computers can understand. It can make 'looks like' a computer understands natural language. It can measure similarity of between words. It makes it easier to handle by vectoring words. It can make inferences through vector operations. Because the meaning of word itself is digitized as a vector. It makes reduce memory than one hot encoding.
Word2vec is a group of related models that are used to produce word embeddings. Word2Vec is a way to vector the meaning of a word to reflect similarities between words.
There ara two ways of Word2Vec. One is CBOW(Continuous Bag of Words) and the other is Skip-gram.
CBOW predicts center word from context words.
We input one hot encoding of context words to input layer of word2vec It outputs one hot encoding of center word. There is a difference that CBOW needs to average about vectors after projection. But Skip-gram's input is one, so it does not need to.
We can decide around word's count. This range is called a 'window'. If we have set the size of the window, we can make a data set to changing center word and context words.
for example)
- Sentece is "I want to go market because I want to eat fresh fruits."
- window is 2
Picture is a total dataset for CBOW.
This picture shows artificial neural network of CBOW. Context words input to input layer. Center word outputs from output layer. So Word2Vec needs center word's one-hor vector.
CBOW get a average that about context words multiply with weight. Because inputs count is big than 2 but output count is only 1.
Skip-gram predicts context words from center word.
We input one hot encoding of center word to input layer of word2vec. It outputs one hot encoding of surround words.
This picture shows artificial neural network of Skip-gram. It has one Input so there isn't a process getting average about inputs.
Skip-gram is known to perform better than CBOW.