neural network embedding的作用[1]：

• Finding nearest neighbors in the embedding space. These can be used to make recommendations based on user interests or cluster categories.
• As input to a machine learning model for a supervised task.
• For visualization of concepts and relations between categories.

翻譯過來就是：

• 在embedding space中發(fā)現(xiàn)模式，比如基于embedding做聚類實現(xiàn)推薦功能
• 作為監(jiān)督學(xué)習(xí)任務(wù)的輸入
• 可視化類別間的概念和相關(guān)性

neural network embedding 的優(yōu)勢

neural network embedding解決了one-hot embedding存在的兩個不足，假設(shè)我們想要得到3w本書的embedding以向量代表書的一些信息：

• 隨著類別的增多，one-hot embedding的維度增加，比如有3w本書，就要設(shè)置3w維的向量
• one-hot embedding之間，不同category的相似度都為0，這很尷尬，比如同一個作者寫的兩本書之間的相似度是0，大量信息都被丟失了。

基于這兩個問題，我們可以得出一個結(jié)論：對于category variables, 理想的embdedding至少需要滿足兩個條件：

• 表示的向量維度要遠(yuǎn)小于類別的數(shù)量，比如3w本書，那么表示他們的向量就應(yīng)該遠(yuǎn)小于3w
• 相似的類別間應(yīng)該可以通過embedding進行相似度的計算
  參考資料[1]中，寫的相當(dāng)好。

the ideal solution for representing categorical variables would require fewer numbers than the number of unique categories and would place similar categories closer to one another.

Also like word vectors, entity embeddings can be expected to learn the intrinsic properties of the categories and group similar categories together.

category embedding的訓(xùn)練

可以通過一個有監(jiān)督的神經(jīng)網(wǎng)絡(luò)模型去訓(xùn)練category embedding，過程類似于詞向量的學(xué)習(xí)。
假設(shè)我們的任務(wù)是看每本書是否是列夫托爾斯泰的書，得出的使分類任務(wù)loss最小的weights就可以作為每本書的信息，這些信息中包含了一本書是否是列夫托爾斯泰的書相關(guān)的信息。以這個embedding為基礎(chǔ)，列夫托爾斯泰的書之間embedding的相似度會很高。（我的問題：那詞向量訓(xùn)練出來后，可以表示什么信息呢？具體一點）

結(jié)果的可視化

TSNE

(TSNE is a manifold learning technique which means that it tries to map high-dimensional data to a lower-dimensional manifold, creating an embedding that attempts to maintain local structure within the data. It’s almost exclusively used for visualization because the output is stochastic and it does not support transforming new data. An up and coming alternative is Uniform Manifold Approximation and Projection, UMAP, which is much faster and does support transform new data into the embedding space).

交互式的可視化

The problem with static graphs is that we can’t really explore the data and investigate groupings or relationships between variables. To solve this problem, TensorFlow developed projector, an online application that lets us visualize and interact with embeddings. I’ll release an article on how to use this tool shortly, but for now, here’s the results:

參考資料：

[1] Neural Network Embeddings Explained —— https://towardsdatascience.com
[2] github項目地址