计算机科学
作者:Ravid Schwartz-Ziv,Naftali Tishby发布时间:2019-11-06 最后修改:2019-11-06
摘要:尝试使用信息论解释神经网络
作者:Naftali Tishby, Fernando C. Pereira, and William Bialek发布时间:2019-11-06 最后修改:2019-11-06
摘要:信息瓶颈原理
作者:Sanjeev Arora, Yingyu Liang, Tengyu Ma发布时间:2019-09-15 最后修改:2019-09-15
摘要:The success of neural network methods for computing word embeddings has motivated methods for generating semantic embeddings of longer pieces of text, such as sentences and paragraphs. urprisingly, Wieting et al (ICLR’16) showed that such complicated methods are outperformed, especially in out-of-domain (transfer learning) settings, by simpler methods involving mild retraining of word embeddings and basic linear regression. The method of Wieting et al. requires retraining with a substantial labeled dataset such as Paraphrase Database (Ganitkevitch et al., 2013). The current paper goes further, showing that the following completely unsupervised sentence embedding is a formidable baseline: Use word embeddings computed using one of the popular methods on unlabeled corpus like Wikipedia, represent the sentence by a weighted average of the word vectors, and then modify them a bit using PCA/SVD. This weighting improves performance by about 10% to 30% in textual similarity tasks, and beats sophisticated supervised methods including RNN’s and LSTM’s. It even improves Wieting et al.’s embeddings. This simple method should be used as the baseline to beat in future, especially when labeled training data is scarce or nonexistent. The paper also gives a theoretical explanation of the success of the above unsupervised method using a latent variable generative model for sentences, which is a simple extension of the model in Arora et al. (TACL’16) with new “smoothing” terms that allow for words occurring out of context, as well as high probabilities for words like and, not in all contexts.
作者:A. M. Turing发布时间:2019-07-04 最后修改:2019-07-04
摘要:图灵的Imitation Game论文
其他
作者:陈皓发布时间:2019-03-12 最后修改:2019-03-12
摘要:如果10年前我看到这样的文章该有多好。 前提是我能有现在的领悟,所以能看得进去的话。 不过我10年前刚毕业时候真的是弱爆了,现在想想那个时候自己的懒惰和不思进取,即使看到了这篇文章,很可能也不会用心去真正看懂。
召唤蕾姆