Thanks.

It's interesting how the state of the art is outpacing publishing.

From a quick scan that appears quite similar to the approach in papers like "Parsing Natural Scenes and Natural Language with Recursive Neural Networks" (2011)[1]. Edit: I see they cite this paper too.

[1] http://nlp.stanford.edu/pubs/SocherLinNgManning_ICML2011.pdf

There's also better embeddings than word2vec now:

http://nlp.stanford.edu/projects/glove/

The characterisation of Glove as better than Word2Vec is controversial. I'm on mobile now, but one of the word2vec authors had a Google doc going through the claims, and pointing out that similar performance was possible from word2vec by changing the parameters word2vec is used with.

Edit: a link about this. https://news.ycombinator.com/item?id=8660624

Speaking from personal experience. I get paid to do deep learning. One of skymind's biggest app areas is text.

That being said: I will be benchmarking deeplearning4j's glove with word2vec here soon. Any machine learning algorithm is better when you tune it.

I personally like glove due to having less knobs. The mechanics involving document statistics being part of the gradient update is also interesting.

I've also messed quite a bit with the distributed representations.

I'm not partial to any particular implementation. I'll use what works. That being said, I'm not armchair. I'll be backing this up with my own data as well.

I'm aware the tone was a little condescending and I won't take back what I said. I will actually back it up though ;).

My main point is just because something is controversial shouldn't stop you from trying it. That's what research is: trying new things.