They are getting a 78x speedup w/o hardware support which is pretty good: they think they can speed it up another 4x if they had the right hardware support. So it looks useful now with possibility to get better.
So long as I've been involved with neural networks for text analysis it's seemed to me that we really should be using sparse activations because any particular document only involves a limited set of concepts.
For instance a search engine for patents might be looking at a patent for adhesive tape which activates a certain set of concepts but is not going to activate concepts involved with bicycle derailleurs or public key cryptography: a sparse representation reflects this and dense representations don't.
So long as I've been involved with neural networks for text analysis it's seemed to me that we really should be using sparse activations because any particular document only involves a limited set of concepts.
For instance a search engine for patents might be looking at a patent for adhesive tape which activates a certain set of concepts but is not going to activate concepts involved with bicycle derailleurs or public key cryptography: a sparse representation reflects this and dense representations don't.