Yep - also we have a large tendency to use feedforward NNs right now, but I have a sneaking suspicion that the future lies in something closer to recurrent NNs. Or probably something more complex, like automata-ish properties (IIRC there is also some NN that uses Turing-like devices).
Hopfield nets for example provide associative memory.
It may not all be groundbreakingly efficient, but very worthwhile.