That 20 watts is to run the network. Our brain has had a billion years to work out details of the architecture and encode a lot of basic stuff as instinct (and it still sucks at a lot of things). You should be counting that energy cost as well - we didnt get from nerve nets to frontal lobes overnight.
Earth is about 4.5B years old, life is about 3.7B years old, multicellular life (including life with neural nets) is about 600 million years old. I don't think the span from microbe to multicellular organism counts in brain evolution.