If we’re going down this road of theorizing about the human brain based on DNNs, what is the deal with dropout? Could we help human brains with generalization by randomly removing 10% of our newly created connections at the end of each day to improve long term learning? :)
That’s called synaptic pruning and, while it most happens as a human matures, there’s evidence indicating that it occurs during sleep in adults to help consolidate the most important connections and remove the unimportant ones. It’s not exactly like dropout, but at a high level it kind of looks like it.
