The house is recouped by society though inheritance and property tax. You keep the physical object, but it has to be continuously paid for by generations to remain private.

If keeping things copyrighted had a continuous cost then copyright holders would be incentivized to release things to public sooner.