> And if I drill a hole and collect 500 watts from it on a 100 sqm plot, am I effectively siphoning the heat from my neighbors plots?

Pretty much. But the Earth's crust has a lot of thermal mass, so there's enough energy stored there to last for a long while.