Basically I agree with everything you say, with one exception. There are many methods to refine gold, but none of them are particularly safe, involving, for example, the evaporation of substantial amounts of mercury, gold cyanidation (what Turing's mother believed killed him), chlorine gas, or boiling aqua regia. This kind of thing can be made safe, but it's even more difficult than making mercury processing safe.

So I suspect that the main issue is storage and transportation, not refinement.