The short answer is that there are many entrenched interests who are not keen on the idea: Uranium mines, nuclear industry, weapons manufacturers, not to mention the oil, gas and coal lobbies, even the alternative energy suppliers like solar. Thorium would rain on all their parades.
The real answer is the R&D costs are not worth it because Uranium is still extreamly cheap. The fuel costs for a modern reactor are a small fraction of total cost.
Also, fosil fuels a not going to be replaced with wind, solar, fission or fusion any time soon because they are not really portable for anything smaller than a boat.
That "waste" is still perfectly usable fuel. You could extract the plutonium and uranium and use that to make more fuel rods. You could burn it in fast breeder reactors, in a hypothetical future where mined uranium becomes expensive enough to warrant the use of fast breeders. Or you could just ship it to Canada, where they have heavy-water-moderated reactors that can use regular nuclear waste directly as fuel, without any chemical reprocessing.
I keep saying that nuclear waste storage shouldn't be thought of as permanent disposal. It should be thought of as keeping a reserve of nuclear fuel for the future.
No conspiracy theory necessary. There are only a small number of people who can design and build things of nuclear reactor level complexity from scratch and they mostly work at government labs and for major engineering firms. The government wanted these people to focus their attention on the designs that would be useful for the cold war weapons programs so the thorium reactors just didn't get built.
Exactly. Mainly, the current nuclear power industry, i.e. Areva, Westinghouse and GE make all their money on the fabrication fo fuel rod assemblies. They tried to make thorium fuel rods work in Indian Point Unit 1, but the layout of the rod bundles was tough to optimize to allow fertile material to become fissile and cycle correctly to be efficient at heating up the primary coolant.
Add this to all the regulatory hurdles and the big three nuke companies see no reason to pursue such a financial risk.