That's how an orbit works. If you think about a low orbit around the Earth, you are constantly falling towards Earth and missing, because you are going so fast around it. Likewise, the Earth is constantly falling towards the Sun, but because it is going sideways so fast it keeps going around instead. You say "aim towards sol" - but in order to do that you effectively need to stop going sideways. Once you have done that, it doesn't matter whether or not you are also travelling towards the Sun - you will be soon. That's what we mean when we say that in order to hit the Sun you need to slow down.
This assumes there is no planet or moon handy to whip around, to end up going in a completely different direction, with no extra energy expenditure.
It is tricky (but possible, with cleverness and a careful schedule) to gain or lose energy this way, but it doesn't matter. If your closest approach is well within the sun's photosphere, it doesn't matter how fast you're going when you get there. So, you can do it with essentially zero delta-v, starting and ending with the same total energy as an object would have co-orbiting with earth, but on an extremely eccentric orbit.
It's not terribly rare (on a geological timeline, at least) for comets to dispose of themselves this way.
Anyway, what is so great about dropping them in the sun? Jupiter swallows comets frequently. Mars is a squalid dump, and so is Venus, at least below the clouds.
Great explanation. I think many people have the wrong default intuition for what an orbit is. I don't think they realize that it means going so fast that you fall perpetually around an object rather than just eventually hitting the object.