I'm becoming a fan of the tiny black hole theory. A lot of this seems to point toward a three-way tradeoff between distance, albedo and mass to try to find a sweet spot that explains how something massive enough or close enough could have avoided observation. A small black hole at a pretty significant distance is a rather tidy answer.
We're already pretty lucky, if you think about the whole goldilocks thing. Maybe every other species in a habitable system has a little black hole on the edge too.
Since finding it pretty much means being able to see it, would we ever be able to find a tiny black hole? I would expect its size to be sub-kilometer and if it's at its aphelion to be pretty much invisible.
Like another poster said - just chuck a few rocks at it and see which way they curve. That nifty centrifugal launcher would be great for throwing some birdshot in that direction.