Please do reconsider adding a visual indicator. Direct manipulation is a powerful UI technique and helps retain user focus. It could be something as simple as the mouse pointer changing when hovering over a component that can be drilled down into.
Hovering the navigation on the left could also highlight all linked components as well, giving a better sense of what you can drill into (without having to discover for yourself by moving your mouse all over the diagram).
