The only issue I have with the voice in my head is that it makes me read slowly, because I have to imagine the sound of the word instead of its meaning only.
This was the actual advice I was given when proof-reading text I had written. Reading it in your head allows your brain to auto-correct issues in the text. Reading it out loud is supposed to slow you down to "disable" the auto-correct. It's not perfect, but I definitely found and could correct more of my mistakes before submitting the text.
I stopped reading novels for a long time because of this. Focused on the voice reading the audio of the words, while ingesting little of the content's meaning.
I've intentionally practiced reading aloud lately to counter it. (only sometimes, generally with someone else to read to; it helps)
I can actually read in both modes: With a voice in my head, complete with sound and intonation (nice for poetry), and without the voice, which is much faster.
I guess one can train yourself to "switch", e.g. by focusing on the next word faster than the voice can follow.