It's prosody, rhythm, accent, word choice, and so on.
By the time you've mastered all those, you're practically halfway to becoming a professional voiceover artist.
Remember, trained voiceover artists have been mimicking voices for a long, long time. Their timbre isn't always perfect, but faking voices doesn't need deepfakes.
If you're looking for it sure. But I'm willing to bet existing technology is sufficient to catch an awful lot of people off guard. Hearing a familiar voice is usually quite disarming.
and Facebook is currently generating billions of voice profiles based on recorded WhatsApp sessions. Soon these profiles are sold to advertising agencies.
With modern voice conversion technology it is perfectly possible actually.