I think you might both be right, but never agree with each other, due to the trickiness of language. That is to say, depending on your definition of "genius" (and depending on the still not fully understood nature of genius), you might on the one hand agree that he qualifies as being a "genius", but isn't "one of the most brilliant men alive."

Setting aside any purely semantic disagreements (for example the fact that "one of the most brilliant men alive" is highly dependent on the number of other brilliant men around), I think part of what OP is arguing against is the implication that most people attribute a specific kind of genius to him (and OP should definitely step in to disagree with me here if I am mistaken). For example, most people may agree that Warren Buffet is a genius, but it's clear (to me at least), that they mean genius in a very different way in that context than when applied to Musk. The implication seems to strongly be that Musk is not merely a savvy businessman, but a "brilliant scientist" or "brilliant engineer" or something as well, something I think no one would assign to Buffet. Buffet bought Duracell, but no one believes he is key to battery technology. For the record, I am not arguing about whether Musk is or is not this kind of genius, just that Musk is a unique case in that the conversation exists on this axis, arguably much more so than even with Steve Jobs (who most people at best attribute "design genius" to, but will readily admit is not an "engineering genius", and in fact may even attribute the ability to see "past that" as one of his strengths). So I think the frustration I detect in this argument is that Musk seems to often be imagined in the same ranks as the actual Nikola Tesla perhaps, which they feel is unearned, and then the defense given is more appropriate for an Edison or Ford-esque "businessy crossover genius", vs. a description of direct technical accomplishments. Again, I am not arguing either way as to where he should be placed, just pointing out that I see a lot of this sort of "talking past each other", since "genius" colloquially implies something that doesn't make "Steve Jobs" immediately come to mind, vs. Einstein.

Indeed. But it’s well documented that both Jobs and Edison were shrewd businessmen first, rather than technical geniuses.

Geniuses at self-promotion and manipulating others, certainly. A world full of Jobses would be an unlivable dystopia. A world full of Wozniaks, now that would be interesting.