Look at the dates of the papers I linked. The quote:

Approaches based on cheaper monocular or stereo imagery data have, until now, resulted in drastically lower accuracies --- a gap that is commonly attributed to poor image-based depth estimation. However, in this paper we argue that data representation (rather than its quality) accounts for the majority of the difference.

Is from December 2018. Your knowledge of what smart people think is currently outdated, because the new results are so recent.

I also agree that Musk was making claims about no need for LIDAR based on his intuition. He could have been terribly wrong, because it was just his intuition. But he is not wrong.

BTW - Smart people were wrong about AI many times. They were wrong about ANN being dead end in eighties, they were wrong about symbolic AI. They were also wrong about necessity of LIDAR. In a year of two they will update their believes as more evidence will be piling up.

Because the question isn't about what a theoretical best system consists of. No one (including Musk) denies more sensors could theoretically be beneficial. Adding a gravitational wave detector to your car would ensure the car doesn't accidentally drive into black holes, making it more safe. This additional safety would not be economical.

Same principle with LIDAR. Adding $5k hardware per car and a ton of R&D doesn't seem economical to Tesla, since they think they can get pretty much the same with just their current sensor suit. Time will tell if they are right and what AP + sensor suite customers will choose.

I've mentioned this before but, how many technologies do you see companies working on where they say, "well we have one approach we think is gonna work, but we're also going to spend a ton of resources and time developing this other solution that we don't think is going to work, that would be useless if the first approach does work. Oh and it would make our cars 5k more expensive at least."