> a single axis on which an ML system underperforms a frigging doctor
You do realize the salience of the fact that this isn't some arbitrary axis, right? It just so happens to be the axis that involves the doctor making the correct diagnostic. Which is, you know, the entire reason doctors are a thing.
> slightly
I wouldn't call a factor of two "slightly," but that's neither here nor there.
Are we reading the same paper? In the graph I'm looking at the axis where the model underperforms the doctor is labeled "No inaccurate/irrelevant information", which has nothing to do with making the correct diagnostic.
The three important axes "Answer supported by consensus",
"Possible harm extent = No harm" and
"Low likelihood of harm" it is performing really similarly to the doctors, probably similar to the graph a single middle of the pack doctor would have.
Are you reading a different graph or am I misunderstanding something about it?
You do realize the salience of the fact that this isn't some arbitrary axis, right? It just so happens to be the axis that involves the doctor making the correct diagnostic. Which is, you know, the entire reason doctors are a thing.
> slightly
I wouldn't call a factor of two "slightly," but that's neither here nor there.