Correlated errors. Naive averaging will lead to overconfidence and it is not trivial to model the correlation. Boosting is worth a shot though.

My point was that ensembles of deep neural networks are commonly used and yield higher accuracies.

More importantly, what happens if you put a radiologist opinion (or multiple) in such an ensemble?