The article's point is broader than that: the entire idea of physiognomy, "the practice of using people’s outer appearance to infer inner character", is not scientifically defensible. There's no mechanism that would cause someone's inner character to be reflected in their appearance in any consistent way. This remains true even if it's a computer algorithm, not a person, making judgements based on people's appearances.
"There's no mechanism that would cause someone's inner character to be reflected in their appearance in any consistent way." That's a pretty big leap. Down Syndrome has consistent physical characteristics that correlate with a particular set of cognitive/behavioral characteristics. While it is important to carefully critique scientific findings that may be motivated by political biases, it is also important to give science as a process the chance to find truths even if we might not like their political implications.
Sure, that's true, and I bet a Down-Syndrome-recognizing neural net could be made quite accurate. I guess what I mean to say is that there's no general mechanism that would cause someone's inner character to be reflected in their appearance in any consistent way. If you want to predict behavior from appearance, you have to actually do the science and prove that there's an underlying mechanism before you can trust that your neural net is recognizing something meaningful.
Since Kepler, science is not about mechanism but about prediction. There is often a rather fetishistic disdain of mechanism, that gets relegated to philosophical, coffee-table curiosities. See, for example, the various mechanisms that explain Newtonian and relativistic mechanics. Have you heard about them? Probably not; nobody cares, as the theories allow to make all the predictions than you need, and you do not need anything else.
The fact that there is no mechanism to explain a phenomenon that you can predict is not a problem. It may be a shortcoming of our understanding, but not in any case "scientifically indefensible", as you claim.