... so here's the thing (speaking as someone who has taken many of these tests, been in admissions committees, and researches these tests). Take learning disabilities, and now expand it out to consider the similar factors, but with things like cultural background, life circumstances, and so forth and so on. Learning disabilities are not the same as those other things -- I don't want to equate them -- but they share some of the same issues with the test being normed on a certain standard population, and once you get outside of that, weird things happen.
Those density gradient plots in the linked blog post are interesting and useful to think about, but they're kinda hiding the fact that the vast majority of the data going into them is based on test-takers with a certain standard background, characteristics, and so forth. That in turn shapes the contours of what is error in those plots and analyses. The problem isn't necessarily that the tests are useless per se, it's that it's hard to interpret them in a way that accommodates people in nonstandard situations, or even accommodates the idiosyncracies of differences between people in general, differences that wouldn't matter in the real world.
The errors of using ACT-only, GPA-only, and so forth in the post is pretty interesting, but the author is missing the fact that that table has never really been the point of contention. The point of contention is whether that table's patterns apply equally across divisions of gender, race, SES background, age, disability status, and so forth and so on. They raise the point that the improvement in error from switching from ACT to GPA is comparable to (or better than) adding ACT to GPA, but isn't the real question whether doing so increases certain types of "predictable error variance", in the sense that you could predict the residual from things like SES, race, and so forth and so on?
I think GPA is seen as more acceptable than ACT because it's exchangeable in an important sense with the criterion being predicted. That is, if you want to know college GPA, maybe secondary school GPA is a little flawed, but at least it's ostensibly similar in terms of what it actually is. I think people have a sense that, say, you aren't using standardized tests as the criterion for college graduation, so why use it for prediction? Why have college and college GPA at all? Why not just let people take standardized tests and skip the whole degree program thing? There's reasonable arguments for doing that, but also reasonable arguments for not doing it, and many of those are the same arguments for and against using the test for admissions.
Those density gradient plots in the linked blog post are interesting and useful to think about, but they're kinda hiding the fact that the vast majority of the data going into them is based on test-takers with a certain standard background, characteristics, and so forth. That in turn shapes the contours of what is error in those plots and analyses. The problem isn't necessarily that the tests are useless per se, it's that it's hard to interpret them in a way that accommodates people in nonstandard situations, or even accommodates the idiosyncracies of differences between people in general, differences that wouldn't matter in the real world.
The errors of using ACT-only, GPA-only, and so forth in the post is pretty interesting, but the author is missing the fact that that table has never really been the point of contention. The point of contention is whether that table's patterns apply equally across divisions of gender, race, SES background, age, disability status, and so forth and so on. They raise the point that the improvement in error from switching from ACT to GPA is comparable to (or better than) adding ACT to GPA, but isn't the real question whether doing so increases certain types of "predictable error variance", in the sense that you could predict the residual from things like SES, race, and so forth and so on?
I think GPA is seen as more acceptable than ACT because it's exchangeable in an important sense with the criterion being predicted. That is, if you want to know college GPA, maybe secondary school GPA is a little flawed, but at least it's ostensibly similar in terms of what it actually is. I think people have a sense that, say, you aren't using standardized tests as the criterion for college graduation, so why use it for prediction? Why have college and college GPA at all? Why not just let people take standardized tests and skip the whole degree program thing? There's reasonable arguments for doing that, but also reasonable arguments for not doing it, and many of those are the same arguments for and against using the test for admissions.