I see zero discussion of test scores and people with learning disabilities. Those who cannot be properly accommodated will be unfairly represented and often have to put up with a lot more barriers (such as frequently messed up accommodations on test day leading to anxiety). Usually these accommodations help normalize the specific testing scenario and people with disabilities do not need as drastic accommodations in real job setups. The tests are designed for the average test taker but have lots of hiccups for those that deviate from the average.
It’s a difficult problem to solve, but it’s unfair if a student gets easily distracted or anxious when taking a test around others in a time constrained setting. Those types of external factors generally shouldn’t be represented in a student’s test results yet often are. If tests are to be a good indicator for all students, I think accommodations have a ways to go.
If tests are indeed a generally good indicator of a student’s success and are continued to be used in such a manner, I feel it’s important that the experience is fair for all types of test takers. Many US tests such as the CPA exam do a great job of handling accommodations. However, many others (like the PSAT) have a lot of room for improvement.
> I see zero discussion of test scores and people with learning disabilities. Those who cannot be properly accommodated will be unfairly represented…
I have long wondered how these interact / can support people who are a couple of sigma out from what we have decided is the “norm”. I can see how additional test time can help someone with anxiety or distraction issues, which can help someone with other strengths shine.
But how can this work outside the academy? A trial lawyer has only so much time to work on the case before it goes to trial or before some response to a filing is required. An engineer building a rocket still has to get something designed before assembly. Arent such time accommodations are hard to implement in the “real world”?
There are nice counter examples to my question. For example dyslexic people struggle with all sorts of cases, yet technology can help many of them (e.g. new kinds of fonts), and not just current students but people already in the work force.
I can also see the school accommodation helping someone who is separately working on their anxiety issues — though mental health cost support is quite poor, at least in the US.
But at their root: do these accommodations help or do they provide an unrealistic hope to the student?
NB: I want everyone to have a fair shake, and am a fan of ADA accommodation and affirmative action and related diversity efforts. I ask this question within that context.
... so here's the thing (speaking as someone who has taken many of these tests, been in admissions committees, and researches these tests). Take learning disabilities, and now expand it out to consider the similar factors, but with things like cultural background, life circumstances, and so forth and so on. Learning disabilities are not the same as those other things -- I don't want to equate them -- but they share some of the same issues with the test being normed on a certain standard population, and once you get outside of that, weird things happen.
Those density gradient plots in the linked blog post are interesting and useful to think about, but they're kinda hiding the fact that the vast majority of the data going into them is based on test-takers with a certain standard background, characteristics, and so forth. That in turn shapes the contours of what is error in those plots and analyses. The problem isn't necessarily that the tests are useless per se, it's that it's hard to interpret them in a way that accommodates people in nonstandard situations, or even accommodates the idiosyncracies of differences between people in general, differences that wouldn't matter in the real world.
The errors of using ACT-only, GPA-only, and so forth in the post is pretty interesting, but the author is missing the fact that that table has never really been the point of contention. The point of contention is whether that table's patterns apply equally across divisions of gender, race, SES background, age, disability status, and so forth and so on. They raise the point that the improvement in error from switching from ACT to GPA is comparable to (or better than) adding ACT to GPA, but isn't the real question whether doing so increases certain types of "predictable error variance", in the sense that you could predict the residual from things like SES, race, and so forth and so on?
I think GPA is seen as more acceptable than ACT because it's exchangeable in an important sense with the criterion being predicted. That is, if you want to know college GPA, maybe secondary school GPA is a little flawed, but at least it's ostensibly similar in terms of what it actually is. I think people have a sense that, say, you aren't using standardized tests as the criterion for college graduation, so why use it for prediction? Why have college and college GPA at all? Why not just let people take standardized tests and skip the whole degree program thing? There's reasonable arguments for doing that, but also reasonable arguments for not doing it, and many of those are the same arguments for and against using the test for admissions.
Tangent - I think that my particular disability (ADHD) helped me with my test scores. I was terrible at studying, but on the tests I would hyper-focus and do extremely well.
On the other hand, on the GRE, the writing portion was last and I could not bring myself to care. I scored in the 50th percentile, which for someone who regularly scores in the 99th percentile on standardized tests (including ones with writing portions), is pretty bad.
It’s a difficult problem to solve, but it’s unfair if a student gets easily distracted or anxious when taking a test around others in a time constrained setting. Those types of external factors generally shouldn’t be represented in a student’s test results yet often are. If tests are to be a good indicator for all students, I think accommodations have a ways to go.
If tests are indeed a generally good indicator of a student’s success and are continued to be used in such a manner, I feel it’s important that the experience is fair for all types of test takers. Many US tests such as the CPA exam do a great job of handling accommodations. However, many others (like the PSAT) have a lot of room for improvement.