> The OP's point is that when measurement error is as large as it is for college admissions, using a simple lottery to narrow the cohort is as good as any other method wrt getting the best students
Not really. Lets say every student scores within 10 percentage point of their true "merit". So if you only pick the top 1% scorers you actually could get students down in the top 11%. So why not just pick top 11% by random? Because when you pick a top 11% scorer you extend your range so you now could get as low as top 21%. Ranking them still matters quite a lot even if error range is larger than their pick range.
You can pick a different random set of inputs and come to the opposite conclusion.
It could be true that there is no correlation between true "merit" and test scores when you get to, say, the 99%th percentile. Especially since the predictive ability of these tests must be suspect now that you can study strategies for them.
In that situation, Harvard shouldn't value a 1600 any more than a 1540.
Not really. Lets say every student scores within 10 percentage point of their true "merit". So if you only pick the top 1% scorers you actually could get students down in the top 11%. So why not just pick top 11% by random? Because when you pick a top 11% scorer you extend your range so you now could get as low as top 21%. Ranking them still matters quite a lot even if error range is larger than their pick range.