Hacker News new | past | comments | ask | show | jobs | submit login

It won’t average out perfectly. There will still be lucky and unlucky students.

Of course it’s better than a fixed order, and if it’s easy to switch then might as well. But we should keep thinking about how we can make it even better.




Since the effect looks very small, it looks to me like it's only a problem because it adds up if it happens for every assignment for every course. I don't think it needs to average out perfectly; it looks to me like you'd have to be astronomically lucky/unlucky for it to matter if each assignment is in random order.


Some courses are only graded based on a small number of tests. I actually went to UM and a grade might be something like 30% midterm 60% final 10% homework (obviously different professors have different systems). In that case if you get unlucky just twice on the two tests you basically get the full penalty.


I'm not sure how much a +/- 0.3 (out of 100) deviation from average on a single course matters even if you end up dead first/last for both midterm and final in that example. I mean, it will matter sometimes. But it's (by far) not as big a deal as if it happens for all your courses.

Still, yes, you could flip the order from midterm to final instead of randomizing both and the effect goes to more like +/- 0.1 out of 100 for the luckiest and unluckiest.


Yes, that sort of mirror-sampling would reduce variance. The problem is, though, you need to know all the uses of randomness in order to properly counterbalance them, and these systems are already enough of a pain to use.

(For example, if you have two, you can simply swap: but what about other biases? like if it's broken in half to assign to 2 grades. Or what about if there are three exams? And what about balance across other courses? if you want to do variance-reduction and tricks like antithetic sampling, you need to know all this in order to structure it properly - get it wrong, and you may make things worse.)

So that's why simple random shuffling would be preferred. It allows total ignorance of all other uses (past present and future), handles all ordering biases, and can be done independent in parallel across arbitrary sets of courses/exams/grades/students.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: