I work in the industry. There are a couple of reasons they don't work, at least as currently used.
1) Over saturation/overused: Especially as High Schools begin using these, students get too many, too frequently, and it just becomes a sort of background noise. Back when students still had email as their prime mode of digital communication, we had the exact same issue. Studies on their effectiveness were based on relatively short time spans where nudges were new & novel: The novelty has worn off.
2) A simple nudge can't help a complex problem. A college may nudge "You haven't registered yet! Don't forget to register for the next semester". That's an extremely frustrating message to receive when you're poor and can't pay next semester's bill, or some other roadblock.
3) It has been used to justify not scaling human resources for students, so any benefit is offset by lesser real human contact with advisors. People complain about administrative bloat at universities, but that doesn't apply to advising: I've seen caseloads rise from ~300/advisor to ~500/advisor. This is especially problematic for overcoming #2 above: Advisors can often help with complex issues that student's believe are insurmountable.
My observations on how to do it better:
Only use nudges for low-stakes issues, and even then choose carefully so students don't become numb to them. High-stakes issues need human outreach. With limited bandwidth by advisors to do this work, it needs triage: Even if it's high stakes, don't waste time reaching out personally to low-risk students. Determining risk requires a two-pronged approach:
1) A statistical model that uses historical data to identify students at risk of poor performance or attrition. A competent data analyst can build a reasonable model without having to go too far into advanced data mining. "Deep Learning" tends to be fairly useless here: There's not enough data. There's also a few good vendors out there that will perform more advanced risk modeling, partitioning a school's population into a dozen or more groups and building models for each of them. These aren't cheap, but I've seen it pay off when correctly paired with one-to-one human outreach initiatives. In a single year I saw it reduce attrition by 1%, not to mention academic performance increases, and that more than paid for the cost. I've also seen schools pay the invoice and expect things to magically change without aligning their resources to the risk analysis output and waste a lot of money.
2) A rule-base system of identifying students at risk. Things like a late payment; a good student getting their first D or F in a class; delaying registration for the next semester; requesting transcripts to be sent to another school; and many other factors.
Using the high risk population identified in #1 and #2, that is where you focus your one-to-one human outreach efforts.
1) Over saturation/overused: Especially as High Schools begin using these, students get too many, too frequently, and it just becomes a sort of background noise. Back when students still had email as their prime mode of digital communication, we had the exact same issue. Studies on their effectiveness were based on relatively short time spans where nudges were new & novel: The novelty has worn off.
2) A simple nudge can't help a complex problem. A college may nudge "You haven't registered yet! Don't forget to register for the next semester". That's an extremely frustrating message to receive when you're poor and can't pay next semester's bill, or some other roadblock.
3) It has been used to justify not scaling human resources for students, so any benefit is offset by lesser real human contact with advisors. People complain about administrative bloat at universities, but that doesn't apply to advising: I've seen caseloads rise from ~300/advisor to ~500/advisor. This is especially problematic for overcoming #2 above: Advisors can often help with complex issues that student's believe are insurmountable.
My observations on how to do it better:
Only use nudges for low-stakes issues, and even then choose carefully so students don't become numb to them. High-stakes issues need human outreach. With limited bandwidth by advisors to do this work, it needs triage: Even if it's high stakes, don't waste time reaching out personally to low-risk students. Determining risk requires a two-pronged approach:
1) A statistical model that uses historical data to identify students at risk of poor performance or attrition. A competent data analyst can build a reasonable model without having to go too far into advanced data mining. "Deep Learning" tends to be fairly useless here: There's not enough data. There's also a few good vendors out there that will perform more advanced risk modeling, partitioning a school's population into a dozen or more groups and building models for each of them. These aren't cheap, but I've seen it pay off when correctly paired with one-to-one human outreach initiatives. In a single year I saw it reduce attrition by 1%, not to mention academic performance increases, and that more than paid for the cost. I've also seen schools pay the invoice and expect things to magically change without aligning their resources to the risk analysis output and waste a lot of money.
2) A rule-base system of identifying students at risk. Things like a late payment; a good student getting their first D or F in a class; delaying registration for the next semester; requesting transcripts to be sent to another school; and many other factors.
Using the high risk population identified in #1 and #2, that is where you focus your one-to-one human outreach efforts.