This is a fundamental, and perhaps insoluble, problem with the moral principle of liberty and self-ownership: to what extent should you be permitted to voluntarily limit, surrender, or exchange that ownership?
One can certainly make a case that even limited-scope non-compete clauses in employment contracts are an affront to human dignity; on the other extreme, there are those who would claim that freedom necessarily includes the "right" to sell one's self into indefinite servitude. Where do we draw the line? I don't see an intrinsic "bright line" or Schelling Focus on the question. What is the "statute of limitations" on the Present Self being constrained by the choices of the Past Self (at least, in the context of contract enforcement)?
I can't claim to have a complete answer to that question, but it seems that every time that the line is drawn too far towards the direction of slavery (i.e. away from individual liberty) there is a substantial power-imbalance.
That seems to suggest that any situation where there is a large power (information, monetary, etc.) asymmetry between two parties will lead to one side being heavily disadvantaged, almost certainly due to the intentional structure of that arrangement.
If true, that would suggest that any circumstance where there could be a large power imbalance between parties must be carefully moderated and that limiting "individual freedom" by not allowing people to sign away their rights in a way that mostly benefits someone else could be a reasonable way of approaching this problem.