This is a marvelous articles indicating talking about dangers and trade-offs.

The thing about the diagrams is if the data was points on a coastline, maybe the most overfit curve would be the most useful. It seems like this kind of thing doesn't have an absolutely objective answer but depends on context (why one talks about tradeoffs, not laws).

Yes, you're correct, the sweet spot between under- and overfitting is completely dependent on the nature of the data set, and cannot be determined a priori. That's why cross-validation and similar techniques are critical for determining the right trade off between bias and variance.