Basically, as parent said, they're concerned that doing something oddball with the land will actually decrease its value or at least have some unknown effect on its value that they have no experience to evaluate.
Add to that, post-2008, most banks/credit unions are just going to err on the side of just not making the loan rather than making one which has some unquantifiable risk associated with it.
That said, it's all a bit arbitrary. I live in a relatively normal house on some very nice property but the house has enough quirks (it's very old) that I fully expect that whoever buys it some day will just build a new house.
Add to that, post-2008, most banks/credit unions are just going to err on the side of just not making the loan rather than making one which has some unquantifiable risk associated with it.
That said, it's all a bit arbitrary. I live in a relatively normal house on some very nice property but the house has enough quirks (it's very old) that I fully expect that whoever buys it some day will just build a new house.