The problem with Django is not swapping out defaults. The problem is it doesn't like to sit at the edge of an application where it belongs. Django should be equivalent of any other MVC-style framework (like Qt etc), sitting right out in the "interface" layer of your software. But it just doesn't work well out there, mostly due to how tightly coupled it is with its ORM.
Django isn't that, though. That would be something like Flask. Django is batteries-included for one (broad) purpose: you are going to use a relational database to make a CRUD app and we will make that easier for you by standardising a load of stuff. That constraint buys a lot of power.
Nothing is ever just pure CRUD, though. What would you say is the minimum level of business logic where one should consider Django the wrong tool for the job?
One of the problems I find is gradually complexity creep from the business. It starts off as a little clean method here, a signal there etc. Before you know it you've got a domain model tightly coupled to CRUD primitives.
I'd say you should be able to quantify it. "a domain model tightly coupled to CRUD primitives" is not an intrinsic evil. If you were to genuinely save money or reduce risk by, say, moving off a relational database you would just need to pick that point and motivate for a rewrite.
The much higher risk IMO is building a hyper-flexible application from the start knowing what you need today is a CRUD app, just in case in future you need something else. That's how you get shelfware and awkward conversations.