This, totally. I manage a large B2B CRM with complex user-auth requirements and we use an off the shelf provider. It was an easy decision when we began, but now it has become a bottle neck in many ways. As your application becomes big you need a lot more customization to your auth logic. Large customers have specific customizations that off the shelf auth providers don't provide, or make it complex to implement. Over time auth and RBAC becomes crucial to your application. Engineering effort required to read through your providers documentation and implement workarounds exceeds a purpose built, native solution.