I hear you! Ended up building such a system so many times, so much we've built an open-source, self-hosted system to do it out of the box! https://cerbos.dev

Would love your feedback!