You don't have any responsibility for what you build? Maybe it's on you, too. (As a PM, I HATE when engineers are full of confirmations and feedback loops but cop the 'not my fault, I just build exactly what you tell me' attitude the moment there's something to fix)
The PM is responsible for the requirements. If my PRD fails to be explicit or is not involved in the QA process, the PM needs to take ownership. Yes, everyone is responsible for the UX, but the PM especially.
Yeah well, when I find a bug I go to my PM and I say "Dude, I messed up. Here's the problem. Here's what you need to tell the customers to do/not do. Here's what i need to fix it."
When there's a problem with requirements and the users are screaming, and the system isn't doing what they want (but exactly what they built), the PM comes to me and says "We fucked up. You need to drop everything and fix this right away."
This has been the case with almost every PM I've ever worked with.