I think this is a false dilemma; you can have the high-quality implementations and be more open.
I've criticized Matrix before for their "protocol-first" approach and "too neutral" stance towards clients (which they've changed somewhat it seems; previously [1] was a table of clients with no clue what to choose, now it at least has "featured clients"). I feel they repeated the same mistakes as XMPP, which has not improved their client list.[2] Protocol nerds will say that's a good thing, but all it really does is ensure your protocol remains marginal because most people just get confused. People choose software, not protocols.
But you can write a high-quality client and a specification and allow people to write their own apps. IMHO Signal is needlessly restrictive. Sure, focus on your own implementation and the quality of that first. 100% the right decision. But there's no reason to not at least allow some things down the line. Signal is just a few months shy of their tenth birthday – they're well past the "ensure the quality of our official client"-phase.
As soon as you do that though, it becomes a nightmare to adjust anything about the protocol, and you end up with incompatible clients. So you can use the app perfectly with friend 1 that has the official app, but with friend 2 who uses one client, sending photos doesn't work, and with friend 3 voice calls don't work, and adding friend 4 to a group chat somehow breaks it entirely for everyone.
Friend 2 insists on using their client because it has dark mode, and for the average user, what they see isn't "friend 2 is extra and has a broken client", they see "that app fails to send pictures about a quarter of the time, let's use whatsapp".
At the end of the day, the problem with this model is that it expects free labor to take over the next part. Which might work for a little bit -- until it doesn't. Then you have the situation we're currently in where everything related to matrix is mediocre.
I don't know if there is a straightforward correlation. I agree that my first Matrix experience was also not that satisfactory, but my university switched from XMPP to Matrix. I really liked conversations and quicksy. It just worked for me out of the box even with OTR stuff. However, it seems that there was not enough development on the server side, which I guess it led to the switch by our computing Center. Also the whole German health system as well as the army is switching to Matrix. I still think it is completely over engineered but it has a decent push.
https://blog.koehntopp.info/2024/02/13/the-matrix-trashfire.... explains why Matrix is lacking market share, and I think Signal's decision to be aggressively closed is due to a justified fear of becoming that.