Have you tried it with a different homeserver (either hosted or some other community server)? Matrix.org has a fucktonne of users (~5 million from memory), and given how little funding they have it's shocking that they can afford to run a homeserver at all. I've run my own homeserver for the past few years and I haven't ever had any issues with performance (even when talking to people on Matrix.org).
I can attest to that. I run a Matrix homeserver for a small community, and it's blazingly fast all the time.
Everyone should migrate off matrix.org sooner rather than later. I fully expect seeing regional public Matrix homeservers popping up here and there eventually, just like regional public Jabber servers did back in the day.
That's not my experience at all. It got better when I left some large rooms. But I still regularly get things like messages that the server is offline, taking a minute before the UI shows that my message was sent, after my laptop was suspended, it takes minutes before element becomes usable again. I run my own IRC bridge, and if they send more than 5 messages a minute I start losing messages and they get out of order.