The issue is that usually these terminals mean "higher throughput" when they say "faster", not "lower latency". The lowest-latency terminal in every test is Xterm, often by a LOT. Alacritty for a long time was actually quite bad at latency--and notably had a high variance on its latency, which is particularly miserable--but I think it improved recently? From what I remember of these benchmarks, someone using urxvt isn't going to be impressed by the supposed speed of Alacritty, if we are talking latency (and I agree: we should be, and everyone should use Xterm, which is actually an insanely good terminal).
As for throughput, I have lived in the terminal for decades, and as long as the various layers don't have massive buffers I honestly don't care how slow the terminal is: if I am dumping megabytes into my terminal backscroll I probably am going "oh shit" and am frantically hitting Ctrl-C... a slow terminal with a small buffer handles that almost immediately. I get the impression that there are maybe some use cases involving high-rate screen updates for apps that happen to run in consoles but are really GUIs... I don't use many of those and in fact try to avoid them, but I could maybe see an advantage for a high-throughput terminal to improve their simulated frame rate?
People writing fast terminal emulator care about throughput, latency and resource utilization. Kitty for instance deliberately limit throughput to keep both latency and CPU/GPU usage reasonable.
Also, by what metric is xterm so fast? I accept the idea that it is fast, even the fastest, but "a lot" seems suspicious to me. From the keyboard to the monitor, there is a lot of hardware and driver latency, I guess tens of milliseconds, so the effect of the terminal, should be relatively minor. I suspect xterm is so fast because the tool used to test it relies on the X server, and because xterm communicates with X directly, the latency will be really low from the point of view of X, but how is it end-to-end? Do we get the same results, with, say, Wayland?
The other one where xterm is uber low is https://lwn.net/Articles/751763/ is using this tool with software-based screen capture https://pavelfatin.com/typometer/, not sure how reliable that is as a proxy for end-to-end (at the time of the benchmark Wayland wasn't supported)
As for throughput, I have lived in the terminal for decades, and as long as the various layers don't have massive buffers I honestly don't care how slow the terminal is: if I am dumping megabytes into my terminal backscroll I probably am going "oh shit" and am frantically hitting Ctrl-C... a slow terminal with a small buffer handles that almost immediately. I get the impression that there are maybe some use cases involving high-rate screen updates for apps that happen to run in consoles but are really GUIs... I don't use many of those and in fact try to avoid them, but I could maybe see an advantage for a high-throughput terminal to improve their simulated frame rate?