I think the developer/publisher are just responding to market pressures.
Quake Live (at least for its first few years) was relatively pure and old-school, including in the ways mentioned in this thread, but it was only moderately popular and didn't seem to do well financially. The most (financially) successful modern games tend to focus a lot on extrinsic motivations to play -- including character progression, unlockables, gambling crates etc., but also things that give extra context to individual matches, like statistics and ranking points -- rather than just the core gameplay. People also have less tolerance for being the newbie who loses 100-0, and expect to be matched against players around their level.
Not all of these things are relevant to your question, but some of them help to explain the trend toward lobbies, matchmaking queues, and other obstacles to just clicking on a server list and jumping into a game.
Quake Live (at least for its first few years) was relatively pure and old-school, including in the ways mentioned in this thread, but it was only moderately popular and didn't seem to do well financially. The most (financially) successful modern games tend to focus a lot on extrinsic motivations to play -- including character progression, unlockables, gambling crates etc., but also things that give extra context to individual matches, like statistics and ranking points -- rather than just the core gameplay. People also have less tolerance for being the newbie who loses 100-0, and expect to be matched against players around their level.
Not all of these things are relevant to your question, but some of them help to explain the trend toward lobbies, matchmaking queues, and other obstacles to just clicking on a server list and jumping into a game.