My personal experience: We run over 1.1m test cases to verify every PR that I submit, and there are more test cases that don't get run on every commit and instead get run daily or on-demand.
At that scale getting quick turnaround is a difficult infrastructure problem, especially if you have individual tests that take multiple seconds or suites that take multiple minutes (we do, and it's hard to actually pull the execution time down on all of them).
I've never personally heard "we don't have the budget" or "we don't have enough machines" as answers for why our CI turnaround isn't 5 minutes, and it doesn't seem to me like the answer is just doubling the core count in every situation.
The scenario I work on daily (a custom multi-platform runtime with its own standard library) does by necessity mean that builds and testing are fairly complex though. I wouldn't be surprised if your assertion (just throw more resources at it) holds for more straightforward apps.
At that scale getting quick turnaround is a difficult infrastructure problem, especially if you have individual tests that take multiple seconds or suites that take multiple minutes (we do, and it's hard to actually pull the execution time down on all of them).
I've never personally heard "we don't have the budget" or "we don't have enough machines" as answers for why our CI turnaround isn't 5 minutes, and it doesn't seem to me like the answer is just doubling the core count in every situation.
The scenario I work on daily (a custom multi-platform runtime with its own standard library) does by necessity mean that builds and testing are fairly complex though. I wouldn't be surprised if your assertion (just throw more resources at it) holds for more straightforward apps.