If they kept CPUs 100% busy, how would they deal with a surge in demand or a failure situation? You HAVE to keep spare capacity or 1 failover crashes your entire fleet of 5 billion servers.
You could correctly argue that usage increases with scale.. so while you with 1 app can maybe only keep a server 20% busy, Amazon can afford to keep their servers 70% busy.. which is part of the story.
But the other part is you are paying for the convenience. You can get raw CPU way cheaper than what lambda gives it out at. Even if you only kept your CPU usage at 10-20% - for example Digital Ocean would still be cheaper than the Amazon Lambda app version. You are literally paying a surcharge to not have to set up your own servers, or deal with maintenance. I find it very unlikely you will save big money with lambda over your own servers.
You could correctly argue that usage increases with scale.. so while you with 1 app can maybe only keep a server 20% busy, Amazon can afford to keep their servers 70% busy.. which is part of the story.
But the other part is you are paying for the convenience. You can get raw CPU way cheaper than what lambda gives it out at. Even if you only kept your CPU usage at 10-20% - for example Digital Ocean would still be cheaper than the Amazon Lambda app version. You are literally paying a surcharge to not have to set up your own servers, or deal with maintenance. I find it very unlikely you will save big money with lambda over your own servers.