I was just discussing this with a friend. Obviously there's a certain degree of multi-tenancy sharing that's most cost effective. For electrical power use and removing heat there's a floor where, if not being used, the best you could do would be to turn a system off, which has its own issues. Go too far and you won't be able to also run a spot market in CPU instances like AWS does.
> For electrical power use and removing heat there's a floor where, if not being used, the best you could do would be to turn a system off, which has its own issues.
What issues? If your cloud architecture can't save power when demand is low, how can that be a positive?
> Go too far and you won't be able to also run a spot market in CPU instances like AWS does.
As I understand it, a spot market isn't ever an architectural goal, it's a way to exploit the practical realities of unused resources/capacity. If that necessity goes away, that'd be a win.
I see at least 3 possibilities:
1. The spot market is about filling temporarily-vacant VM slots on host physical machines
2. Additional physical machines must be kept online so that VMs can be quickly spun-up, and the spot market is about capitalising on this necessity (this is really a variation on Option 1)
3. The spot market is about deriving at least some revenue from the unutilised fraction of an always-on fleet, encouraging customers to run heavy jobs during times of low load on your cloud
I doubt Amazon's fleet is always-on -- I imagine they have fairly significant capacity in reserve to keep up with growth and spikes -- so I doubt it's Option 3. (This is a perfectly obvious question but I really can't find a definitive answer. Maybe my Google-fu is weak today.)
I think Amazon’s hands are mostly tied when it comes to powering off machines. Many VMs are run indefinitely, so whatever machine they’re on has to stay powered on. They may have live migrate (I think they do, but no official word), but they can’t be too aggressive about that or customers will notice.
I stumbled across a 2012 article on this [0] that says they'd 'retire' instances. Wouldn't force the VMs over to another host, they'd just terminate them. They'd generally give forewarning.
I wonder about live migration. I imagine there must be some brief interruption even there.
> Many VMs are run indefinitely, so whatever machine they’re on has to stay powered on
Amazon must be adequately motivated to find a way to power-down unused capacity. My guess is they have some kind of system to maximise this.
Yeah, I’ve experienced several instance retirements. Most with a fair bit of warning, one a few minutes after the instance stopped responding. I haven’t gotten a retirement warning in several years, so it seems they're doing something to mitigate hardware failures, although my infrastructure is small enough it could just be luck.
Plus Xen has live migrate now, so I assume AWS’s frankenxen would have it as well.
There is indeed a brief interrupt, which is why I said they can’t be too aggressive with it.
Anyways, I’m sure Amazon does what it can to power down machines, I just think they’re stuck leaving much the data center running because they can’t kick off running VMs.
