One of the most interesting things I learned at Microsoft: demand for cloud is not driven by performance. The most popular SKUs are cheap older-generation things because we’ve already reached the point where computers are fast enough for a lot of massive use cases. When they roll our faster SKUs, very few people move.
Companies keep producing faster and faster server CPUs, but what the cloud providers really want is more reliable ones. When a CPU dies, that node is offlined. VMs on it are redeployed elsewhere, which customers notice (and which will impact SLAs). The node then sits there until someone goes and collects it. Fetching it and collecting usable components is expensive (but worth it, because a node with 1TiB of RAM is worth pulling the RAM from). Having a failed node now means that you have unbalanced network capacity (that rack has more capacity than it needs), but you can’t easily reduce power by turning off a fraction of a network.
Faster chips are useful only because they can often, instead, provide the same performance for lower power (and lower power, in addition to being cheaper, often also improves reliability).
I found that fascinating because it explains why they’re so excited by any use case that looks like it will trigger a spike in demand. It also means that there’s a lot of scope for disrupting the cloud market from the low end. I think companies like UniKraft will have a big impact here: being able to scale down is the killer app for the cloud, because renting 1% of a server is often enough, and if you pay 2% of the price of a server each year to do it then you’re still paying less than if you bought a server. A lot of cloud workloads are incredibly inefficient. The big providers have no incentive to make it easy for people to buy 10% as much compute as they currently need, but if you do that then you can undercut them by a huge amount, even if your profit margins are much higher.
Companies keep producing faster and faster server CPUs, but what the cloud providers really want is more reliable ones. When a CPU dies, that node is offlined. VMs on it are redeployed elsewhere, which customers notice (and which will impact SLAs). The node then sits there until someone goes and collects it. Fetching it and collecting usable components is expensive (but worth it, because a node with 1TiB of RAM is worth pulling the RAM from). Having a failed node now means that you have unbalanced network capacity (that rack has more capacity than it needs), but you can’t easily reduce power by turning off a fraction of a network.
Faster chips are useful only because they can often, instead, provide the same performance for lower power (and lower power, in addition to being cheaper, often also improves reliability).
I found that fascinating because it explains why they’re so excited by any use case that looks like it will trigger a spike in demand. It also means that there’s a lot of scope for disrupting the cloud market from the low end. I think companies like UniKraft will have a big impact here: being able to scale down is the killer app for the cloud, because renting 1% of a server is often enough, and if you pay 2% of the price of a server each year to do it then you’re still paying less than if you bought a server. A lot of cloud workloads are incredibly inefficient. The big providers have no incentive to make it easy for people to buy 10% as much compute as they currently need, but if you do that then you can undercut them by a huge amount, even if your profit margins are much higher.
4
