cpu – Why can’t we vertically scale a system infinitely?

Because… there’s only so fast that we can make one thing go.

Memory isn’t likely to be a problem any time soon. But that’s because memory is already horizontally scaled. You can always add more modules, or more chips on a module. You could have a room-sized or building-sized memory module.

However, room-sized memory will be slow because the speed of light takes quite a lot of nanoseconds to reach across the room. Better to have the memory near the CPU, so instead of one CPU room and then one giant memory room, we alternate: one CPU, a few memory sticks, another CPU, another few memory sticks, etc. And this is what you get when you have multiple servers.

We can always make pointers bigger if needed, that’s no big problem. You’d need a new type of CPU, but if you’ve got a city block full of RAM then you can surely afford to pay Intel to create a 128-bit CPU.

Vertically-scaled memory would mean putting more bits in a chip, and we don’t know how to make them that small, and we certainly can’t make them smaller than atoms. But it’s okay because memory already scales horizontally.

Hard disk/SSD – Same thing – it already scales horizontally! You can make circuit boards full of terabyte SSD chips, stack up the circuit boards into a big rack, multiply the racks to fill a room, fill up every room in the building. You can use software RAID to pretend it’s one big drive.

Vertically-scaled HDDs/SSDs would mean putting more bits in a chip (or on a brown spinny thing), and we don’t know how to make them that small, and we certainly can’t make them smaller than atoms. But it’s okay because drives already scale horizontally.

Network is measured in gigabits per second. We use optic fibers and modulate them on and off to achieve gigabits per connection. The limiting factor for vertical scaling is how fast we can turn the light on and off. We don’t know how to make light turn on and off a trillion times a second, and it certainly can’t be any faster than the frequency of the light wave.

Networks can scale horizontally, because you can put multiple fibers in a cable and connect them to multiple network cards.

But I think you forgot something. You want to have fifty quintillion bits per second coming into your network port, but you only have a 4GHz CPU to process those bits? CPUs are the real bottleneck because they have to do things in order. They can’t pretend to scale vertically just by scaling horizontally. A 10-core CPU cannot pretend to be one CPU that’s 10 times as fast. And if you’re going to buy a 10000000-core CPU to process your fifty quintillion bits per second… at that point, why not buy 1000000 separate 10-core CPUs? They’re much cheaper because everyone uses them, they’re not a custom design just for you. Because it’s faster to interleave things to reduce distance, a 10000000-core CPU would probably be designed like a rack of servers anyway. May as well just buy a rack of separate servers.