Why is it cheaper to rent many single core servers (VPS or physical) than it is to rent a multi-core server?

To me it seems that pricing reflects a marketing decision more than it does the reality of the hardware.

Multi-core has more power than single core, therefor you get more power in a single system as customer.

In the virtual world, it is easy to simulate a single core server. The server uses relatively few resources, and for small instances a single physical box can support hundreds --- potentially even thousands --- of nearly-idle virtual servers.

That's how you get really cheap virtual hosting for a few dollars per month. They know that on average the sites will get approximately zero traffic so the amortized cost is tiny.

If you start requiring multiple cores your demands go up considerably. It means that when they allocate your slices to you they must limit many other processes.

Consider if the hosting is running on a 12-core e7 server. We'll assume it has enough memory and other resources, we're only interested in CPU time.

Let's say every virtual machine is single core. Any time any server wants to spin up some work they can throw it on the first available core and everything is wonderful. Such a server can host several hundred simple servers, since in practice most will usually be idle. The virtual machines are mostly idle, but when they operate they require only 1/12 of the box. Scheduling is easy because only the tiniest sliver of the 12-core resource must be available. A single process can be scheduled at any time, there must only be vacancy on a single processor.

Now let's say you want a quad core VM. And we'll say other processes on the box also require dual core and quad core VMs. This time when your virtual process gets busy, instead of requiring 1/12 of the box, you need to be scheduled requiring 1/3 of the box. Because others are also scheduling large chunks of resources, scheduling processing on the box is much harder since four threads must be available. Only three processes can be scheduled simultaneously (3 virtual machines each taking 4 cores = 12 cores.) Since you are requiring so much of the resource the hosting company cannot share the box with many customers. While they might be able to load several hundred of those single processor VMs discussed earlier, they can only load a few tens of multi-processor VMs on a box before it is full.

The cost for VM management, for power, network, hardware, and environment for that class of server is about $6000 per year. So in the one case you've got a physical box with perhaps 200 low-utilization virtual machines running on it, meaning they are quite cheap. On the other case you've got the same physical box that can only support perhaps 20 virtual machines running on it. It's going to be 10x more expensive.

This is why on hosts such as those at Equinix and Amazon, you see costs of single CPU machines at 1.3 cents per compute hour if you stay on the heavily loaded cheap boxes. But if you move over to the "compute optimized" versions that schedule multiple CPUs for your use, you start into 11.6 cents per hour or more. By the time you reach the bigger boxes with 16, 32, and even 36 cores you are effectively taking up an entire server to yourself with perhaps a few micro instances slipped in the side, so the prices are quite high.

Perhaps a vehicle analogy to help:

Imagine a very large road, perhaps something like a modern 4-lane-per-direction highway. Let's imagine it is toll-based. Consider how many motorcycles and motorscooters we could fill it with. Instead of being 4 lanes, it could perhaps be 12 lanes. You could fit a huge number of vehicles before saturating the roadway. As small vehicles their damage done to the roadway is negligible. Your toll per vehicle can be extremely small. Now consider the same width of road, instead filling it with double-wide oversized trailers. The roadway effectively becomes two lanes. The vehicles are not agile so distances between them increase. The vehicles are heavy and cause significant wear and tear to the road surface. The toll per vehicle will be many times the toll of the all-motorcycle highway.

Lots of tiny instances is cheap. Bigger instances require not just more space for themselves but also limit the availability to others.

That doesn't mean the hosting companies are not making a profit, they certainly are. The cost to have an always loaded dedicated Amazon EC2 system is typically far more than the cost to stick your own similarly-equipped machine in a reliable co-location facility...

But the relative prices they charge, a fraction of a cent per hour for a single processor low-use instance versus several cents for a multi-processor instance, that comes from actual costs. Requiring that multiple cores be simultaneously available to your instance uses an order of magnitude more resources in order to be available to you,

Or, to summarize: You pay for what you get.

Linode pricing scales linearly with cores, so maybe you just need a better host :P

If you start requiring multiple cores your demands go up considerably. It means that when they allocate your slices to you they must limit many other processes.

I believe this is really the crux of it. The assumption of the seller when renting a single core is that it will have low utilization -- that even one single core is greatly more CPU power than the client needs regularly. The assumption of the seller change, though, when renting multiple cores, because why would a client opt for multiple cores when even one goes idle most of the day -- the assumption is that if you're paying for multiple cores, you're actually putting them to use frequently so the pricing structure reflects that. This only applies to virtualized servers, but its safe to assume that all single-core instances today are virtualized.

On the physical-server side of things, any entry is costly, but you will see a fairly gentle increasing curve to pricing. That's just the typical performance-premium/commodity/obsolescence curve, IMO. The newest and most-powerful servers can fetch the highest prices, more modest machines fetch market-rates, and machines of a certain age can be offered for service more cheaply, since their capex was recouped long ago, as long as there is demand and there is bandwidth and rack-space to support them.

Linode pricing scales linearly with cores, so maybe you just need a better host

Linode is pretty good. I also like Digital Ocean, which you can rent and power-on by the hour (in fact, they have no monthly billing rate to speak of for VPS, only the hourly rate times the number of hours in a month.) You can choose a VPS from there in a variety of sizes, and I think prices actually follow RAM capacity most-closely (roughly $10/GB month) with CPU cores and disk capacity (all SSD) following in general but not in total proportion.

EDIT -- and no sooner than I had spoken, I load the Linode site and see that they have moved to basically the same hourly billing and basic cost structure. Last I looked at them, they were still billing monthly, IIRC, though I could be mistaken, though the monthly rates are in alignment from what I recall.

Particularly for low-end virtual servers, containerization technologies like Docker are really commoditizing the market. In all but name, most of the services with these kinds of hourly pricing structures are just places to park your container instances for a few hours, occasionally or to meet capacity needs on-demand. I think the writing is on the wall that this kind of thing (that is, Micro-service architecture) is the new normal going forward for most workloads. The days of huge, monolithic services and servers are numbered for all but a few rarified uses.

The days of huge, monolithic services and servers are numbered for all but a few rarified uses.

For small-to-medium setups, I think there's a lot of truth to that.

For larger operations, a dedicated ops team with a data center and leased hardware and negotiated first-tier bandwidth is going to be half the cost of premium virtual cloud providers, even after considering daily surge usage. Our off-peak load is > 50% of our on-peak load (because we're fairly international in our user base) and our bandwidth costs through Amazon would be enormous compared to what we pay direct to providers.

"Rarified uses"? True, there aren't that many sites that push dozens of megabit every hour of the day. But those who do, can't live on cheap cloud providers, and would pay through the nose for top-tier cloud providers.

On that point I wasn't speaking so much about big enterprises migrating to cloud-based infrastructures -- although it does make a lot of sense for small and medium-sized entities without needs that would occupy at least a moderately-sized server room, complete with staff.

I was speaking about more to Micro-service architecture and containerization -- which is the majority of even Google's own architecture, these days -- just running on their own hardware in their own data centers.