Expected, I suppose. But it's still surreal to me that I'd need rows of full rack servers I used to install 25 years ago just to equal the core count here. Many, many, more rows to get anywhere near the capability.
I suppose it's more surreal than comparisons I made back then because there's very little difference, really, in the operating system and popular languages.
My surreal moment was reading the Genetic Programming series by John Koza. He discussed a 1000 socket Pentium II cluster which was used for a genetic search.
I did some back of the napkin math and realized that was roughly equivalent to my Zen 2 box.
So of course my failure to replicate the author's success was not the fault of my hardware.
If you like GP, you might want to check out Gene Expression Programming [1, 2]. A lot simpler to implement, with some nicer features. This excellent paper [3] has a lot of great ideas to get inspiration from (it uses GP).
Similar situation here. Circa 2000 I had a few separate machines in my lab making lots of noise, taking up lots of space and drinking up power. Now I run a much larger cluster of nodes on my zen 2 desktop in virtual machines. If I need some more I rent them from amazon for a few hours rather than having to go and buy something expensive.
Hah interesting. I remember a John Koza book but that was before the PII era I think. Also he liked Lisp (so maybe that's why he needed all that computing)
In the end it seems GA/GP was an "overoptimizing" for very non-convex/non-linear search spaces (much more than real problems) and that we're better off with NN/SGD representations of problems
There was a whole series of them, GP I-IV. I believe it was GP II that was discussing the Pentium II cluster.
I have had a lot of fun this year building a neural network from scratch as well as genetic programming. For real tasks, it does seem that the former has won out.
I have a layman's suspicion that GP will find itself useful in some way and have a resurgence in the future, but it might just be naïve wishful thinking.
This makes me think of Slashdot jokes from 20 years ago about 'imagine a beowulf cluster of these'.
I once worked for a server manufacturer that did a setup with hundreds of 1U size, dual socket Pentium 3, 1.0 GHz CPUs. In the era when the only way to have more than one core was a dual socket motherboard.
I'm curious if they will keep up the increase in core counts. It might get more difficult to do that unless the new TSMC processes work very well. They probably would have to go to 16 core (2x 8 core CCX) chiplets as I don't think they can easily fit more chiplets into Epyc.
Well you can expect 128 Core from 5nm or 3nm. I would guess 3nm makes more sense from a die size perspective. I imagine Zen 4 on 5nm would be the same as Zen 3 except with 16 Core and the same amount of cache, since the L3 Cache are now shared.
But the problem isn't so much with the Core Count but the TDP per Core. Imagine it is 3W per Core, 128 Core gets to 384W. And that is excluding the IOD.
I imagine they will need to get quite creative with the cooling process to pull that off.... I have heard of in die water cooling, but that sounds like the only option.
I replaced 4 servers (doing computation work not serving things so no need for availability) at work with a single desktop computer with a ryzen 7... Between the sata vs nvme the RAM speed and the CPU speed, tasks finish faster on rhe Zen2... Oh and it is 1/8 of the power as well. My only problem is that I can't get WoL to reliably work, only the mode that wakes up with any packet works.
We have disk shelves now, wouldn't the next step be a "network shelf"?
And probably a "GPU compute" shelf later on with some kind of high-speed high bandwidth backbone tying them all together.
Turn the whole rack into a glorified backplane.
¯\_(ツ)_/¯ sounds like something IBM would do.
We've had pretty high parallelism in server CPUs for a while, eg Ultrasparc T1 had 32 threads in 2005, later generations with 128 threads (T5). But a high number of threads and cores is a worse solution than being able to keep up high single threaded execution speedups, as most programs are not multithreaded because parallel programming is so hard.
The T1 was unusual at the time. It did have 32 threads, but provided by 8 VERY anemic cores. It was only competitive in some very niche scenarios. A "real" 8-core SMP Sun box was still pretty much a full rack and 6 figures.
I remember when the big company I was working for started using T1s for databases (I guess caused by Oracle marketing & change of licensing model) and asked us "consumers" to start using those systems.
I was skeptical so I downloaded some SPECint benchmark results ( https://www.spec.org/benchmarks.html ) of T1s, Power and Xeon, compared them and thought "mh, probably a bad idea" and then I ended up having to invest quite some time to convince my management to keep using "normal" servers.
On the other hand, later, I had a bit of fun hearing stories from other colleagues telling me how slow those T1-machines were once they started running on them their normal DB-workloads => after months of everybody complaining about bad performance, everybody went back to normal servers => a lot of time & money (& nerves) spent for nothing.
Yep. We've historically had a lot of CPUs or multi-cpu machines that were good for running "embarassingly parallel" loads but wasted capacity for general purpouse use, a bit like this one! More hw threads or cores has traditionally not been good news from software POV.
This kind of machine is nice to look at from afar but for most apps trying to get anywhere near a 64 fold speedup on your application on this would mean scrapping the code base and doing 1 failed rewrite, followed by 1 marginally successful one, taking up ruinous amounts of calendar time and engineering resources.
But of course it's diferent now than in 2005, because today we can't do any better.
It’s not though in the server space. Many server workloads are inherently parallel because the support multiple concurrent users. Think database servers or web servers.
By sustaining single core speedups, I meant big speedups, like we were observing bin the days before mainstream CPUs resorted to multiore. It was rather more than 10%.
And we'd still have be able to make processors with lots of slow threads of course for applications where that's cost or power effective, that's comparatively very easy.
Yes, having one core at 2x clock is definitely better than having two cores at 1x clock, I'd agree with that.
These days however, the difference between fastest core performance on low-core processors and fastest core performance on high-core processors is rather a lot less than 50% (Comparing like for like, eg AMD to AMD and Intel to Intel).
I really wish they would make the model numbers more distinct. "EPYC 7xxx" seems to encompass everything from 1st gen to 3rd gen, I have to look it up every single time.
EPYC = Brand
7 = 7000 Series
25-74 = Dual Digit Number indicative of stack positioning / performance (non-linear)
1/2 = Generation
P = Single Socket, not present in Dual Socket
When scanning the product stack or test results it's really annoying to look for the last digit though. But I guess Intel does that too with the whole vX thing at the end
I agree but I think that's on purpose, as part of their "finally get some share in server space", to avoid scaring any non-tech decision maker into thinking this is yet another new product or whatever that need to be thought about before signing on it.
In their other markets they have very clear generation naming, even going out of their way to give naming-room to their laptop chips.
AMD really want to be in server space because that's the safe spot (in revenue) for the next time Intel comes backs, and that's the one they missed last time they were ahead.
Damn I wish we could simply buy a single socket AMD EPYC instead of a dual socket Intel Xeon. When compared at last gen Rome, scientific workloads compiled with Intel compiler that relied on AVX-512 a lot performed piss poor on AMD while other regular stuff flew ..
It is too hard to buy these awesome chips and have them NOT run the serious stuff that our group wants to. I wish AMD would come up with a solution that makes it magically work with code compiled for AVX-512 :-(
If you can recompile but the code was written to only target AVX-512 you can use https://github.com/simd-everywhere/simde to near effortlessly map the intrinsics to AVX2 (or lower).
Recompiling is not an option? Also seems weird that the binary only ships with a AVX512 codepath and not AVX2 as well.
I remember there being also an issue of linked MKL libraries shipped with various software (e.g. Matlab) not using the appropriate AVX2 codepaths on 'unsupported' AMD processors and isn't was falling back on slow non vectorized code.
I think the key issue here is the use of the intel compiler on a non intel product. Can you not recompile your code with GCC or LLVM? are the performance metrics that bad that they are not options? AMD has made the system... Engineer your solution if you want to use it :)
What's the workload if you don't mind me asking? Presumably the difference isn't just down to the smaller width of AVX2 but new instructions in the AVX-512 family?
The prices felt off mostly because CPU are not the only cost, and in many cases it isn't even the most expensive cost on Server. Memory is, especially ECC Memory. Memory price per GB hasn't fall at all. And in many cases they are even more expensive. And it is only in recent few years you start seeing "High CPU Instances" because it is cheaper to include a Higher Core Count in the plan than to increase the Memory.
And because of the increase in VM / Core Count and Tenant, the server are now equipped with multiple High Speed Network connection. And if you look at it from a Cost Per Network Card / Port / Bandwidth / Server / Tenant. Price hasn't fallen much either. Adding in your cloud / VPS instance are paying Electricity, Co-Location and Networking charges, none of these have an Moore's Law and are part of the TCO. And if you include the continuous raise of wages, things doesn't look all so good.
But Zen 2 is barely deployed so its effect on the industry is not here yet. Linode already use Zen 2 and they are already offering better price plan than DO or other Medium Cloud vendors. I expect Zen 3 to be big with many of the Cloud Vendors. And you should see those reflected on pricing.
Disks have gotten ridiculously fast. One NVMe card can move 5GB of data or 1,000,000 IOPS over 4 PCIe lanes. With 128 cores, you need to parse 39MB/sec per core, or dispatch 7.8 thousand I/Os per second for each drive in the system.
This processor has 64 PCIe lanes, and a drive uses 4. If you use half the lanes for disks, you’d need to multiply my numbers by 8. So, in addition to running application logic and talking to the network, each core needs to parse 312MB or dispatch 62K IOs per second.
Those numbers are certainly achievable, but hitting them requires some care. So, this processor is fairly well-balanced for moderately compute intensive workloads, such as query processing, or maybe frontend logic (which would probably want a machine with more network and less disk).
Note that the 128 cores are just to keep up with storage that easily fits in 1U or a fairly compact desktop tower case.
If you measure speed as the IO to compute density, these CPUs are actually slow and bulky by historical standards.
Some workloads can utilize almost all the power. Some workloads can not. Server workloads usually can scale almost infinitely. And we don't really have a choice, single-core workloads usually don't scale (most of the recent advancements are from implicit parallelization using speculative execution).
VM data centers. One of these machines can be loaded up with ram and host tons and tons of instances without over provisioning. Some of those VMs will be mostly idle, but others will be at full workloads, depending on the tenants.
Also, there are sites like Source Hut or Travis that run thousands upon thousands of CI jobs every minute. I worked at a place where we were resizing thousands of photos and hundreds of incoming videos ever minute. One of these could easily replace 10~20 nodes in a transcoding farm for companies that do these types of workloads on physical servers.
There are tons of commercial applications that would keep these things busy during business hours, and some that would also keep them busy 24/7.
No we don’t know how to use them. The abilities of single systems have far outstripped our ability to manage all those resources. It takes a lot of work to drive a system like this to the rails, and very few people even bother trying. The closest most people come to exploiting a machine this large is by carving it up into normal-sized virtual machines with independent operating systems. This makes the utilization look acceptable but in reality much of the resources are just being squandered.
It's easier to scale to multiple cores than to multiple machines, and cloud services have been doing multiple machines for decades.
If this wasn't needed or wasn't useful then we also wouldn't ever see companies buying more than a single rack (still want redundancy and hot spares ofc), but they do.
So yes we know how to use all these cores. You take what's currently 4 machines and consolidate it to a single machine. Or similarly 40 machines to 10 machines, etc...
I think you're exactly right. These cores are designed to be sold en masse to the cloud providers, who are looking for density in the rack, lowest total cost of operation (up front cost is nearly irrelevant), and above all power and thermal efficiency.
AMD knows they have some serious advantages there, especially on the efficiency front, with Zen3, and they're going pedal to the metal.
I suppose it's more surreal than comparisons I made back then because there's very little difference, really, in the operating system and popular languages.