The “high-level CPU” challenge (2008)
yosefk.com
yosefk.com
Actually if you've actually looked at things like SSE memory copying/comparison routines, it's not "a few bytes", it's more like a factor of 100x+. REP MOVSB is 2 bytes; a SSE memcpy, highly unrolled - incidentally, also a bad idea for modern CPUs - is easily a few hundred. I've seen ones over a kilobyte(!) The former is essentially the same speed as the latter, but makes for significantly less instruction cache utilisation, which is increasingly important today.
You see, RISC happened for a reason.
When memory bandwidth was not the bottleneck, it was a good idea. Now, not so much. Even ARMs which are considered "RISC" are growing into uop-decoding decoupled OoO machines like x86, and adding more instructions with each new generation. I like to mention this article, where the "purest" RISC, MIPS, turns out to be the least power-efficient:
http://www.extremetech.com/extreme/188396-the-final-isa-show...
I'd say that CPUs are certainly getting more high-level and CISCy, but not in the same way as their original proponents imagined. If you don't think so, try beating POPCNT, CRC32, AESENC, etc. with a sequence of simpler instructions...
* mips didnt have predication to begin with
* arm didnt have delay slots to begin with
* and then afaik 64b arm uses more registers and removes shifter operands
So definitely more mipsy.
That plus the earlier period in VLSI when there was an extreme premium on the number of gates you put on a single die. If you could fit an entire simplified CPU on a small die you could win big, and this advantage continued as FPUs, cache, and more and more cache were added.
Nowadays gates aren't exactly free, but dies are constrained by other considerations like power and size (yield) balanced against what more gates will gain you. Outside of tiny microcontrollers, I don't think any one is obsessing over the issues that limited the 1978 8086 to 29,000 transistors, or the 1979 68000 to 68,000 (https://en.wikipedia.org/wiki/Transistor_count).
Basically, RISC at some point meant {reduced instruction set} computer, now it means {reduced instruction} set computer.
well, there is this: http://www.micron.com/about/innovations/automata-processing
"Alan Kay: [...] "Just as an aside [...] a benchmark from 1979 at Xerox PARC runs only 50 times faster today. Moore’s law has given us somewhere between 40,000 and 60,000 times improvement in that time. So there’s approximately a factor of 1,000 in efficiency that has been lost by bad CPU architectures.""
Is he complaining that machines today aren't a bunch of micro-coded, multi-chip computers like in the good old days, or does he have a better proposal to the modern multi-core, superscalar OoO processors coupled to GPGPU multi-threaded engines?
Because complaining about Moore's Law feels a bit meaningless if DRAM is still 100ns away.
You need to be an Alan Kay to utter such bullshit repeatedly and still have crowds of worshippers taking each of your utterances as gospel while chanting that you singlehandedly invented computing as we know it.
I'd rather point the blame on software: If we were banging on hardware directly like in the 70s, we'd probably see a massive performance boost, too… just hope that your customers accidentally happen to have the hardware you optimized for and don't want to run anything else on it.
This isn't theoretical, I've done it personally and this is just one core.
So the claims are anything but outrageous although that is basically what GPUs are - transistors going to actual FLOPS instead of dealing with memory latency.
Intel cores however are structured like this because most code isn't written with cache locality in mind, they are written in god awful pointer chasing spaghetti styles so all the work that Intel puts in to make that go fast pays off.
Skylake almost had SIMD scatter and gather, which could have been an enormous boost to properly written software, but they had no reason to put it in when they are curb stomping AMD on one hand, and very little software takes advantage of the SIMD units they have now on the other.
Of course, there are a couple of applications where FPGAs are actually faster than CPUs.
Source: I'm an FPGA developer with a software engineering background.
Yep. At a tiny price of tenths of W of TDP.
FPGAs shine in communication, not in computing. Fast isolated memories, wide channels (as wide as you like), etc. And there are far more tasks where you have to juggle gigabytes of data as fast as possible than the tasks where you have to maximise FLOPS.
1) FPGAs are much slower due to the overhead of the flexibility they provide. With a lot of effort, you can get some FPGA designs (not even all of them!) to maybe 400 MHz on the more expensive FPGAs, and then it will do much less per cycle compared to an ASIC (CPUs are ASICs), so you need deep pipelines, with all the cost, complexity, and restrictions this entails.
2) CPUs have a lot of memory on-chip in the form of cache (several MiB), and they have sophisticated prefetch logic. In FPGAs, memory comes in several 100 blocks of, say, 4 kiB (depending on FPGA model). All memory management is purely manual.
3) CPUs tend to have a much higher external memory bandwidth.
4) Complex operations that are not directly provided by the FPGA fabric, like floating-point arithmetic or large memories with more than two ports, are expensive in terms of area and performance.
5) Modern desktop CPUs are incredibly well optimized: high clock frequency, out-of-order execution, superscalarity with several execution units, SIMD, automatic cache management, branch prediction, ... It's just very hard to beat that, and this is only possible for certain, very restricted applications. Highly parallel DSP (Digital Signal Processing) with integer/fixed-point arithmetic comes to mind.
One of the biggest leaps in general computing performance in recent years have been the general availability of SSDs. SSDs makes you computer significantly faster, but not by changing anything about the CPU.
We have glaring inefficiencies in every level of the software development and runtime stack. I have been writing software for 20 years. The sluggishness of the software, difficulty of writing and debugging has remained constant. Creating a modern Boostrap singlepage app is as difficult and time consuming as the old MFC 4.0 stack. Similar speed of delivery too.
Hardware is fast enough. We just don't bother using it.
Omitting the three orders of magnitude in performance due to sheer brute cost is how you get to "only 50 times faster today". There's an additional order of magnitude of overhead in Squeak's bytecode-interpretation strategy, which was a good strategy on the Dorado but not on an AMD64; you can squeeze that out with JIT, and you get to four orders of magnitude discrepancy between Kay's offhand claim and reality. If you build a 500-node cluster and use a modern JIT compiler on it, the "efficiency that has been lost by bad CPU architectures" evaporates.
As you implicitly point out, though, this is parallel performance, not serial performance.
Kay's group has done a substantial amount of awesome research since he said that in 2007, dramatically exceeding Kay's expectations for performance on modern machines. To a great extent, Kay's Communication Design Group (funded by VPRI and SAP) is the closest thing we have today to the 1970s Xerox PARC, pioneering new enabling technologies and interaction paradigms.
They were expensive to design and run (traded power dissipation for speed), and eventually CMOS caught up with ECL.
It's still fairly young technology but there has been work on processors for HLLs. The one above (if my understanding is correct) is effectively a combinator processor, and its input language is a subset of Haskell.
(and now I notice it's already linked in the article comments, but I'll keep it here for those interested.)
I'd be inclined to think that the more general machine Yosef wishes for would come with time and more languages implemented. Heck, maybe restricting the kind of languages best supported is a necessary feature - you can't say that von Neumann doesn't best support imperative & procedural languages after all.
...which doesn't make much sense, because the benefit of small but powerful multicycle instructions is precisely so you can spend the time decoding and executing them (and less time fetching them) while the CPU is waiting for some memory access. One factor that many people seem to forget is that instructions also need to be fetched from memory, and the less bandwidth that takes, the better.
This way hot spots can be optimised (even in a JIT engine in runtime) into hardware circuits.
I played a bit with some crazy ideas like mixing HDLs with a low level C code (see the Mandelbrot example here: https://github.com/combinatorylogic/soc/tree/master/backends... ) - this sort of things can become practical with a tighter integration between a CPU core and FPGA fabric (in my case it was a soft core, of course).
So if us software engineers don't know the hardware side well enough to design hardware, I'm not really sure electrical engineers understand the software side well enough to design hardware, either.
Not saying there aren't EEs that understand software. Just saying I've never met any.
EDIT: wait, I know one EE who uses source control. I finally got my wife to start using Git. She kept looking over my shoulder at home, asking me "what is that?" pointing at my changelog in SourceTree. "Oh, you know, it's just an annotated record of every change every member of the project has ever made, including when they did it, in such a way that we can share these changes with each other without having to have long discussions about which particular folder on which particular network drive is the 'latest' code. Oh, you think that would be helpful in your work? Who would have thought?"
1) Of course, anecdotes don't prove the general case.
2) There's some conflation here between knowledge of toolchains and knowledge of software. Arguably, they are not the same--they might be orthogonal domains of knowledge. Knuth (who knows a lot about CS) does not use email, and I would be surprised if he used git for his books.
That's one of the things the Reduceron (mentioned elsewhere) does.
The talk is on the Day 2 track here: http://www.hotchips.org/archives/2010s/hc26/
Also what about FPGAs? If compiling a specific program down to an FPGA is possible, what kinds of optimizations can we bring in? (I am indeed waving my hands here, but we already have had genetic-proframming optimized FPGAs for over 15 years in research.)
I wrote tfa to explain some of those reasons and nobody seems to read it, they just keep saying what they always do.
Someone is wrong on the Internet!!
Not to mention numerous DSLs (also pretty high level) running on GPUs (e.g., I'm running a derivative of the OpenSCAD language on a GPU, for a fast and precise slicing). As HLL as it gets.
FPGAs blur what it means to "run a HLL", since you can optimize the circuitry to a point where there is no explicit representation of the "code" or "microcode" anywhere so the question becomes pointless. Heck even what part of physics is used is blurred! [1,2]
[1] https://en.wikipedia.org/wiki/Evolvable_hardware (Ref: Adrian Thompson's work)
[2] http://www.damninteresting.com/on-the-origin-of-circuits/
As long as we're talking about alternative hardware architectures that can enable computation speeds otherwise not possible, we ought to include parallel computers, GPUs, FPGAs in the mix. The connection machine doesn't count?
Why exclude FPGAs? As long as I can have a program that will reprogram an auxiliary FPGA to do what I want more efficiently, it counts as an alternative processor with a potentially odd architecture where a gate acts as a switch in one program and works as a memory bit in another. They are named "programmable" for a reason.
i don't see the logic here.
In practice memory lookup is O(1) but there is hard limit to the amount that can be used. Except of course this is not true - there are costs to using more memory - because of the cache hierarchy.
https://people.mpi-inf.mpg.de/~tojot/papers/TowardBetterComp...
Do we need to qualify this with saying over 2^64 bytes? What is the mose memory you can get in a machine with current hardware and constant latency? Just goes to show how much more important the space effect is vs. the addressing effect.
No doubt if we tried to have 2^64 bytes of memory we would end up with another layer of cache or 2.
Essentially, the theory goes that the amount of data you can cram into a spherical volume of space scales in proportion to its surface area, IIRC with a density on the order of bits per Planck area.
https://en.wikipedia.org/wiki/Bekenstein_bound
https://en.wikipedia.org/wiki/Holographic_principle#Limit_on...
Sure, though I'd have said "RAM". When you have a bound on information density, you naturally get a bound on worst-case memory latency provided you obey a fixed speed of light.