Comment by ggm | Hacker News Reader

Comment by ggm | Hacker News Reader

Parent Full thread

I have a personal dislike of special purpose hardware. I didn't really realize how much I disliked the idea, until I woke up to GPUs and off-CPU processing and the complexity of dealing with this "thing" bolted onto the side of a beautiful ISA, to do "other things" -which of course is entirely normal, it's always been so, Bell and Howell discussed a wheel of life about the smarts to run a printer in mainframe days, winding up with the printer moving to its own dedicated CPU and being networked and then you repeat the wheel over the network controller.

I first met this with a floating-point systems co-processor on a Dec-10. It was highly problem specific, only a single DECUS tape FORTRAN compiler talked to it (IIRC) and you had this giant freezer sized box (it was lime green, unlike the blue DEC-10 livery) sat next to the CPU doing... mostly nothing. It was basically idle almost all the time. Occupying floorspace, consuming power, making the DEC-10 more complex to operate.

Same-same with 3DES processing cards, on-card ethernet TCP, you-name-it -these things absolutely do improve the world, but at a cost of complexity.

So the idea of designing optical interference/coherence/diffraction "engines" to bolt onto the side of a beautiful ISA, and winding up with something morally like a GPU which has to do complex TCAM like memory, wierd protocols on the bus, all to get what? some context specific speed up of .. text processing?

If this winds up down in the VLSI level on-chip, as an alternate compute element inside the ALU/ISA space, with its own bus path, and simply integrates more tightly into the ISA I'd be fine.

Perhaps thats where it's going.

Wait, so your thesis would've led to no FPU, GPU, hw TCP, etc. We just slowly and linearly chug along with a classic 8086 beautiful ISA.

Well, the beautiful (not so beautiful these days) ISA is always there for you, while we grab a library to use a suitable specialized processor and run our shit 1000x faster for 1/100 the power.

yes. It's a naieve world view, I fully admit. bolt on co-processors are just a fact of life. Mostly, we now run in abstractions "above" this layer of concern.

I feel like this about the TPU and other chips bundled into phones too. I wonder if we're going to wind up with "photoshop specific" bolt on architectures.

So at what point does something become special purpose? Is L1, L2 or L3 cache special purpose? What about io chips? What about the firmware inside ethernet cards, harddrives or the video chips (not even talking about GPUs).

I'm not trying to be combatitive, but it seems completely arbitrary where one would draw a line and taken to the extreme one is essentially left with a CPU and possibly some RAM that might be some ideal computing machine, but can't even talk to the outside world.

On chip to me, is integrated. but with multi core. shared facilities on the die are a bit of a bottleneck sometimes (I don't do this for a living, thats how I understand it)

So the whole 'hyperthreading is cool, 2x the CPU' thing: it turns out that as long as your e.g. integer only, true. If you invoke FP, not true: they stall.

L1 cache in some instances is now big enough some people's code never leaves it.

Nobody who lives above a compiler should ever have to worry about this. I live above scripts. I don't really worry about this. I just hark back to when I ran these boxes, on raised floors, remembering how .. distasteful adjunct computing equipment was. It was completely asynchronous, required it's own resets, interfered with cabling, had effects on the bus (unibus!) you didn't entirely understand, they made life "harder" for operations.

Did they speed things up? Sure!

>one is essentially left with a CPU

CPU? Ooh you lucky, lucky sod. Back in my day, you wanted an OR gate you had to string some NANDS together.

Funny, in this light, GPT-3 accelerator is like strapping a cache of the entire internet and knowledge of humanity to your board.

GPT-3 might be the ultimate contender for https://en.m.wikipedia.org/wiki/Hutter_Prize - now if we could only find the prompts to recover it all

I remember a rather interesting "application specific" thing that happened with Risc PC's in the UK. Because at that particular point in history Acorn and the RISC-OS ecosystem were having to coexist with the Windows ecosystem, people wanted to run their DOS and Windows applications on their Risc PC's. The 80386 and 80486 were bolted onto daughterboards and sold as special-purpose add-on processors, just to run those applications.

I wonder how you feel about secure enclave chips

There a sort of necessary evil. They definitely complicate things, look at the debates over Tannenbaums work writing the os for secure enclave bios interactions. And, like protection rings they come with questions about bugs and subverted state.

I have worked on systems doing HSM like activity which depended on vendor specific TPM. They're a pain.

I think the better way to look at it is that the norm is a very heterogenous architecture with all sorts of purpose built accelerators. However, while Moore’s law was in play the general compute ate all the accelerators (hence integrated fpus, simd, etc). Now that it’s dead, we’re going to go back and this is likely the new normal unless we somehow unlock a new era of exponential growth (eg optical compute). However, at this time that seems less likely so we’re likely to see more of these accelerators. The challenge of course is scale - specialized accelerators by definition have smaller market niches which in turn limits how cheap they can be and the creativity of what people think to do with it (imagine what computing would look like if only supercomputers had floating point units to support numerical simulations).

Amdahl's law always resonated for me for this reason: you only gain speedup equal to the portion of the total work that can be accelerated.

From that perspective, speedup is less about hyperefficient design of a brand new thing, and more about recognizing large portions of existing workload that are amenable to acceleration.

More successful accelerators have been borne from the latter approach (superscalar/hyperthreading, GPUs once OpenGL/DirectX dominated, fixed function video decode hardware) than the former (Itanium/VLIW)*.

Or in simpler form, never start a value proposition with "First, rewrite all your software..."

* ML is an odd duck, as it's somewhat co-evolving with its own accelerators?

Even a single CPU operating like that is an illusion. Your ISA is controlling all sorts of stations within the CPU and getting good performance often means understanding hazards between the parallel processing occuring across different pipeline stages and stations.

This is very true. I never much liked #pragma and special-purpose hints in the compile cycle but they're a fact of life.

I am not in this deep. I am basically howling at the moon, at this point.

General-purpose hardware can never be as efficient as special-purpose hardware, unfortunately. All computation has a structure to it that is defined by the way the data flows into the operations, which in turn suggests some optimal physical organization. So you first perform the computation on the general purpose hardware you already have, until you have to do it so often it becomes advantageous to design and build a more optimized structure for it.

If you're looking at deep learning, the end game is probably hardware a bit like the brain: a hundred billion of specialized, low-clocked coprocessors crammed in a small space with only local communication to their neighbours.

Isn’t the solution better “mechanical” and language integrations for “off-cpu compute”?

CUDA and subsequent libs arguably provided this and caused a pretty massive acceleration in the amount of GPU-utilising code that got written. Now, CUDA might still be pretty awful to write, but if general-propose-GPU API’s continue to evolve and refine, we might be able to get to a point where something like an LLVM compiler can auto generate efficient GPU code for matrix ops, much like how it can auto-vectorise certain patterns now.

Neither of the main GPU makers have any motivation to support this, that I can see but I think if it happened it would be a very good thing.

I think AMD and Intel have encouraged this with opencl and other efforts. Nvidia hasn't and since nvidia/CUDA is pretty much the standard for ai and gpu acceleration....

"beautiful isa"? you've seen x86, right?!

Nothing can replace the wonder of the DEC-10 instruction set. This gem from a commentary to the instruction set manual:

"...By the way, for the sake of consistency, AOBJN should have been called AOBJL and AOBJP should have been called AOBJGE. However, they weren't. ..."

I don’t understand this “stick to your guns(CPU)” mentality.

With Moore’s law dominating any ideas about software optimization, the mantra of “wait until the CPU is twice as fast in two years” has dominated the industry for long enough.

Not everything needs to fit in an idealistic CPU and we should allow for more complex computational machinery.

Let’s allow for new hardware, new programming paradigms, new ways of thinking.

This is kind-of a-historical. It worked, as long as Moores law worked. The special purpose h/w bought speedup for "now" and within a tick-tock cycle was overtaken by general purpose CPU, unless there was countervailing investment in that special purpose hardware: which happened to be basically the GPU and almosty nothing else.

FPGA's and ASIC were tuned to do highly specific tasks, and then wound up being much the same: prove it in an FPGA, make an ASIC, then see it move on-die as the VLSI matures.

Now Moores law has bottomed out, specific purpose processing asynchronously makes more sense.

I just don't like it, the same way angry old men growl at the moon at night.

There was only a very narrow time window where a single-CPU-without-special-purpose-h/w dominated.

The 8-bit era and before relied heavily on co-processors. Sometimes they were the same or similar to the main CPU - e.g. the Commodore 64 floppy drive had a CPU as powerful as the main CPU of the machine, and you could run code on it - but it took special effort to target them. The most successful 16-bit home computers had co-processors and special purpose hw (e.g. the Amiga had it's copper and blitter, but some models also had a 6502-compatible core controlling the keyboard, and many SCSI controllers at the time had full CPU's - mine had a Z80).

The PC got cheap by relying on the CPU to be able to control most things in an age where special purpose hardware was expensive due to low volumes (the Amiga "only" sold about 5 million across multiple models), but it didn't take long before a typical PC started getting micro-controllers and full on CPUs everywhere (e.g. in your hard drive [1]) because the CPU speed as mostly not gotten fast enough.

Just for a short window of maybe a decade it got fast enough to outpace the cost of low volume special purpose HW.

[1] http://spritesmods.com/?art=hddhack

I guess you dont like arms or legs then. We should all look like amorphous gelatinous blobs of undifferentiated flesh. Look, I have my own quirks - but you gotta squash those ones that make no sense.

I think more and more Channel I/O like objects is the future. Add on processors (service processors if you may) are part of that idea.

Reply on news.ycombinator.com