John C. Dvorak on Intel's First Neural Network Chip in 1990 | Hacker News Reader

John C. Dvorak on Intel's First Neural Network Chip in 1990

thechipletter.substack.com

95 pointsklelatti73 comments

Great! I was trying to remember the model name of this processor the other day in a conversation, and no amount of googling could bring it up, all my results were swamped by all the new AI-du-jour chips everyone is working on. The young person I was talking to could not believe there was a neural network chip in the 90s and looked at me like I was having some kind of memory issue.

I remember the 80170 just not getting that much attention when it came out, at the time I was just out of high school, hammering away on Turbo C++ and working through the third volume of Parallel Distributed Processing and thinking to myself, "someone should make a chip that does this" and then eventually reading about the 80170 having come and gone. I vaguely recall a demo where they would quickly flash NIST digits at the chip and it could recognize them with high speed and accuracy.

I think what maybe killed the effort was the extreme pace of Moore's law at the time, made a NN chip more efficient than software? No biggie in 8 months we'll just double the density and speed of the current x86 chip and beat it in software. There's probably also the extreme internal competition in Intel to kill anything that might even vaguely threaten x86.

Anyway, good flashbacks. I should dig up my copy of PDP, I remember skimming over the second volume on biological models and I think at this stage in my life that might be the most interesting.

> extreme internal competition in Intel to kill anything that might even vaguely threaten x86

Actually, Intel has tried pretty hard to kill x86, multiple times, and failed. Most recently Itanium, but before that iAPX-432, and a couple others. New grand architectures to replace crusty old x86. But IMHO they were too academic. As you suggest, when an x86 is cheaper _and faster_ and compatible with existing software ... good luck replacing that.

iAPX-432 was supposed to be the last ISA Intel ever made, but it didn't quite work out that way [1]:

> Gordon Moore believed Intel’s next ISA would last the lifetime of Intel, so he hired many clever computer science Ph.D.’s and sent them to a new facility in Portland to invent the next great ISA. The 8800, as Intel originally named it...

> This ambitious project was alas several years late, forcing Intel to start an emergency replacement effort in Santa Clara to deliver a 16-bit microprocessor in 1979. Intel gave the new team 52 weeks to develop the new “8086” ISA and design and build the chip. Given the tight schedule, designing the ISA took only 10 person-weeks over three regular calendar weeks...

> Intel’s original 8800 project was renamed iAPX-432 and finally announced in 1981...

[1] https://dl.acm.org/doi/pdf/10.1145/3282307

> Gordon Moore believed []Intel's next ISA[] would last the lifetime of Intel

> deliver a 16-bit microprocessor in 1979. [...] the new "8086" ISA

> Intel's original 8800 project was renamed iAPX-432 and finally announced in 1981

I mean, he wasn't wrong...

Yep, he was spot on. And crazy to think that they only spent 3 weeks designing it.

I don't know if they were meant to 'kill x86' or just support a different market.

Windows NT was initially developed on the VLIW Intel i860[0]. It was also used in the NeXT Dimension and SGI Reality Engine graphics systems, and Stratus used it for some of their super-reliable systems.

There was also the i960 which I think was mostly used in embedded systems[0] including the early F-22 flight computer which had 35!.

[0] https://en.wikipedia.org/wiki/Intel_i860

[1] http://www.righto.com/2023/07/the-complex-history-of-intel-i...

Not sure 'academic' is the right word; IMHO it's a market issue. Users care about software, not architectures. This business is littered with the corpses of companies with 'better' in every way architectures, when the vast majority of folks that write checks are saying 'yeah, cool, but I just want it run Excel the same way my current one does but a little faster'.

"academic" as in "theoretically we can have object-oriented instructions in the CPU!" ... "theoretically the compiler can do all the instruction scheduling!" ... in practice oh please no

> The young person I was talking to could not believe there was a neural network chip in the 90s

This wasn't the only one, there were others. There was a startup here in Oregon called Adaptive Solutions that was working on a NN chip called CNAPS[1]

[1] https://proceedings.neurips.cc/paper/1990/file/beed13602b9b0...

Whenever I run a training session on the history of ML people are shocked to learn when the maths were first developed for things that are reported as new on TechCrunch.

> Great! I was trying to remember the model name of this processor the other day in a conversation, and no amount of googling could bring it up, all my results were swamped by all the new AI-du-jour chips everyone is working on. The young person I was talking to could not believe there was a neural network chip in the 90s and looked at me like I was having some kind of memory issue.

And if you move from "chip" to "special purpose hardware" you can go back to 1958 with the Mark I Perceptron (Assuming "neural net" requires a hidden layer, it qualifies).

this of course demonstrates the limit of LLMs. without some kind of temporal/spatial knobs, they'll overwrite similar but distinctly different datasets.

I used to enjoy reading John’s columns back in the day. I guess I always wondered in the back of my mind whether he was related to the keyboard layout Dvorak. Turns out he’s his nephew according to the article. Now I’m wondering if they are both related to the famous Czech composer.

Anyway, it’s pretty wild for me to think of a fairly plausible alternate history that could have developed if, say, Andy Grove had an unshakable belief in the power of neural nets in the late 80s and 90s and was willing to continually reinvest Intel’s huge x86 profits in that direction, developing TPU-like ASICs until they started working well for real world applications. We might have ended up with something like Llama2 7B 15-20 years ago (although inference would probably be painfully slow, like 5 tokens a second), or at least Whisper.

You can only short-circuit technology trees so much most of the time. Storage/networks/etc. probably weren't up to the task that early.

We’ve had servers with hundreds of gigs of RAM for a long time, although they were obviously expensive back then. I really don’t see a specific reason why we couldn’t have done it then from a hardware standpoint. I think it’s really because of the AI Winter and Minsky’s book that showed a simple perceptron couldn’t handle XOR that made the whole space seem like toxic waste to VCs and corporate tech executives of the time.

> Now I’m wondering if they are both related to the famous Czech composer.

A quick search didn't turn up anything, but given the location and the Czech name, I suspect a relationship could be found at some distance.

Interestingly enough, I learned the composer was a train-spotter (in modern terms, a railfan).

When I hear the name "John C. Dvorak" I think of his short column from the late 90s in PC Magazine about texting.

Years before smartphones, feature phones could send texts just fine, but no one really used that.

Dvorak was frustrated that he sent his father a text from the tarmac but the party on the other end never checked or responded to it. We just thought that sending texts on the TALK PHONE was corny and awkward. It was for the weirdos.

Would be great if this message in a bottle was online, but I could never find it.

> feature phones could send texts just fine, but no one really used that.

By "no one" you must mean everyone not in high school. Texting was a huge cultural thing in the US even before smartphones.

Even before phone texting, the cool students were abusing the "call back" number on pagers.

     GRAND CENTRAL
    HACK THE PLANET

I heard recently that the "text to vote" for American Idol was actually a marketing push from cell carriers to get kids to text. Before that it apparently wasn't very popular, and it's one of those "If you get them to do it once, they'll do it again" tech thresholds.

Also around the same time, Twitter came out and had the gimmick of being able to text-to-post.

That was not a gimmick, that was their core feature at the time. You have to remember that Twitter launched before mobile internet (cell or wifi) was common place, and the fact that you could link your number turned every dumb flip phone into a twitter capable device. Sure by 2010, smartphones had taken over, but for 3-4 years that was one of the main ways people interacted with Twitter.

I meant gimmick in the sense of a unique feature that distinguished it from all the other message boards and social networking things that existed at that time. Nobody I knew used texting to post (because it was expensive in those days of charging per character), but it got a lot of press for being able to post from your phone.

I was in high school in mid to late 90s. I had a tiny Samsung that to this day I miss with its tiny form factor - smaller than a Razor. It definitely had texting abilities, but I was a total hermit and had no friends back then, so I don't know.

Texting also became expensive once the carriers figured out people wanted to use it. I remember being pissed off in the mid-2000s when I posted an item for sale on Craigslist and someone texted me about it instead of calling - that text probably cost me a quarter!

(Now, of course, I can't imagine calling a random person or putting my phone number on an item listing.)

16-year old me started a head start program so I was driving to the next town over and taking courses at the community college in addition to my high school stuff, so I was given a cell phone for eventualities.

Which I naturally used to rack up a $90 bill in SMS charges in the first month alone. My father was rather displeased and it took me a fair few mowed lawns to pay that off.

I really miss being a kid and reading JCD.

There were some really good magazines for learning about tech back in the day.

Interesting. I remember the i860 getting a lot of press during that era (maybe just a bit later) as an alternative architecture for Intel (that also didn't end up going anywhere). I don't recall hearing anything about this 80170 chip.

Back in 1990 I worked for an ASIC company and there was a local startup here in Oregon near Intel called Adaptive Solutions [1] that was working on a NN chip called CNAPS[2] - they were coming to us to fab it. They only lasted about 5 years IIRC before going under.

[1] The founder was Dan Hammerstrom now at Portland State University https://www.pdx.edu/profile/dan-hammerstrom

[2] Back Propagation Implementation on the Adaptive Solutions CNAPS Neurocomputer Chip https://proceedings.neurips.cc/paper/1990/file/beed13602b9b0...

A few people mentioned they miss his writing, so i thought it was worth pointing out he still writes occasionally: https://dvorak.substack.com/ though it's less technology focused..

Used to like Dvoraks articles and Cranky Geeks back in the day.

Unfortunately he joined up with 'v-jay' Adam Curry to do a podcast. That show is basically Alex Jones lite. For example laughing about the white supremacist terrorist killing of Heather Heyer, edge lording it didn't happen, "Crisis actors" etc.

https://en.wikipedia.org/wiki/Charlottesville_car_attack

Ugh.

I would post the link to the episode, but don't want to give them any traffic.

> That show is basically Alex Jones lite.

Yeah that's not far off, but I think JCD is kind of captive to that. Curry has always been a whack job to some extent but kind of amusing when he's not laughing at terrible shit like you've pointed out (which I missed but don't doubt).

Here is every mention of Heather Heyer on their podcast with recordings and transcripts.

https://nasearch.online/search?page=1&q=Heather%20Heyer&sort...

You mean the case where a 4 foot 9, 330 lb woman had a heart attack on the sidewalk, and not a single molecule of the driver's car ever touched Heather Heyer? While every single one of the people actually hit by the car, survived?

I admit I don't know what Dvorak said about it, however.

> not a single molecule of the driver's car ever touched Heather Heyer?

I thought the medical examiner said otherwise: https://wset.com/news/local/medical-examiner-heather-heyer-d...

Weird detail to obsess about, since any death caused during the commission of a crime results in a felony murder charge anyway.

It depends on the state, Virginia definitely has felony murder. But the guy was just charged and convicted with plain old first degree murder. Intentionally driving your car into a crowd would seem to make that an open and shut case.

The felony murder statue would apply if say, he had some accomplice, who also committed a separate felony during the commission of the crime (like for instance, they had stolen the car), but it couldn't be proved that he was also complicit in driving the car into the crowd. But since the indirect cause of the death was the accomplice's aid in stealing the car, he could get charge with felony murder.

Not sure how the car touching Heyer is relevant. If the guy hit something else which then hit Heyer, his actions caused her death the same as if he had hit her himself.

Even if its something like, causing me to jump out of the way and hurt myself in the attempt, you bet your ass you can be held liable.

Nothing struck her and she didn't move and strike anything.

> Nothing struck her

He hit somebody, who was it? https://wina-am.sagacom.com/files/2018/12/WINA-car-in-eviden... (from https://wina.com/news/064460-james-fields-trial-every-pictur...)

The coroner's report is available: https://bloximages.newyork1.vip.townnews.com/richmond.com/co...

in which you might note the cause of death is "blunt force injury to the torso" and the details include "complete transaction of the thoracic artery".

i went and googled the link. Here is the link for those wondering

https://www.noagendashow.net/

John C Dvorak's pet peeve of the day! (Wokeness, probably)

Check out Cranky Geeks on YouTube, classic John C.

Used to love him on This Week In Tech back in the day, as well. They were trailblazers of the medium who just fell off for whatever reason.

I think there's maybe a little too much credit being given to JCD here, this quote:

> Don't be left out. There's something big lurking beneath the surface of this technology.

Is about as generic of a "wooo new tech keep reading to maybe learn more one day" as I have ever seen. That type of quote wouldn't be out of place on a hyperloop article.

That was a very early chip!

MathWork’s Neural Network Toolbox for MATLAB didn’t come out till 1991.

It didn’t use that chip.

Single core CPU only back then. I don’t recall any quantitative measure of how fast it was, other than some problems took days. Every new Intel CPU generation at that time was cause for celebration: massive time savings!

MATLAB ran on all kinds of machines then, including Cray supercomputers. But I don’t know of any reports of what kind of NN speedup a Cray delivered

These were tiny tiny problems!

>> Every new Intel CPU generation at that time was cause for celebration

Fast forward thirty years and I dread the arrival of the IT guys in my office with a new tower. Between new Widows versions, slow networks, thin clients that rely more on those network, and the constant rush towards dirt-cheap equipment, I loath every "upgrade" to my office machines.

Ah, I haven't heard that name since it no longer became good business to troll Apple fans.

I just want to find some online archive of his later last-page columns, the ones with the superfluous bold for emphasis on like four words per sentence. I loved those.

JCD is a much better writer than he is a podcaster.

To be fair he's just gotten old and doesn't have his wits about him as even his writing sucks now. The NA podcast still merits the occasional listen, but Curry went evangelical and their community has become full of that particular breed of "patriot" that I don't like to associate with. I don't think they do either, but the irony (given their anti-advertising model) is that these types of folks pay their bills, so now they're stuck keeping them happy.

the show used to be good fun. general banter, silly conspiracy stuff. it started sucking well before Trump and then the election made it so much worse.

haven't listened in years but somehow i ended up with JCD as a connection on LinkedIn, so every time I log on there's some no agenda hot take in the feed somewhere.

Yeah I used to listen to them quite a bit back in the day but in the lead up to the 2016 election they became insufferable. There was one Clinton story in particular they kept bringing up so I went through their show notes to the source and sure enough they were mischaracterizing everything about it. I emailed John pointing this out and he emailed back with one of the most dismissive replies I've ever gotten.

"this is an excellent deep analysis that I credit the show for."

Wow, thanks John, I'm glad I have the show to credit for doing some basic analysis.

I called him out on having a chemistry degree but not understanding how the prospects of using hydrogen in steelmaking isn't some hippy-dippy, NWO bullshit, but actually a really awesome thing to work toward. Got the equivalent of a shrug and he told me to start my own podcast.

loved his columns when i was a kid. No agenda is obviously too much for non republicans but his other podcast can be pure gold. Especially on 2x speed

Why would a non-republican have an issue with No Agenda?

I think they mainly have an issue with idiots and hypocrites. They are fans of RFK Jr, Tulsi Gabbard, and lots of others; they also slam lots of republicans as well (for being hypocritical idiots).

He and Curry are rabidly anti-Democrat party.

To be fair, they're rabidly anti Uni-party.

Curry just seems to genuinely like Trump too much for my tastes.

He went that far? Because Trump is a noncomformist, I'm guessing?

The concepts of "Trump being a uniform piece of shit" and "Trump being the target of a weaponized judicial system" aren't mutually exclusive but he can't seem to reconcile that fact. I like to think it's because he has an appreciation for Trump's genuine skill at showmanship, but I tire of it.

There is inarguable value in the twice weekly media roundup that they do, so long as they stick with mainstream sources and stay away from silly medical blogs and Cuban ex-CIA guys. No one is perfect I suppose.

FWIW I don't think either of them are any sort of bigots, but sometimes old school chauvinists and maybe xenophobic in a way that makes you groan but not particularly offensive. I hear way worse daily on my university campus. It's the criticisms that they shy away from that make people irritated, I think.

> FWIW I don't think either of them are any sort of bigots

I just poked around at some recent transcripts and there was plenty of anti-trans stuff.

At 12-14 years old, I remember jumping straight to the "Inside Track" section of PC Mag first. One of the best in his prime, and a Patriot.

ITM https://www.noagendashow.net/

Very interesting!

The way analog chips work should approximate more closely the way brains, and nature in general, works, right?

I suppose quantum computers would be even more accurate, but we're decades away from getting any practical applications out of them. Whereas this technology existed in 1989(!).

Have there been more modern iterations of this technology, specifically in the area of AI?

...

Reading through Wikipedia on the topic, it seems like analog chips are used in neuromorphic engineering[1]. The article lists many more recent advancements. Why are these projects still seemingly stuck in the research phase?

[1]: https://en.wikipedia.org/wiki/Neuromorphic_engineering

Every time I hear John C. Dvorak I think about his emphatic prediction that the iPhone was going to be the dumpster fire of all dumpster fires and Apple will regret ever thinking they could make a phone.

I stopped reading his columns after that.

Commercially successful, huh?

John Dvorak is such a strange guy. His uncle August invented the Dvorak keyboard. He used to be a contributor to early TechTV and ZDnet productions. Now he runs a far-right podcast and Mastodon server that seems extremely popular with Neo-Nazis and other ethnonationalists...

Personally, I try not to give a shit about anything he has to say.

Your personal attacks are incorrect, I’m sure you do in fact have differences in political ideology but your dog whistles calling him a far right neo ethnonationalist nazi has no place on this website as they short circuit all potential conversation regarding this article and frankly you are othering him.

Go ahead show one of his far right neo ethnonationalist nazi posts or clips ? Citing some trolls on an obscure mastodon account does not give you cover.

My favorite part of this website is open and informed discourse. I have been schooled multiple times, I have learned things, I have had my mind opened to wonderment and amazement but these dog whistles sure don’t do it.

So please proceed, what far right neo ethnonationalist nazi posts or clips would you like to cite that emanate from him? Or is this more of “I’m calling your manager” kind of comment to shut people up and other those who do not think exactly like you?

Just a nitpick, there were no dogwhistles that I can see in the comment you replied to. Dogwhistles use coded language, not direct descriptions - the idea is that they’re only supposed to be noticed by their target audience. Calling someone “far right” can’t be a dogwhistle, and nor can claiming they’re associated with neonazis and ethnonationalists.

Just visit his Mastodon instance and start scrolling. It's extremely NSFW. I just did it, and immediately caught some sort of Swastika / Serial Experiments Lain meme.

If I visit your website and regularly encounter Nazi content, you're a fucking Nazi.

Any critical thinking and questioning the hive mind is labeled “far right” without presenting evidence of what is “far right” and “extremist” among the opinions presented by those being attacked adhominem.

Reply on news.ycombinator.com