A CT scanner reveals surprises inside the 386 processor's ceramic package | Hacker News Reader

A CT scanner reveals surprises inside the 386 processor's ceramic package

righto.com

314 pointsrobin_reala107 comments

A bit of a trip down memory lane for me. I performed an analysis of the thermo-mechanical cyclic fatigue in later packages using detailed CAD, FEA and empirical tests. A lot of work went into finding it wasn’t a big deal for the most part. Still, I don’t recommend that museums power cycle old PCs daily…

Knowing nothing about how survival/durability testing is done in VLSI: how did you do the empirical tests?

For example, I know that thermal samples for the Pentium 5-era Xeon (Jayhawk) were produced, but I'd always wondered Intel went from the dummy to realizing "oh, shit, this is going to be way too hot in the long run."

I can’t really speak to the thermals other than as an input to my work. I was narrowly focused on the cyclic loading based on the temperature gradients (etc.) I was given.

For museums, would it be an option to instead of a cooler have a temperature control unit that keeps the package at a set temperature no matter wether the PC is operating or not? Just heating the chips surfaces might be cheaper than having the full PC on 24/7 with a semi constant load.

The PC would still heat up when started. You would need very precise temperature control to avoid that. That could be quite difficult to do

With a couple of kelvins of tolerance a PID controller could handle this fine

>386 has eight pins labeled "NC" (No Connect)

and Cyrix 486DLC hijacks 7 of those :)

A20M# (F13) - when supported by motherboard you can L1 cache whole ram instead of leaving first 64KB uncached

FLUSH# (E13) - when supported by motherboard you dont have to use hacks and flush L1 on every DMA access. Hacks (BARB mode) seemed clever at the time until everyone had a Sound Blaster DMAing audio constantly invalidating cache while gaming.

RPLSET (C6) RPLVAl (C7)- L1 cache status debug outputs

SUSP# (A4) SUSPA# (B4)- suspend support, wakes on INT and NMI. Good for laptops.

>The surprising thing is that one of the No Connect pads does have the bond wire in place

Somehow Cyrix picked this particular pin (B12) for KEN# input (enable L1 cache) :O

>From the circuitry on the die, this pin appears to be an output

Meaning the _one_ NC pin Intel CPU actually wires, an output no less, Cyrix demands driven low to enable cache.

Author here for all your CT scanning questions :-)

What is your CPU's yearly deductible?

Is the CPU destroyed by the process or did you reassemble this particular specimen?

I took the metal lid off the chip to improve the scan quality. If I had left the chip intact, it would probably be fine. (I assume the X-ray levels are low enough to avoid damage, but I haven't confirmed that.)

Also 386s are very resistant to radiation, I believe they still use them on the ISS for that reason (a radiation-hardened version but still)

This isn't about CT scanning, but about the chip itself.

Since the bond wires are just hanging out in air, does this mean that a chip like this could be ruined by dropping it which might cause the bond wires to move enough to short something?

Thanks for all your hard work!

I'm not an expert in this area but I'd expect that the bond wires' mass is low enough relative to their stiffness that any shock sufficient to bend them would also shatter the ceramic package.

If the chip is subjected to a few thousand g's of shock the wires can bend and short.

This failure mode is quite low on the list among others, but it is something that people did investigate. For example: "Swing Touch Risk Assessment of Bonding Wires in High-Density Package Under Mechanical Shock Condition" https://asmedigitalcollection.asme.org/electronicpackaging/a...

Yes, that can be a problem. Story from one of my professors who worked on instrumentation and telemetry for a defense lab. They built a data recorder for artillery shells. In the early "flight" tests, the recorders failed and nobody could figure out why. They worked before and after. Then someone realized the high acceleration bent some of the bond wires in the chips, causing them to touch and short. The fix was surprisingly simple: make sure all chips face top down.

Does it look like the almost connected pins could have been purposely severed during production? ie: could they have been connected and then using a calculated pulse of power, disconnected?

If they installed wire bonds and removed them, there would be visible remnants on the die, which aren't there.

What CT scanner was used? The images are surprisingly detailed for something so small, while we are used to coarser scales of human anatomy.

It's a Lumafield scanner, but I don't know the specific model.

What is the last node/cpu that had the smallest features visible at optical microscope scales?

With a good scope we could inspect 0.35um chips just fine. I honestly didn't look at die photos much after that until we started getting SEM images of 32nm and smaller chips

Genuine question: the website doesn't work in Russia. Did you restrict the access or is it my ISP doing that? Someone tries to prevent me from studying of very niche info on ancient Intel CPUs. Thanks! P.S. Big fan of your work!

One word: VPN

That's three words

I’m upvoting you.

ya'r good boy

Or is it 5

Some smaller sites ban ips from countries that continually try to hack into your server or just make a ton of requests, it happens to be that traffic is often from Russia and China. Could just be that.

I block Russia, China, and Iran from my sites. They represent 0% of the revenue, and 99% of the login attempts.

Yeah. In the same vein I also don't distribute my app in the Play Store in certain countries. I realize it completely sucks for them, but it's purely a business decision. Certain countries are just very vocal in terms of negative reviews, simply swap 5 star and 1 stars due to cultural differences, and also bring in almost zero revenue. The net result of distributing in these countries is literally negative: they hurt my ratings and reviews and don't make up for that in terms of money.

I did find that, while running a financial startup, I was able to significantly reduce attacks on the server by disabling access from Russia and China. Not saying that's happening here, just my experience. That was a while ago so I'm sure things have changed since then.

Thanks for your reply! I hope this is the real reason of blocking. If that's not the case, that's at least not effective. Less effective than idk placing a banner in the header or whatever.

I mean I eventually read the article. Sorry for that. But we're at "Hacker News", sporting hackers ethics, aren't we?

Opposing the invasion of Ukraine and the biggest existential threat Europe's faced in a couple generations seems pretty ethical to me.

We should be jamming American media down Russias throats like we did during the Cold War.

Radio Free Europe and Radio Liberty were one of the very first things that the Trump presidency stopped.

By preventing some computer history enthusiast in Russia from reading an article on a processor from 1985? Really?

That is it was more financially effective to block an entire country, than analyzing attack patterns and blocking by ASNs or IP-ranges. Correct?

Startups don't have enough free time to analyze individual ASNs, because they don't have enough people for that.

That and financial businesses usually don't operate outside their host country anyway. Though you do want your customers to see their accounts when they're traveling.

Yes. Multiple countries.

In all fairness, this isn’t a good use of that technique. But most websites are of no interest outside a handful of countries.

It is your ISP. (don't ask me how I know but please research this before posting)

It could be this: https://blog.cloudflare.com/russian-internet-users-are-unabl...

Because the author is the opportunistic racist:

> kens on April 10, 2022

> Are you trying to access from Russia? Russia is currently blocked.

https://news.ycombinator.com/item?id=30974444

> kens on Dec 3, 2022

> Unfortunately there are also many people in Ukraine who didn't personally do anything to deserve what's happening. Consider the country filter a small reminder of the ongoing war and a suggestion that you might find better opportunities outside Russia.

https://news.ycombinator.com/item?id=33846782

Yet he doesn't consider to 'find better opportunities outside of the USA' despite the actions of the USA government in the last 30 years.

Probably your ISP, or more precisely, the ТСПУ box they were required to install. You can use this tool to test your connectivity to these hosting providers that the government dislikes: https://hyperion-cs.github.io/dpi-checkers/ru/tcp-16-20/

Ken himself did block access to his website from Russia for a while after 24/02/2022, but right now it loads for me after a CF captcha.

What kVp/mAs do you use for this? How are you avoiding the artifacts seen from medical imaging? Curious, in school for CT in the medical field.

Lumafield does all the work; I just get the images :-) The data says 130 kV, 123µA. The whole scan took 21 hours: 1200 projections of 60 seconds each. I assume that they avoid artifacts by using a whole lot more radiation than medical imaging would permit.

I’m assuming each image was taken with 123 microamperes? Or is that total dosage over the 21 hours? If it’s total that’s much less than medical dosage, but if it’s per image that’s a lot more!

Thanks for the info, how interesting!

(for those who don’t know, mAs = mA • seconds = milliampere seconds. It’s how Radiographers measure how much x-ray photos are being produced by the tungsten filament in an X-ray tube. kVp is kiloVoltage potential and it’s how we measure the speed and thus the penetration power of the X-rays. 130kvp is slightly more than the 120kvp used for an avg human chest radiograph)

Any rational argument to use mAs instead of mC?

I asked Jon Bruner at Lumafield for details:

The scan was performed on our Neptune Microfocus scanner, configured with a 130 kV source. Current varies on this source depending on scan settings; in this case 123 µA. Each voxel in this scan is 12.8 microns; for smaller parts that we're able to move even closer to the X-ray source we can achieve 3-6 micron voxels.

Compared to medical CT scans, this is much higher resolution--medical CT scans have voxels on the order of 0.5 to 1 mm! This is possible because we're able to apply much higher X-ray doses in industrial scans. Medical CT scans are typically on the order of 120 kV, at higher current but for much less time--perhaps a few seconds compared with minutes to hours for an industrial CT scan.

> From the circuitry on the die, this pin appears to be an output. If someone with a 386 chip hooks this pin to an oscilloscope, maybe they will see something interesting.

Would be a fun surprise if the 386 had its own Halt and Catch Fire mode.

It had ICE mode (precursor to SMM, but used for production testing and low-level debugging), and according to this article, the pins were exposed at least on some of the chips you could buy:

https://www.rcollins.org/ddj/Jan97/Jan97.html

>On the standard Intel 80386 DX, asserting the undocumented pin at location B6 will cause the microprocessor to halt emulation and enter ICE mode.

[this is written from an ICE perspective - for "emulation", read "normal operation"]

This mode was introduced in the 80286, but I don't think the pins were exposed except in the special bond-out variant for ICE, and maybe early samples. You can trigger it in software (opcode 0F 04 on the 286, or by enabling a bit in DR7 on the 386), but then the processor disconnects from the bus and you have to reset it.

On the 286, you can get it to dump some otherwise hidden internal state, by using a prefix that no longer exists on 386s: https://rep-lodsb.mataroa.blog/blog/intel-286-secrets-ice-mo...

Went to a computer fair circa, gosh, 1989? My Dad bought me a 386 DX 25MHz with like 4MB of RAM and a whopping 40MB hard drive. This was a remarkable upgrade from the Tandy 286 16MHz that I was using. The 386 we got was not the standard 20MHz or 33MHz, 25MHz was some kind of hype thing, as I recall. The 33MHz was the bomb, but of course that cost more bones $$$$. The computer fairs were cool.

For 89 that's screamin! I remember early 90's getting a 50 mhz 8mb Gateway and it was amazing. Even just MS Paint and MS Word kept my sister and I plenty entertained making up stories and pictures to go along with them. Then I found MS DOS and QBasic and here I am posting on hacker news on a Saturday afternoon.

My father bought and built my first PC with an AMD 386DX40 in 1991 . I have good memories from these computer, and from the Spectrum +3 that He bought a year prior.

I'm just glad someone is putting hybrid packaging information in the public domain. The generalized background information is really helpful for engineers new to this very small area. This wiring is not as complex as the old military hybrids for sure. It may be six layers but there is only one monolithic.

The anecdote about the 16-pin religion and the reluctance to use more pins is so good. It's often assumed that (later) successful companies were always making fantastic decisions in the earlier days, when in reality there were a few bizarre and harmful assumptions that were holding it back and needed to be forced out in order for rationality to prevail.

To be fair packaging used to be very expensive in US. I remember one of Asianometry? videos touching on Japanese businessman traveling to Texas in ?seventies? and learning how expensive lead frames were while he could manufacture and ship them overseas at fraction of the cost. Sadly I cant find that specific episode anymore :(

I think it might be this one ?

https://www.youtube.com/watch?v=nNpuiJitKwk

This one is about history of packaging technology. The one I remember was about particular Japanese company, similar to "YKK: Japan’s Zipper King".

The reluctance to use more pins is very understandable.

At the time, Intel was primarily a memory manufacturer, and they had vertically integrated the complete workflow for anything that could fit into a 16-pin DIP. Anything that didn't, required them to outsource testing and packaging, or purchase expensive new machines. When CPUs were still being pushed against the wishes of upper management ("A computer has only one CPU but lots of memory chips, so the memory is a better business"), it was a hard sell to invest lots of money for an uncertain market.

That lower level "Signals" CT image (layer 2) would have been an amazing background for the "Intel Inside" logo stickers. It has the proper era aesthetic and everything.

Anyways.. this is what I really like about kens work.. the accidental discovery of beautiful structures while trying to answer abstract questions. Thanks for doing all this!

Where's A0 and A1?

Since the 386 is a 32-bit processor, the address specifies a 32-bit word and doesn't use address bits A0 and A1. But what if you just want to read a byte or a 16-bit word? The trick is that the 386 provides four Byte Enable outputs (BE0#-BE3#) that indicate which bytes in the word are being transferred. Of course, it's not that simple. If the lower 16 bits of the data bus aren't being used, the upper 16 bits of the data bus are duplicated on the lower 16 bits to make 16-bit buses more efficient (somehow).

Neat, saves two wires.

Yes and no. You replaced two address pins with four byte enable pins.

But the byte enable pins also implicitly communicate size, which would otherwise require another two pins. So this byte enable scheme breaks even (at least for chips with 16bit or 32bit buses).

The main goal is simplify the design of the motherboard.

Super cool! This was the CPU in my very first PC (which I got to build myself, under the tutelage of a family friend). I remember that it was cooled by nothing but a tiny stick-on heatsink and a small plastic fan that clipped on top of that.

8MB of DRAM, a 250MB spinning disk hard drive, 5.25 and 3.5 inch floppy bays, removable bios that I had to sort through a tupperware of chips to find the correct unit, some unnamed AGP video card that I had to slot removable chips into as well and a great big 16" CRT.

I think I had to install a special serial card in an ISA slot to use a mouse too.

> some unnamed AGP video card

Do you mean VGA rather than AGP? AGP came much later than the 386 and wouldn’t have been supported by its motherboard chipsets.

ISA slot, likely.

Is it possible it could’ve been EISA and that’s why it seemed different?

I can’t remember if those were available on 386s or started in the 486 era.

A Local bus ? The VESA Local Bus (VLB) was a thing in 486 boards and, I think, early Pentium boards. but was predated by privative local buses. I don't know if there was one on 386 boards.

EISA was typically reserved for higher end server and workstation type motherboards. Consumer-grade PCs never really got to experience EISA.

Probably correct, though I think I had a 486 board with an AGP slot? I didn’t have anything newer than that until the Core 2 Duo came out.

AGP came too late for the 486. It may have been VLB, a short-lived step between ISA and PCI (and then AGP).

I just looked up a few old motherboards and I think you are correct! I remembered it was a brown PCI-looking bus and incorrectly assumed the only thing that matched was AGP - the VLB looks exactly like what I remember.

Same size as a normal ISA connector but "deeper" (2 rows of contact if you could inspect them)? EISA

Full ISA connector (potentially missing the bit in the middle) and then a further piece? VLB

Shorter than ISA but higher density? AGP (it's even a bit shorter than PCI)

Was it at least a Pentium? Can't be AGP otherwise.

Going to ignore PCI-X, PCIE and obscure AGP variants

I was talking about what was on the card itself rather than the interface, because the poster I was responding to mentioned the card, but your point about interfaces is well made.

Back in the day - late 80s, very early 90s - I’d see Amstrad (ugh!) 286-based desktop systems on sale in our local branch of Dixon that included graphics cards fitted with VGA chipsets, but cards compatible with the AGP interface on then newer motherboards didn’t cross my radar until the second half of the 90s.

The VGA cards often had a mouse port, I think. I don't recall having to add a serial card on 386s, but maybe we did. IBM machines were really oddball, too, with that fancier type of bus. I stayed away from IBM.

> ... with that fancier type of bus.

This one? https://en.wikipedia.org/wiki/Micro_Channel_architecture

These old ceramic packages are to my mind peak aesthetic for chips.

> In later Intel processors, the number of connections exponentially increased.

Pedantic note: I think "quadratically" makes more sense here: we're talking about two dimensions.

If you look at the numbers, the number of pins is roughly exponential over time, increasing about 10% per year. Also take a look at Rent's Rule.

Fun fact my friend remixed MRI sounds into a track: https://youtu.be/3NbbWPSOwvE

Someone turned an MRI into a loudspeaker, played Yo Yo Ma playing Bach through it, and wrote a paper about it: https://www.youtube.com/watch?v=VYAvxe9X3s0

kens - Presumably they chose the pin assignments to make it easier to arrange traces on the motherboard side. Or did they?

That's an interesting question. Looking at the layout, my guess is that they didn't worry about the motherboard routing. I'm not sure they could do much to make it easier in any case.

How do they attach those bond wires? Seems difficult and fiddly!

with a bonding machine :) doing that manually can be tedious, nowadays for IC that are bonded and not flipchipped it's all automatic. Manual bonding is still very used in research.

friction (ultrasonic) welding

Hey @kens, congrats on the page! Extremely super small usability note/suggestion: if you changed your inputs (above the tool that lets you see all of the layers) to something like this:

    <input name="layer" type="radio" onclick="show('https://static.righto.com/images/386-package/layer0.jpg')" id="layer1">
    <label for="layer1">Pins</label>

then it would be possible to click the label name (i.e. Pins, I/O Vcc, etc.) instead of having to click the small radio circles.

It's a small thing, but I think it's a lot more fun/easy/fast to click the different label names rather than the circles. It's truly a small nit - just in case it's an easy fix for you. Cheers!

(just to make sure: you need to add a unique "id" attribute for each "input", and then make a <label> tag for each label referencing that id in the "for")

Thanks for pointing this out. I should have remembered the label tag. I've updated the page so it should work better now.

> (just to make sure: you need to add a unique "id" attribute for each "input", and then make a <label> tag for each label referencing that id in the "for")

Nesting the <input> inside the <label> is simpler. Then you don't need the id and for attributes. I think it avoids an unclickable space between them too.

Reply on news.ycombinator.com