Hacker News Reader

Comment by PopeDotNinja | Hacker News Reader

I worked on that as a QA Engineer. My favorite accomplishment was discovering that one version of a Crusoe-powered system would irrevocably hang while playing Tomb Raider 2 for between 15 and 45 minutes. It took an entire VLSI team over a week to debug that one. The problem had something to do with a voltage drop on a certain chipset.

I also enjoyed discovering that HTTPS didn’t work on Internet Explorer 5.5, jump jets recharged at 10% the expected rate in MechWarrior 2, and a certain modem worked on PCI slots 1 and 3 & didn’t work in PCI slots 2 and 4.

View on HN

I was on the "silicon level debugging" team, which took test cases like this and boiled them down to x86, then through the CMS to p95 assembler, then down to the circuit and device level to figure out the speed paths. There were definitely some crazy issues we found. Honestly, I couldn't have done that job without relying on 80%+ of my undergrad courses (Computer Engineering @ Waterloo).

The issue I remember the most clearly was one that an engineer Simon Barrett discovered: a specific sequence of MMX instructions would cross-talk (and cause the occasional bit to flip) on two perpendicular busses in the pipeline when run at a specific voltage/frequency. It sounds simple enough, but imagine trying to figure that one out.

Hah... and I thought this was weird:

https://youtrack.jetbrains.com/issue/JBR-2310#focus=streamIt...

Have you considered attaching a kernel debugger, or symbolicating those panics?

Oh my god! This is why my $4,000 laptop has been crashing!?

Only if you have a 10th generation Ice Lake, which is the 2020 Air and 2020 13" Pro. The 16" has a 9th generation Coffee Lake and does not seem to be affected, so if you have this or an earlier laptop it would have to be something else.

Read the whole thing... it's a ride. Serious WTFBBQ with a side of potato salad.

Once Apple and/or Intel notices this there will probably be a microcode update. Until then if AWS or some other host fields Ice Lake based server cores you may be able to crash the VM host by starting up a Jetbrains IDE in a VM. LOL

The parent post on Transmeta QA shows that obscure bugs causing wacky edge cases are not that rare. It's a function of the complexity of modern chips, especially the X86 lineage with its crazy variable instruction widths and legacy cruft. You can, I think, still boot 16-bit DOS and Windows 3.1 on them!

ARM is intrinsically less prone to this due to simpler pipelines and less cruft, but ARM64 chips are still pretty complex. Any real world chip that has been around for a while is going to grow some hair on it. ARM has some shag but X86 is hairier than a wizened yak.

RISC-V is clean... for now. But there are no really high performance RV chips.

I wonder though... do they not fuzz chips? How do these things escape QA at a serious experienced house like Intel? I can understand obscure security issues slipping by but an a hard crash?

>Read the whole thing... it's a ride. Serious WTFBBQ with a side of potato salad.

Did they rule out GPU driver crashes? They said it also occurs in VM, but didn't specify whether they had any guest integrations enabled.

Also happens on the Surface Pro with the same chip, so unlikely.

There was a bug in the Sony Picturebook w/ a Crusoe in it, I don't remember what the bug was. To isolate it, I drove over to Fry's Electronics and replicated the same exact bug using the Intel flavored Picturebook on the display counter :)

I wonder if Rocket Lake is going to have that bug as well.

As somebody who owned both a Crusoe-based and an Efficeon-based machine, let me thank you and your colleagues for the great work you did. I never suffered any stability issues with these machines running various Linux flavours (mostly Debian and Ubuntu).

With strange failures like that, I'm surprised the system even booted at all.

One thing to remember is that this dropped circa-2001. The dominant Windows versions were ME and XP.

Both of which featured driver models of... not too much system protection.

So applications (especially games) looked a lot more like their console cousins, in that they took advantage of all kinds of weird hacks and did things 1,000 different ways.

As opposed to now, when you have more strict intermediary layers, and software sits on top of more defined interfaces.

So things like "the MechWarrior developer in charge of jump jets decided to base timing off the PCI bus speed" or god knows what, isn't that surprising.

Pope was a QA engineer - that means he worked on systems that weren't qualified for shipping. Actual CMS bugs in a shipped device was extremely rare, but the big promise of Transmeta was that would _could_ ship fixes (and did, in at least one case). The perf impact of a CMS fix was typically trivial as it was fixing a translator, unlike an interpreter like the microcode in an Intel core.

I mostly worked on development boards that were the same size that you'd put in a regular AT/ATX case. I did some work on notebooks, too, both on prototypes & production models, too.

I think I know you (Hi!), but my point is that by definition, QA's work is to find the bugs before the customers, so they don't see them. Talking about bugs you find without that qualifier might make people get the wrong impression.

Hola!

The problem had something to do with a voltage drop on a certain chipset.

Did you use any specific stress-testing programs, or just tested with regular applications? Something like the LINPACK benchmark is extremely stressful on the CPU and memory system, and is commonly used today for system stability testing.

I personally just ran whatever I felt like running. We had testing scripts to follow, but I largely ignored them XD My super power was testing things that other people wouldn't normally test. I was good enough at breaking things in unexpected ways that I mostly got away with it.

That sort of detail is fabulous, and reflects really how little we've progressed when it comes down to "brass tacks". Thanks for sharing.

Thats why I was always skeptical of the Transmeta CPUs. They could 'run' the x86 ISA as its on paper, but they'd never be able to completely match all the nuances of the Intel CPUs to get 100% compatibility. The devils in the details as they say.

This statement is pure FUD. There were no meaningful quality issues and the situation is exactly the same for all alternative implementations of the same ISA. You could make the same argument to say that AMD or even a new Intel microarchitecture is incompatible as doesn't have the exact same runtime behavior (eg. shifters on the P4).

The only relevant metric: does it run the applications that customers use.

The Crusoe/Efficeon issues were performance related.

Just as a single data-point: I still had an old Crusoe-based TC-1000 [1] convertible lying around, that must be > 15 years old by now. About two years back I gave it a new hard-disk (some PATA to flash adapter), replaced the battery, upgraded to 768 MB RAM and installed a Devuan i386 OS on it (a Debian derivative). Even was able to make the pen work again (that needed some X-server source code hacks).

The computer is somewhat sluggish with huge software (like Firefox) and especially software that does JIT-compilation, maybe due to limited size of the "code-morphing" cache. For simple java programs the only way to make them start up timely was to fully disable the JIT-compiler.

But other than that it's a usable x86 machine (and all other laptops I know that are of similar age eventually failed to boot much earlier).

[1] https://en.wikipedia.org/wiki/Compaq_TC1000

I had a Transmeta Crusoe powered Windows XP laptop and I never had compatibility issues. Performance issues... that's a different story.

Reimplementing an ISA is not actually an impossible feat; many companies have done so successfully before.

Nothing is impossible, but has anybody done it since 2000 starting from scratch? AMD obviously has been at it from the start, Cyrix, VIA, NEC all did it but any since Intel CPUs started being in the tens of millions of transistors?

Hitachi SH-4 | 10M transistors | 1997

SPARC64 V | 191M transistors | 2001

DEC Alpha 21364 (EV7) | 152M transistors | 2003

SPARC64 V+ | 400M transistors | 2004

Cell | 250M transistors | 2005

POWER6 | 789M transistors | 2007

SPARC64 VI | 540M transistors | 2007

SPARC64 VIIIfx | 760M transistors | 2009

16-core SPARC T3 | 1B transistors | 2010

8-core POWER7 | 1.2B transistors | 2010

Quad-core z196 | 1.4B transistors | 2010

SPARC64 IXfx | 1.87B transistors | 2011

SPARC64 X | 2.99B transistors | 2012

8-core POWER7+ | 2.1B transistors | 2012

Apple A7 | 1B transistors | 2013

12-core POWER8 | 4.2B transistors | 2013

Apple A8 | 2B transistors | 2014

Apple A8X | 3B transistors | 2014

32-core SPARC M7 | 10B transistors | 2015

[insert a crapton of ARM CPUs here]

AWS Graviton2 | 30B transistors | 2019

No x86 ISAs in here. All licensed designs or first party manufacturer. I wasn't saying nobody could make anything as complex as current Intel CPUs.

Cloning other ISAs either from clean room or from licensed designs like ARM seems to be much easier than matching Intel CPUs with a massive breadth of software and less hard specifics at the edges.

I used to be skeptical of AMD CPUs & the various motherboards for a long time, too. I figured that Intel had the resources to test the crap out of my computer, and I really needed it to work.

AMD started out and get the designs directly from Intel to make 8086 CPUs on license and continued onto the 286 until Intel broke their agreement and they had to make clean room 386 implementations. So AMD has been doing this since, essentially, the beginning, and have been able to iterate tracking Intel and not starting from scratch in the 2000s with an entirely new approach. They probably could have done it, eventually, but they ran out of funding and thats the end of the road for that adventure.

Reply on news.ycombinator.com