Dell Tried to Hide Bad Capacitors Problem 2003-2005 (2010)
spectrum.ieee.org
spectrum.ieee.org
They lasted about 6000 hours - we could set it in a calendar and did so many RMAs. For the local 911 center, we'd replace the boards every 4 months, before the caps failed, because they just couldn't tolerate the downtime.
4 months is 2920 hours, which is less than the stated 6000 hours to failure. Though only by 2x.
People would come in saying their PC didn't work. He would pop it open, take out the motherboard, go "ah yes, puffy caps. See right there", then open a drawer full of recycled capacitors, and desolder the bad ones and resolder correct ones back on. And it worked!
The shop ultimately closed but I still think back to that and a few other tricks as especially impressive.
yeah, sounds about right. i was also working in a repair shop around the same time, and we knew that we could repair the boards for our customers. but we also knew that we couldn't get away with charging enough to cover the cost of parts and labour unless we offered a warranty, and we couldn't reasonably offer a warranty on old motherboards.
we did make pretty good money selling upgraded new computers to the people whose capacitors had failed. it always felt a bit dirty to sell somebody a new $800 laptop when their old computer was repairable, and i'm sure dell was doing even better than we were on that racket.
The fact that you "get away with it" is only possible if you offer such a needed service, that people are willing to pay.
If you want to be charitable, leave some money on the table for customers who look in dire straits. But you should not feel bad for making a profit (at least until you find yourself buying politicians).
When I was a student friends and relatives would give me (nice) broken electronics that they were going to throw out anyways, and if I liked the item (e.g. an expensive camera that I couldn't afford) I would put a lot of effort into fixing it.
> but we also knew that we couldn't get away with charging enough to cover the cost of parts and labour unless we offered a warranty
I'm not sure about that. The store charges more than enough to cover it and still gets more business than it can handle while still providing a free warranty for 6-12 months.
I.e., broadly speaking, like lightbulbs.
Only later did I learn that it actually seemed to have been industrial espionage gone wrong! https://en.wikipedia.org/wiki/Capacitor_plague
That's probably also the reason why so many retro enthusiasts just generally "re-cap" almost everything they touch. In a lot of cases, you just don't need to do that, and sometimes it could be worse to replace capacitors not affected by the capacitor plague: Any factory adjustments made to the device could well depend on the exact characteristics of the capacitors that were present in the device when it was manufactured.
This is one reason mining GPUs can be a questionable buy. The miner may have operated them at "low" temperature, but they were on 24x7, whereas a gaming PC may be only on a couple of hours a day a few times a week.
Ultimately it comes down to how much the designers overspec'd components and/or factored in for wiggle room and how much the capacitors can degrade before circuits start acting up enough for the user to notice.
Never met such a gamer
It also depends on how exactly the caps are used. Caps close to a massive heat source, like a power resistor, are likely to dry out relatively quickly, while others, like those used as part of small signal filters, may last literal decades, with not much change in e.g. ESR or capacitance (and hence still conform to factory calibration pretty okay). Admittedly I don't have any hard data for that, only anecdotical experience with measuring ESR and capacitance and saying "this looks fine".
The crucial point is that the "capacitor plague" affected much more than products which were on 24/7 or which had tight tolerances, to the point where it was very well visible to the "casual" consumer. Yes, we may not expect a server mainboard that was on for the last 20 years to still have its caps within spec, but wrt the capacitor plague, the gaming PC you mentioned would have been quickly affected, in an obvious manner, as well.
I wouldn't expect those to be electrolytic, though. Electrolytic capacitors are mostly used when the exact characteristics are not important, like on a supply bus, where your qualifications are typically either "at least" or "no more" than some value. Tripling capacitance usually has no negative effect on a supply bus but huge change of doing signal conditioning, so something like that is more often going to be a capacitor that is intrinsically more stable.
Wait, how many of these are in a typical consumer computer? Even if it's just 20 or so (I truly have no idea) I don't get the numbers to add up.
I've run, say, 6 computer-years worth of capacitors with no problems -- under the assumption above this would be 120 capacitor-years. Let's say at this point there's a 50--50 chance I would have experienced problems (this is not a rigorous survival analysis, of course, just something to get started).
If a single capacitor has a lifetime of just one year, then "lifetime" must be defined as the solution to p^120 = 0.5 which is p=99.5 %. Is it correct that when one speaks of the "lifetime" of capacitors one refers to the point where 0.5 % of them no longer work?
If "lifetime" means 50 % no longer work then I would seem to have been extraordinarily lucky (I'm the 10^-35 %!!) -- I don't hold that for very likely.
What else am I missing?
Capacitors especially and in general most components age and their specs drift. This is not a valid reason for not replacing components. If something isn’t actually all that old, sure, but capacitors especially are just not precision components over long periods.
More generally however, you just don’t need to replace a capacitor in something that works well and that is not anticipated to break catastrophically because of the capacitor plague.
If the caps do eventually drift off so far that it affects operation, replace it then, and then it makes sense to take the opportunity to replace others as well as precautionary measure.
Sometimes you know that a non-plague cap will likely fail because it’s right next to a massive heat source, and you may replace it early with good reason.
But sometimes I see people literally replacing 80 caps or more without reason, when I have the same device sitting at home with its original caps, still doing its job well within spec for years.
The relevant bit from Wikipedia:
> Industrial espionage was implicated in the capacitor plague, in connection with the theft of an electrolyte formula. A materials scientist working for Rubycon in Japan left the company, taking the secret water-based electrolyte formula for Rubycon's ZA and ZL series capacitors, and began working for a Chinese company. The scientist then developed a copy of this electrolyte. Then, some staff members who defected from the Chinese company copied an incomplete version of the formula and began to market it to many of the aluminium electrolytic manufacturers in Taiwan, undercutting the prices of the Japanese manufacturers. This incomplete electrolyte lacked important proprietary ingredients which were essential to the long-term stability of the capacitors and was unstable when packaged in a finished aluminium capacitor. This faulty electrolyte allowed the unimpeded formation of hydroxide and produced hydrogen gas.
I sometimes wonder, how those secret sauces can remain so secret. After all, you have to order them in rather large batches every week, month, year..
E: I can't believe something so easy to Google is being downvoted as though it's false info: https://www.reddit.com/r/retrogaming/comments/ib7nyh/replaci...
Wouldn't be surprised if there's a class action, where everyone gets $5 off a new printer.
Any company can get scammed on crap parts, either because they took a gamble trying to save a few cents or because those parts were all they could get their hands on, but now that's come back to bite them it's up to brother to put things right. It'd be shame for brother to trash their brand over a batch of bad capacitors. I've been happier with my current printer than any printer I've had since I got rid of my old dot matrix Epson, but we're going to need a new printer eventually, and I'd really like to not have to think too much about what brand to get or recommend.
It's true, though in the latter case, it can't be helped.
I worked at an integrator from 2002-2005. Small outfit, building PCs and servers for businesses and local agencies. As a new member of their engineering department, one of the first issues we got asked to investigate was to look at heightened failures of network cards. Back then, we bought the ubiquitous 3Com 3C-905. All of our PCs were subjected to a battery of functional tests before shipping, and we were finding that an abnormal number of the cards were failing.
We started looking at the failed ones more closely, and found them running extremely hot. These were passively cooled cards, and didn't need a heatsink under normal conditions. But on any of these, you couldn't hold a finger against without burning yourself. Functional ones as well.
We got the batch of serial numbers, and send them to 3Com to ask what might be going on. And the answer we got was that they didn't exist.
A bit of back and forth between 3Com, the distributor (a well-known and established company) and ourselves led us all to believe that they were a counterfit batch that entered the supply chain. We tracked down all of the cards from that order, and opened up every boxed PC that was to ship with one (I'll never forget the company president with his business shirt sleeves rolled up and a box cutter in his hand. He set the tone) and we pulled out every one.
We sourced from reputable places, and I don't recall another instance of that occurring while I was there. But it was an interesting event in my career, and one I'll never forget.
The only capacitor manufacturer that comes to mind now is Kemet and their wikipedia page doesn't mention the episode so I don't know.
I commented it on HN [1] not long ago and people said I was spreading conspiracy theory.
And how it used to be a thing called capacitor marketing [2] and how Apple / Mac "used to " set itself apart.
And let me just say this again because somehow the act of understand electronics is completely lost on modern days internet or more like in the West, planned obsolescence is still a thing and just because they aren't as bad as they were ( 6 months or at best 12-18 months ), doesn't mean they are not doing it. And more often than not, it is still the same old capacitor trick 20 years later.
Edit: And some comments below are writing just about that.
Jerry - if you're reading this, you probably single-handedly kept that place running ;)
Was a good pitch for the value of a computer-related major instead of medicine.
Maybe just a coincidence, but it felt like it started there.
[1]: https://www.tweaktown.com/reviews/8685/asrock-x470-taichi-am...
[2]: https://www.gskill.com/product/165/167/1536719010/F4-3600C18...
The funny thing is, this monitor looks like it was designed to die.
Essentially, the smoothing capacitor was placed directly in the stream of hot air going up. Loads of better ways to organise the board or just move the capacitor but no, the capacitor had to be in the worst possible place for it.
I replaced the capacitor with one specced for much higher temp and it is still going.
I am an amateur, but even I know to take this stuff into account when designing my boards. Like being aware which components are sensitive to temperature, which components emit a lot of heat, and what kind of resistances to ambient and ambient temperatures I may have.
The power board is a part of large board that is specifically designed for this particular housing so there is no question about orientation.
We had the electrician come in to verify mains power, but luckily we soon caught whiff of the fact that Dell had a capacitor problem on these models.
We overcame it with a little luck and by aggressively swapping machines as they failed, using out internal spares as a first line and immediately calling Dell for replacements for each and everyone of them.
One client once called regarding 80 Evo's who'd all gone out, one after another, over the course of a single day.
Compare to Samsung where I had a tv with bad caps out of warranty and they came to my house, quickly replaced the power supply and left a note asking if I was happy with the service. I was. I am.
"It wasn't just Dell" There were fraudsters who hadn't been prosecuted when dell was doing this so don't criticise Dell. Yeah...
Turns out mine was pretty badly done, and trying hard to imitate a MacBook Air down to the motherboard, yet half of the components of my (heavily used) XPS 13 showed to be quite unreliable within just few years sadly.
Having said that, what's the most reliable, decently portable laptop around for a Linux-oriented developer? I keep hearing bad things about newer Lenovos as well
Installed linux, everything I care about just worked. (I don't care about finger print readers, maybe that just works too? Hate it either way so dunno).
I set the power setting to "power saver" on the incredibly obvious and discoverable gui selector and I get 10 hours out of the battery. A battery which I did not even replace. You want me to list annoyances, well I can, and they're all minor and all to do with choices that my linux distro made.
Now I've heard brand new lenovos aren't as well supported as 2-3 year old models and I've been hearing that for more than a decade so there is probably something to it. Seems like enough linux devs use them that it shakes out really nicely in that 2-3 years.
If this one exploded I'd probably be looking for another one just like it second hand.
Can't tell you about other T series or X1s or all of the other wild and wonderful lenovo "thinkpads" some of which those who claim to know say "thinkpad yeah, but not really..." Just the T480s I'm saying here is a beautiful machine.
So that met my specs. But what to do is go to the relevant thinkpad page on wikipedia [1] and all the models and specs are listed there. Use it to look through the places where you get ex-corporate and ex-govt laptops. Find the thinkpad that meets your needs. Vastly cheaper sure, but don't overlook just how much /better/ and higher quality it is than anything Dell have for sale. Support won't be worse if something breaks either. I can burn through 4 of these for the price of a Dell but somehow I doubt that will happen or anything like it in the same amount of time as the Dell was actually working.
Seems like you can close it and stick it in your bag like it's a laptop too, which you can't with the Dell because it isn't despite the advertising.
Unfortunately though my company deployed new Dell laptops to everybody, and yes I do have to shut it down whenever I want to put it in the bag. Always great to see technology move backwards...
The representative initially agreed that the capacitor was bad. He grabbed a new board, pulled it out of the bag, and literally the same problem. Same capacitor even, which was unfortunate for me because the new determination was that maybe they're supposed to be like that? They had a tech in the store who claimed that it was residue from some kind of "conformal coating", and that 7/10 boards have capacitors that look like that. Somehow the residue is biased to appear directly under the capacitors, but that wasn't exactly explained. He also claimed that capacitors don't leak, at least the type on this board.
Eventually I just asked for one of the 3/10 boards that didn't have that residue, and the store was gracious enough to do that for me, but insisted that there was no issue the whole way.
They had to flip through 4 boards before finding one without residue under any capacitors, and I still don't know if I was being unreasonable.
A friend thought maybe that capacitor was getting squeezed during production, or the board is just designed such that that capacitor is more likely to fail.
[1] https://www.digikey.ca/en/products/filter/aluminum-polymer-c... [2] https://en.wikipedia.org/wiki/Polymer_capacitor
When sending back to Apple, my motherboard got probably replaced by an identical refurbished motherboard or some such. Either way, after a couple of months, again leaking capacitors. At some point I got angry with Apple, but by then my Mac was outside of warranty. They offered me a discount on next purchase at Apple Store, but I declined. It left a bad taste in my mouth regarding Apple.
Still kept buying Macs for the next few years and luckily no other Macs had issues. But at that point I was sure I would never again buy a first gen (major redesign) product from Apple ever again. I'd rather wait until the early bugs get sorted out and buy a gen 2 or 3 version of the product.
I do hope in the future I can switch to Linux, but right now for work I kinda need Macs still (iOS dev).
Some guy with a similar experience as me at the time: https://discussions.apple.com/thread/377158?answerId=1800991...
Worse, it was fairly hard to get replacements of the power supply units we were using for a time. In the end we were always able to keep a couple on hand, but it would often involve a lot of shopping around and sometimes paying a premium.
That issue resulted in us switching from the cheaper 1U units with single power supplies (based largely on over a decade of no power supply failures), to the "Fat Twin" 2U systems that had 4 "blades" and redundant power supplies. We were doing small scale dedicated hosting, so low cost of the lower end machines was important, but the mini blade setup evened out the price over 4 nodes, but with a steeper initial chassis cost, and more limited disk expansion.
Add: Test it well even if you think it's perfectly duplicated
I mentioned this to our VP of development once, and was sympathetic to the idea that this was something out of Dell's control. He was NOT sympathetic to it at all. Apparently IBM had refused shipment of the same capacitors, because they actually tested them.
My computers from 20 years ago, not so lucky. They're all dead.
I can't even remember how many bulging capacitors I removed and soldered new ones in. I got pretty quick at it.