They found out about the failing transistors via colleagues at a conference. Have any of you learned of something of this magnitude in the same way? It got me thinking that I need to interact with my fellow devs more often.
This is a classic thing with Industry, they qualify a process that is working and shows good performance, but this process needs to be changed for reason XYZ, often because it is maybe a bit too expensive or doesn't align with the rest of their processes. The small change in the process wasn't that small and takes a little while to be identified because by the time you catch it you might be further down the line and this would be caught by a QA process and not a QC process, that might have deemed at that point not necessary because you had no reason to fault the part.
The second part is that some things are rated and verified but not tested extensively, since you might have prototype you might misdiagnose a failure of a component for a behaviour of your prototype, when in fact you had a deeper problem, but timelines with the added fact that so far you didn't think about that problem because it shouldn't have been a problem can catch you really off guard. This is usually where people testing the same thing in an exotic environment can ring alarm bells for others and that often happens at conferences...
People often under estimate how much you can get bitten in the back by such little details that become huge details.
Depending on the electronics and where the MOSFETS are, I would be them I would probably trash the electronics, take the spare that they had, validate components that get in and rebuild a control box and re-integrate it, provided that this is doable. It's expensive but provided that you have no choice that gives you a backup system that you can test code on before pushing it on the actual probe and might help for problem solving by being able to do measurements and test on the actual setup... Provided that they have the time and resources. Otherwise I wouldn't YOLO it given the fact that it might just straight up not work at the moment you need it the most and a little delay is better than nothing and they can spend the time re-checking part of the design that might also be weaker...
But heh, who am I but a random guy on the internet...
They found it was cured by lemon juice, but they didn't understand the details. Over years, they switched to lime juice (less vitamin C), put it in copper pipes (leaches vitamin C). But ships were faster so there was more fresh food available, masking the problem. Then scurvy starts mysteriously popping up again 100 years after it was first "cured."
Hard to keep track of the effects of all the details in the face of various co-dependent things changing simultaneously. Recipe for surprises.
This is a more-consequential example of the things you can learn by chatting with others; it is an extreme example of, "Hey, are you guys using components from Widget Inc.? Their datasheets are good, but sometimes we get a bad batch."
Those little things can save you a ton of time. In this case, it may have prevented mission-failure.
Part of the blame falls to NASA, too. If the outcome is your responsibility, then open-loop trust of a vendor for a known failure-mode may not be acceptable. Integration rad-hard testing may be requisite.
In the spacecraft environment, qualifying components is very difficult -- there's a good chance that NASA has these MOSFETs on an approved list because they've worked well before and have had few (or known) faults. They're probably not on that list anymore.
My guess is the parts failed TID at the more stringent levels, and Infineon didn't follow up with NASA or their contractor because they assumed that NASA was okay with the lower rad tolerance levels typical of space. Usually that would be the case, but Europa Clipper is special because it's going to an extremely harsh radiation environment.
The big question for me is: did the Europa Clipper program order a lower TID and try to upscreen, or did they order the high TID part? If it's the former, it's on NASA. If it's the latter, that's extremely concerning because Infineon should know that nobody orders expensive high TID parts for funsies, and they should have followed up with all customers as soon as they confirmed there was an issue. Just assuming NASA over-specified a part is absurd. The rad hard electronics market is small, everyone knows each other. Trust is king.
Finally, I'm not sure if it's the part in question, but it looks like Infineon discontinued their 1 MRad MOSFETs in 2020, citing low order volumes: https://irf.com/product-info/hi-rel/alerts/fv5-d-21-0004.pdf. In the light of this reporting, I have to wonder if there was more to it than that?
It's more likely that Infineon's folks talking to NASA were equally clueless about this change.
Is there any actual evidence they didn't reach out to every single buyer of the electronics?
The article goes out of its way to say Infineon did not contact NASA. But even in your description, they would not have, they would have contacted NASA's contractor working on the electronics.
I still go back to "if there was actual evidence that Infineon did not notify who it was supposed to, the article probably would have cited it". There isn't, so they instead cast aspersions.
Instead they make a bunch of hay about a statement from Infineon that seems totally innocuous - they didn't notify people they didn't know about. Shocker.
Look, i actually hate Infineon - i've been forced to try to make their wifi and bluetooth modules work properly before ;-)
But this kind of lazy-at-best journalism doesn't help anyone.
Personally, I despise status meetings. 99% of them are worthless fluff.
But, every now and then, you get something like this.
I think that highly usable dashboards are a good way to deal with this.
It's possible that AI could be a big help, here.
I experienced this kind of situation, where only by chance conversation was a crisis averted, very much at my last FT. So much that I'm working on a startup for fostering serendipitous communication for remote teams, like private notes from coworkers left on stackoverflow questions (or anything on the web)
Have you even been in one of those meetings that just won't finish despite everything being done? Making it written doesn't solve the problem. Instead, it makes it worse.
AI it is, then...
Computers are good at this though.
Now the only question is how you can automate the spec comparison such that issues with the spec and the parts used can be automatically compared.
And that starts with a computer readable spec that is updated by the manufacturer.