Hacker News Reader

Comment by ajross | Hacker News Reader

It's a PCB manufacturer.

They aren't buying computers from china. The company that did design the electronics (GE) decided to have this little outfit manufacture the PCBs for them (I mean, we're talking a production run in the low thousands so far -- this is NOT a big project). And the little outfit got bought by a bigger PCB company in China a few years back.

The risk to the supply chain is zero -- anyone can make printed circuit boards. The intelligence risk is limited to the ability to see how chips are wired together, with some ability to guess what ICs are in use by clues in the pinouts.

View on HN

I don't know anything about intercepting electronics, but I would not bet my life (which is what you're doing) that a good signals intelligence agency cannot somehow compromise a PCB.

I agree. I mean if we're guessing at possible dangers, it seems like there are several possible angles that might be tempting to try:

- It's meant to be a stealthy plane, how about tweaking the board to cause a nice and noisy signal leak in certain circumstances?

- find a way of causing the board to fail under suitable circumstances.

- Probably less plausible, if they can figure out what data is passing through the board, then there is potential to engineer side-channels in the hope that they can achieve another compromise that can leverage it. Maybe they never get the chance to take advantage, maybe they do. Or if e.g. they can find an input that an adversary can potentially influence by how they engage with the plane, use that to trigger failure as above.

I'm assuming they're properly inspected and tested on receipt, but wires on circuit boards can interfere with each other in messy ways if you exceed tolerances even a little bit. I recall an amusing case some years back where someone used genetic algorithms to lay out features on an FPGA and got a very good solution that was totally un-reproducible: it turned out their GA had optimized to take advantage of quirks of that one specific FPGA that didn't work on other FPGA's of the same model, by combining things in ways it hadn't been designed for. It seems it's very much possible to design chips that superficially looks like it should act one way but where it acts differently in certainly circumstances.

Doing so in a way that's exploitable without being caught out is probably much harder, but I share your skepticism.

That's not how PCBs work at all. Dear god.

Such mission critical to EMI designs would be submitted to full spectrum EMI tests once assembled (hitting it with noise of all kinds of frequencies AND testing it for leaks of all kinds).

Causing a bare board to fail? lol, perhaps if you have an atomic disintegrator. Assembled boards with actual components? Now you have something you can actually fail but this PCB house does not do said assembly nor have access to that.

Figure out what data? Completely impossible. These are just copper traces. They don't get told "Mr Copper trace, you are going to do i2c and send XYZ data! and you are going to be a pci express and send ABC!" at the fab. Jeez.

Now a real cause for concern, if say a state actor had access to the completed assembly going into the planes, they could identify a place to embed a backdoor. The state actor controlled PCB house could then embed the backdoor into the PCB itself. But at that point you have 2 different security breaches and the PCB guy is probably lesser of your concerns.

The dumber thing would be to alter the current carrying capacity of power traces which tend to be obvious staring at a board. But it would just identical to trolling by causing boards to fail and eventually get it investigated and your supplier contract revoked.

This case reminds me a bit about the US embassy in Moscow, where the US let the Soviet provide them with premade concrete pieces for the construction [1]. The idea was that it would be enough to inspect them to detect if any bugs had been inserted.

This plan didn't work out in the embassy case, but I'm not any kind of expert on electronics, so maybe the lessons learned doesn't translate exactly to the PCB situation.

[1]: https://www.nytimes.com/1988/11/15/world/the-bugged-embassy-...

> They don't get told "Mr Copper trace, you are going to do i2c and send XYZ data! and you are going to be a pci express and send ABC!" at the fab. Jeez.

I do only very basic electronics, so I'm asking here in honest confusion: Why can't you figure that out? I'd have expected that part footprints, high-level topology, fine details of individual traces, and the general need for everything to make sense at all would tell you everything you needed to know.

RF optimizations (squiggles to control propagation delay, weird shapes in corners, notches) (edit: wouldn't these give you the bus's frequency too?) and thermal tweaks (wider traces, bigger vias) would give you information about the pinout of whatever's plugged into each footprint (fast, slow, low-power, high-power, etc). Basic topology (is a line connected to two pins or thirty, is a line isolated or is it part of a sixteen-wide or ninety-wide bus) would narrow that down even more. More important parts will almost necessarily have more lines coming out of their footprints so you'd have more information about the stuff you care about.

Even if we assume there're no standards in use ("941 lines, power here here and here, high-frequency bus with 288 lines here, so that's socket AM3, which means these are..."), I wouldn't have surprised if you'd told me you could figure out what part numbers they're dropping into the slots for microprocessors and similar. What am I missing? Why can't you figure out which lines are i2c to misc peripherals and which lines are PCIe between CPU and key coprocessors?

> > They don't get told "Mr Copper trace, you are going to do i2c and send XYZ data! and you are going to be a pci express and send ABC!" at the fab. Jeez.

> I do only very basic electronics, so I'm asking here in honest confusion: Why can't you figure that out?

You can. OP is trying to be snarky, but he's wrong.

I don't think it's worth getting into an argument with someone with such an obviously demeaning attitude ("Dear God," "Jeez," etc.), but a few points:

1. It's not just bare copper, silkscreen frequently tells you what's going on, even in secretive government airplanes.

2. Even without silkscreen, e.g. on assembled boards where the chip part numbers have been lasered off for "secrecy," you can often uniquely identify a chip family just by the pins connected to power/ground, certain passives like crystals or filters, etc.

3. If it's a chip with highly remappable pins where you can't just look up which ones are i2c in a datasheet (e.g. FPGA), you can often find them routed together with obvious strategies for impedance control and/or shielding.

4. For i2c in particular, you will likely see two wires routed together, each with a resistor-like footprint pulling the line high.

Amateurs regularly do stuff like this... It's silly to assume a state level actor couldn't do the same or better.

The parent commenter is being naive. Of course you can make good guesses at which components fit where, and from there, which lines carry which signals.

Most boards in the plane are probably designed to have as little RF sensitivity as possible, and are also shielded.

However, if you control the copper traces, you can definitely influence characteristics of the circuit. Presumably, those would be picked up in testing, because they are looking for those kinds of defects.

But a semi-passive component underneath a trace, constructed to act a resistor/capacitor/inductor once every million pulses might not be.

Well, it should be possible to hide components inside the PCB, see:

(1) https://hackaday.com/2017/09/11/the-components-are-inside-th...

(2) https://hackaday.com/2019/01/18/oreo-construction-hiding-you...

Analysis should be quite difficult; if you hide them between copper layers, or worse behind components and behind copper layers, I believe it'd require vert inspection with x-rays to detect.

The main question is that of utility. Data exfiltration is highly unlikely. It's possible to trigger malfunctions, but it's not clear this would be useful (when triggered at a certain date, randomly, etc) -- it would be easy to detect and cause the companies to switch to suppliers. I think the main possibility/atractivity would be some kind of radio activation of malfunction. But then you need a significantly sized antenna [1] and a very strong signal to penetrate the circuit shielding (which is probably emp-resistant).

But the main impediment I think is the total erosion of trust in Chinese manufacturing this could bring. Most of the world manufactures in China and they don't want to change this. They wouldn't sacrifice this trust except at a very good opportunity. The same goes from a strategic spying perspective: don't use bugs frivolously or you'll risk raising the awareness level compromising important opportunities.

[1] Which cannot be hidden between layers, although it might be possible to use some of the traces themselves as antennas. If the pcb has an obvious external antenna I/O then of course this would make things easy.

All that said, it does make sense to keep an eye on this possibility, but probably not in this particular case.

This is the kind of attitude that makes breaches viable.

If they have leaked lower level plans, that leak is not necessarily useful in itself, but as you point out that does give knowledge about where to embed backdoors. It is not unusual for damage to come when an adversary manages to combine to things that individually are less protected because people fail to see the risk in an individual breach.

As for figuring out the design without it, even as an amateur there are lots of things that I can often infer from looking at a bare board. E.g seeing where power is fed in and which pins likely lead to ground. Seeing where groups of traces follow, which tends to often imply data or address lines. Seeing how chips are grouped and the like - humans lays things out 'logically' even it's not necessary. I have opened enough electronics to know you rarely need to be able to read the text on the chips to know which is a CPU and which is RAM for example.

And hitting something with EMI tests solves for the amateurs who don't know how to do the same testing themselves. It is no guarantee against a state actor prepared to do lots of their own testing to ensure they've mitigated the effects of the changes they've made until certain conditions are met for any circuitry they've hidden.

The dumb thing would be to assume that we're smart enough that a dedicated adversary with access to some of the most sophisticated electronics manufacturing on the planet can't find a way to fool us.

The biggest flawed assumption you make is echoed in the article too: the assumption it is a bare board. It is meant to be a bare board. It is meant to be just copper traces. Good luck verifying there's nothing else sandwiched between layers, obscured on screening by copper on other layers.

Yes, we don't normally do that, but not because it's impossible, but because it's pointlessly complex when we don't need things to be hidden.

The number of security holes that are the result of assuming something can't be done instead of ensuring nobody gets the chance to try is quite substantial.

It's thought that China got F-35 plans via hacking into the contractors. That would be enough to understand the board.

Actually having the board means they can embed a chip inside it.

Combine those, and they can MITM all sorts of data. Depending on the purpose of the board: They can effectively grant stealth to their non-stealth aircraft. They can cause the F-35 to emit a response to a coded message.

Without much knowledge of the board, there is still hope for causing mayhem. The modifications could check for kill codes on many traces. China could deliver these in a multi-spectral way, not knowing what data actually flows through the board. So the kill codes show up on all the RF frequencies that the F-35 is thought to receive, and on anything optical, and so on. The kill codes could then disable the board, supply high-voltage pulses to other boards, or perhaps even make the board detonate.

> These are just copper traces.

They're supposed to be just copper traces. The point of comprising it is you add something that's not supposed to be there, like a chip embedded secretly inside the plastic and connected to the traces.

I see you skipped the part I made that point eh.

That is possible, but would require first having a security breach with access to the completed assembly to know _what traces_ first. At which point that security breach is slightly more concerning given it's higher level.

>That is possible, but would require first having a security breach with access to the completed assembly to know _what traces_ first.

that's not true.

one can design a pcb to pass conformity tests but fail in a more discrete manner, even without total knowledge of the system. This is made even more possible if the manufacture has knowledge of what conformity tests must be made, and how. This is quite often common knowledge for such manufacturers as they are trying to keep costs to the minimum required to pass the testing needed for the product to be bought by the contractor.

citeseerx.ist.psu.edu had article How Secure Are Printed Circuit Boards Against Trojan Attacks that seems written by an expert

Passive components are not safe from sabotage.

Saboteurs might design in PCP boards that pass aging tests, then fail in reality much faster. Detecting mechanical or chemical failures that are designed to pass testing is not an easy task.

The real concern is naïveté.

This is not an attack on you personally, because your sentiment is not yours alone.

It’s just a reflection of the total disrespect for engineering and manufacturing that Americans seem to have.

> submitted to full spectrum EMI tests once assembled

Under all internal running conditions? That's not feasible.

Thanks for the patience in explaining this, the amount of people voicing completely unfounded opinions seems to be getting higher every day

It is a PCB. If you're really really interested in sabotaging it, you can, erm, make it rust quicker I guess

Never underestimate motivated spooks with unlimited budget and access to the best and brightest.

I believe this is possible. Discussed in detail with a proof-of-concept exploit at the latest CCC:

https://media.ccc.de/v/35c3-9597-modchips_of_the_state#t=901

Edit: this is a slightly different attack, it replaces a resistor with a custom chip that alters the transmission on that line (carefully disconnects the line, turning some 1s into 0s). But it should be possible to fit something like that inside a PCB, when the layers are put together.

Reminds me of that article about the embassy bug that was posted here last week - in theory you could design a passive signal amplifier into the pcb, which only lights up when beeing iluminated at a key frequency. Doesent even have to be energy supplied or otherwise active.

From the accusations of China compromising motherboards a few months ago, the exploits are: Incorporating transmitters that look like surface mount components. Imagine putting a small transmitter into the design on a signal trace where a resistor or capacitor normally would go and then setting up a receiver just outside a base to catch the signal. It could record all kinds of things and transmit it a short distance, weapons loadout, fuel amount, when the aircraft leaves the base, etc. The second is that some components can be squished between PCB layers such that the only way to detect them is through xray inspection.

And for you security deniers that say nothing like this ever happens, https://arstechnica.com/tech-policy/2014/05/photos-of-an-nsa...

For transmitters, sure, that might work with a lot of caveats attached. For example, you need to repurpose wires on the PCB to act as antennas and then hope that the shielding around the elctronics will be transparent enough to pick anything up.

But getting the information to be transmitted is a lot harder. I don't know what kind of component this is, but I suspect that not a lot of PCBs have direct access to that kind of high level information (assuming that there is a kind of system bus that even passes such data around...).

I think in this type of business it is take what you can get as far as technology. A fuel pump PCB could be bugged one way, while a core flight control PCB could be bugged another. The bugs could also carry their own sensors. A simple pressure sensor could report altitude, and an accelerometer/compass could report motion. Those could be placed on the output stage of one of dozens of antenna or radar hardware. Even modulating a light bulb or LED slightly can transmit data that can be seen miles away.

Except that in the US defense industry, anyone working for a foreign company is to be regarded as a foreign national (even if they have a subsidiary that is in the US).

While a part of me agrees, I don't want the Chinese to have access to even non-critical things. What if they insert some kind of small component that can be embedded in a PCB stackup that siphons data from a bus, or introduce a high Q resonator that creates an easily discoverable signature when hit with a certain frequency, or something else (like The Thing - https://en.wikipedia.org/wiki/The_Thing_(listening_device)?

x-ray inspection at the bare-board level would check for both of those things (of course), but usually x-ray inspection is done by the assembler after boards are populated - not before - and I've never heard of it being one on a bare-board as an inspection step post receipt. So something like this actually could be missed.

You wouldn't even need to do it to every board, you could do it to some fraction of them, and have (essentially) a secret frequency vulnerability lying in wait. Basically it would invalidate your stealth/LO (obviously this is dependent on shielding and a lot of other things, but you get the idea).

Pretty sure it’s quite easy to test for any resonators in a radio test chamber.

Any test is only effective if it's actually performed during acceptance, and there's no requirement provision for such test in any acceptance qualification that I'm aware of.

In either case, it's kind of a shitty test given defense applications aren't constrained to the same EMC regulations as the rest of the commercial world.

Couldn't a good state-run sabotage take advantage of assumptions about how much scrutiny/inspection is needed for a given component?

For example, suppose there was some little-known chemistry that would make a PCB likely to fail $X years after manufacture. Or bonus points if it had out-gassing that made other, innocent parts have high failure rates?

That's for new-manufactured boards. My guess is this whole run completed years ago. And surely new boards will be farmed out to some other Mom & Pop PCB shop. These companies are everywhere, electronics assembly is not meaningfully "high tech" any more.

Given that it's a fighterjet board, it's probably just a glorified FPGA breakout board.

For real though. They stick anything and everything interesting in FPGAs for defense aerospace.

Check out this missile guidance computer. https://www.bunniestudios.com/blog/?p=3649 Pretty much just a PowerPC and some RAM glued to a giant FPGA with a lot of off board inputs. These days they'd use something like a Zynq that has the hard processor core on the FPGA die, making it look even more like an FPGA breakout board.

Any malicious intervention would as you say have to be very subtle to pass system tests. Given they cannot know whether and when the planes would ever be used against them, it would be playing the long game at best.

I'm imaging failure cases under high Gs when the thing tries to do some high speed data transfer?

let the traces move enough to introduce cross talk if its vibrated and accelerated in just the right way.

There is a short term gain, even if they never get used against china: the US and it's allies have to spend more in testing to account for the Chinese built PCBs.

I guess Bloomberg might have something to say about PCBs not being a security risk. Did they ever back up that story in any way?

If they were made after the Chinese purchase, maybe. There's no indication in the article that they were though.

> The risk to the supply chain is zero -- anyone can make printed circuit boards. The intelligence risk is limited to the ability to see how chips are wired together, with some ability to guess what ICs are in use by clues in the pinouts.

Raytheon once ran into a "small" problem with it. They actually did whole PCBA in China for AIM 9L.

There is a blogger guy who actually found a whole image processing board with top tier FPGAs that somehow got to an electronics scrap trader around here few years ago.

I would like to read that if you can find the link.

https://www.bunniestudios.com/blog/?p=3649

the link above is about that particular case, but the hearsays of American defence hardware like ICBM gyros popping up around China in scrap piles predates that by few years

Thanks for that, very interesting.

>The risk to the supply chain is zero

Hypothetically, what would happen to a pcb board , 33x33cm in size, with the ground lines running around the edge of that board, when hit with a high power 900 MHz broadcast?

Aircraft go through many tests, including various electromagnetic interference tests. They should know exactly how it responds in such a scenario.

https://en.m.wikipedia.org/wiki/MIL-STD-461

Of course, I was just illustrating with an elementary electromagnetism example that the risk isn't zero.

That's fair. There's always supply chain risk in more nefarious ways as well (rootkit/trojan in bios, etc)

That high power 900 MHz broadcast will travel half an inch, then get absorbed by the metal housing. Military electronics are shielded, both to protect them from EMP and to avoid detection by passive radar.

“Hypothetically speaking”. Heh. Good one.

On that note I don’t understand the willful cognitive dissonance around Chinese espionage. They’re a powerful state aggressively looking out for their own self interests just like all the others.

I don't think anyone who knows what they're talking about is worrying about supply chain risk, what they're worried about is backdoor risk. Controlling pcb manufacturing is a good way to be able to put all sorts of surveillance chips, backdoor chips, etc. on the pcbs.

What? This is exactly where an intelligence agency would step in and add an extra little chip onto the board. There's tons of precedence for this.

Unless the government inspected every board individually, there's no way to guarantee supply chain. At least from a national defense perspective, everything that comes out of China is a possible risk.

To flip the debate around, if the US manufactured PCBs for China, don't you think we would do the same thing?

An extra chip on a bare board would stand out quite a bit.

An extra chip embedded between the layers would only be detectable by X-ray inspection.

Depends on where the chip is and what it did. It's possible to hide chips between layers, hide chips under chips, replace an existing chip with something that looks almost just like it with additional functions.

It's really not hard to imagine some smart people being told to make it work.

It is a big project in terms of people, and it's not impossible that bill of materials was referenced at some point. Even if not, the netlist which must surely have been provided for testing purposes must provide many clues (unrealistic example: AVR8_RX). Even if "only" PCB manufacture was done, there's a lot of information, on par (better, in fact!) with xraying the assembled product.

Last time I worked on PCB design and manufacture liaison this isn't the way it's done. You send artwork (in electronic form) to the sub. They perform a connectivity test on the finished boards but the net list they test against was derived by their CAM system from the artwork data. We never released our CAD netlist data to them and honestly I don't think they'd know what to do with it if we had. Ymmv since this is an old data point.

Yes, the data formats for PCB design data are so old they are commands for a now nonexistent photoplotter, a list of circles and rectangles

I thought this was hard for complicated VIAs

It's no problem - just buy a bigger PCB machine produced in China.

Its just hilarious example of how China is part of the global economy despite Trumps escalating tweets (trade deficit continues to climb by the way if anyone's interested).

I am sure China knows everything about the F-35 its not a secret weapon.

> The intelligence risk is limited to the ability to see how chips are wired together, with some ability to guess what ICs are in use by clues in the pinouts.

No, they could insert fake chips during the assembly. Chips that do almost anything, from interfere with operation to logging information, to transmitting data, to exploding in place.

It's bare board, they don't assemble. They just make the PCB board itself.

This is all assuming that they did the assembly, which I'm willing to wager they didn't.

They only build the board, not the chips that go on it, so no, they couldn't.

Reply on news.ycombinator.com