Polish DRMed trains stop as predicted due to date-based logic-bomb | Hacker News Reader

Polish DRMed trains stop as predicted due to date-based logic-bomb

social.hackerspace.pl

229 pointsp_l103 comments

Open articleView on HN

Previous discussions over the last few weeks:

* https://news.ycombinator.com/item?id=38530885 - 363 comments

* https://news.ycombinator.com/item?id=38628635 - 139 comments

* https://news.ycombinator.com/item?id=38567687 - 298 comments

* https://news.ycombinator.com/item?id=38638865 - 235 comments

* https://news.ycombinator.com/item?id=38570654 - 2 comments

Thanks! Macroexpanded:

Trains were designed to break down after third-party repairs, hackers find - https://news.ycombinator.com/item?id=38638865 - Dec 2023 (232 comments)

Polish Hackers that repaired DRM trains threatened by train company - https://news.ycombinator.com/item?id=38628635 - Dec 2023 (139 comments)

Polish train maker denies claims its software bricked competitor rolling stock - https://news.ycombinator.com/item?id=38570654 - Dec 2023 (2 comments)

Dieselgate, but for trains – some heavyweight hardware hacking - https://news.ycombinator.com/item?id=38567687 - Dec 2023 (293 comments)

Polish trains lock up when serviced in third-party workshops - https://news.ycombinator.com/item?id=38530885 - Dec 2023 (359 comments)

The discrepancy between yours and his comments number is fascinating.

The difference mostly goes away if I turn off "showdead" in my preferences.

Ah, yeah, that would explain it!

This train DRM is crazy. At the beginning of the month they found geofences

> We (@redford, @mrtick and I) have reverse engineered the PLC code of NEWAG Impuls EMUs. These trains were locking up for arbitrary reasons after being serviced at third-party workshops. The manufacturer argued that this was because of malpractice by these workshops, and that they should be serviced by them instead of third parties.

> We found that the PLC code actually contained logic that would lock up the train with bogus error codes after some date, or if the train wasn't running for a given time. One version of the controller actually contained GPS coordinates to contain the behaviour to third party workshops.

> It was also possible to unlock the trains by pressing a key combination in the cabin controls. None of this was documented.

> The key unlock was deleted in newer PLC software versions, but the lock logic remained.

> After a certain update by NEWAG, the cabin controls would also display scary messages about copyright violations if the HMI detected a subset of conditions that should've engaged the lock but the train was still operational.

> The trains also had a GSM telemetry unit that was broadcasting lock conditions, and in some cases appeared to be able to lock the train remotely.

https://social.hackerspace.pl/@q3k/111528162462505087

am I correct to assume that none of this was in the manual?

That's effectively the core of the accusations against NEWAG.

The manual was supposed to allow comprehensive maintenance and repair, without requiring manufacturer's input.

The people investigating this claim that much of this is undocumented. I don't speak Polish nor have access to any of the primary documentation for the train.

We may see more info next week as they are scheduled to present at the 37th CCC. https://events.ccc.de/congress/2023/infos/index.html

Q3k is active here, https://news.ycombinator.com/threads?id=q3k

At least one of the trains had its DSU (Dokumentacja Systemu Utrzymania - Maintenance System Manual) linked online (it's specific to that one train). It should be enough to perform full maintenance.

Nowhere in it any of the "lockouts" are documented.

It's very hard to see what legitimate reason would be used to document such a check in the manual.

the reason would be so you’re not being dishonest and selling a misleading product

A tiny note: they found the geofences over a year ago, a month ago was when they went public.

My question: Who wrote this software? Software engineers with 0 moral and ethical qualms or with an extreme incentive to get rich?

Or heroes who simultaneously reported what they were hired to do

Not saying they were -- but not a bad alternative to declining the work outright

If they did, none of the moths-long reverse engineering would have been required.

Ever heard of parallel construction?

Yes, it's "Technique not appearing in this episode", to paraphrase.

The heroes also wrote heroically bad date checking code.

If they had reported things... well, it wouldn't take over a year to deal with this.

> Software engineers with 0 moral and ethical qualms

I would like to challenge this. The ethical problem comes from this two facts: they had code which disabled the vehicle in certain conditions AND this was not communicated in the manuals/contracts.

The two together is what is extremely dubious. If you implement a time based expiry system, and tell about it to the customers that I think is fair. The customer might of course opt to not buy the equipment under those terms, or only pay a significantly reduced fee, but that is just business. And it might be good business for both participants if everything is well documented and transparent, and these restrictions are reflected in the price, and everyone enters into the agreement with full knowledge and consent.

Why does this matter: Look at it from the developer’s perspective. Imagine that they are told by their bosses that they need to implement technological measures to disable the product when the licence expires. Would developing that be unethical? I don’t think so. It becomes unethical at the later point when the train operator is not told about this. How much of that is in or out of the view of the developers is an other question.

I’m not saying this is what happened. Clearly the gps based lockdown is not consistent with this story even under the most charitable interpretation. Something is fishy here and needs investigating. (The kind of investigation where the police with a warrant takes the repo and the communications of the company, and interviews everyone separately to figure out who did what and when, and who known what and when.)

But while the overall conduct of the company appears to be unethical, it is not necessarily true that the software engineers in question were also unethical. It depends on what they did know, and what they were asked to deliver.

They weren't selling software licenses here, they were selling a very real piece of hardware. It would be ethical to disable noncritical software updates under some time based constraints, its completely unreasonably to disable the use of the hardware all together. The customer bought it, that's their hardware to use however they wish.

It would be completely unacceptable to me if an auto manufacturer baked in a time lock in a car I purchased. Heck, Apple got slapped for appearing to throttle performance based on age. Why is it reasonable for a train manufacturer to pull this stunt?

To be clear, plausible deniability is also a big gray area when it comes to moral or ethical questions. It a timelock in the software seems unusual, engineers aren't off the hook simply because they didn't ask. If they did push back or ask for an explanation and were given false answers, sure they likely didn't do anything most would consider immoral. But if they just wrote the time lock because it was in a spec and didn't ask why a train should include a time bomb? Totally unethical in my book.

> They weren't selling software licenses here, they were selling a very real piece of hardware.

I don’t know. I didn’t read the contract. Neither did you I suspect nor did the developers.

Jet engines are leased by the hour. Yes they are very real pieces of hardware bolted to your airplane. That has nothing to do with the business model. Why can’t the same be true for trains?

Even if I would know how trains are usually sold/purchased, which I don’t, I could be convinced by a boss that we are trying a different model.

Why is this important? Two reasons: if your mental model is that these kind of things are done by “unethical developers” then you are not looking for the real culprits, and your interventions trying to prevent such things will be inefective.

Do they turn off the jet engines if they don't pay, or do they send them a bill? There is a huge difference.

Also, writing it in a contract doesn't make it ethical or even legal. Just because someone writes it in a license agreement doesn't mean that we as a society should blindly accept it. We don't allow people to sell lots of things, why should we just say 'well it was in the license' because someone tried to sell 'being able to have turn off your own passenger train for more than 10 days'?

> Just because someone writes it in a license agreement doesn't mean that we as a society should blindly accept it.

People who buy trainsets are sophisticated buyers. They have lawyers, accountants and engineers advising them. You don’t sucker a semi-senile grandma into buying your trains unseen. Nobody buys trains on a drunken dare, or as an impulse purchase.

Because of this, if the financing is more viable that way, companies should be able to purchase, rent or lease their trains under whatever model makes the most sense in their particular circumstances. This of course assumes that the manufacturer deals fairly and the purchaser is properly informed about what they are buying, leasing or renting. (Which does not seem to be the case here, and that is unethical and a problem. The kind where people should go to prison in my opinion.)

I don't have anything substantial to add, but this got me:

> or as an impulse purchase.

as the whole case is about an Impuls purchase! :)

>Jet engines are leased by the hour.

I'm... very surprised by this, do you have any resources I could read about it?

https://aviation.stackexchange.com/questions/12528/jet-engin...

https://www.sps-aviation.com/story/?id=2623&h=Engine-Leasing

The engines also have an always-on connection back to the mothership via satellite, eg:

https://www.forbes.com/sites/johngoglia/2014/03/13/aircraft-...

Thank you for the links!

This was very informative, thanks! It's so easy to assume a very simplified (or outright incorrect) model of how the world works and then point fingers in the wrong direction...

This is not an argument I necessarily believe, but to steel-man this one:

You know your trains will need service after 12 months in service or some number of miles. Absent that service, the train could fail, the failure could be catastrophic, and a catastrophic failure on a train kills people. You also know that municipal train operators in a great many parts of the world will absolutely run their trains until they kill someone rather than pay for downtime and maintenance. Therefore, you put in a lock on the train that if it hasn’t been serviced after 12 months, the train disables itself to force the owner to get the train serviced.

The background to this is European Union mandating unbundling of maintenance from purchase contracts.

Specifically, it's been no longer possible for manufacturers to claim that maintenance documentation is trqde secret or otherwise not possible to be made available to third parties, which opened the door for third party workshops to do deeper maintenance.

And the train manufacturer started losing tenders for maintenance.

Yeah, no part of the rest of this story suggests the charitable interpretation is the right one. I can see a case for being more aggressive about ensuring large machines get serviced before they can do harm, but I don’t actually think that’s what this company was doing.

It's part of their PR.

When original issues were big in media, they made PR campaign good other workshops were incompetent.

To compare in aviation terms, it was like plane manufacturer claiming Part 145 certified MRO with the necessary type certificates weren't good enough and you should only service at manufacturer - despite having outright bought the machine in total (no leasing)

It should be the job of a rail or a transportation agency (like the FAA does with aviation). They should decide which trains operators can run which trains in a public space. If some of the operators are reckless with maintenance, they would lose the license to operate.

And in fact that's how it's (supposed to be) done.

With the maintenance and repair shops having railway equivalent of Part 145 certification, just without type ratings IIRC.

I'm curious if the "12-month if-s" could activate in a running train. Hopefully not, but somehow I wouldn't be surprised...

Thank you for taking the time to steel-manning this one!

If this were the situation I would expect the train to show operators clear warning messages that service is due, and ultimately a message that says something to the affect of the train has been disabled or put into limp mode until it is serviced. I wouldn't expect these triggers to disable a train to only be discovered later when, for example, a train won't start without warning or stops running when its parked at a different shop to have the service done. I also would expect the service warnings to be based on something like hours of operation rather than calendar time, all heavy equipment I've ever heard of or worked on tracks service schedules this way.

This argument also gets to a more common trend we've had in recent decades where those with authority step on others freedoms because they believe they know better. Individuals should be able to make their own choices and be responsible for the consequences. In this case, the train operator should be aware if the service needs and risks if it is missed. If anything goes wrong they are responsible for it. Even if the triggers to disable the train were put in with the best of intentions, if I were the manufacturer I would worry that installing such a system could potentially put me entirely liable for anything that goes wrong.

> This argument also gets to a more common trend we've had in recent decades where those with authority step on others freedoms because they believe they know better. Individuals should be able to make their own choices and be responsible for the consequences. In this case, the train operator should be aware if the service needs and risks if it is missed. If anything goes wrong they are responsible for it. Even if the triggers to disable the train were put in with the best of intentions, if I were the manufacturer I would worry that installing such a system could potentially put me entirely liable for anything that goes wrong.

This, specifically, is the piece where I think there's some moral ambiguity, and specifically I do not think one has the moral ability to completely disavow the outcomes that the use or misuse of one's product causes, especially when they affect third parties. If you know that use of your product under certain circumstances will cause a large amount of harm to people other than the owner or operator, and you know those circumstances are likely, and you don't do anything to prevent that, I think you have some moral culpability. Whether or not you care is a different story, and this certainly isn't a legal argument, but I think you're responsible for the outcomes of the use of your labor and resources, especially when those are easily foreseeable. I think specifically in the case of selling a train to a municipal train operator - if I told you that the trains in Poland were known for derailing because the national train operations service was financially underwater and never repaired them, would that change your opinion? (It's not true, as far as I know, but would you find it surprising if it were?)

And, absolutely to your first point - if the goal of what you're doing is to prevent unsafe operation of your product in a situation where you legitimately believe it can cause grievous harm to third parties, then yes, you do all the things you say in paragraph one. That's why I'm saying I don't think that's what the train operator was doing, but I don't think the argument is totally cut and dry that the manufacturer has no moral right to stop the trains, and I don't buy the argument that the moment you sell the products of your labor to someone else you fully absolve yourself of the moral liability for the outcomes of the use of that product.

(And again, I'm repeatedly using the word "moral" in here, because this isn't a legal, statutory, or contractual argument, it's purely a moral one. I also recognize the world's a complicated place, we all have to make decisions in which there's not a clear good answer, and nobody lives a truly pure and moral life, so take this in the spirit of an old fashioned debate about how one can live one's best life, and not a specific condemnation or Twitter-esque outlining of what precisely a witch is while one gathers kindling.)

Or, you'd make sure that your liability was limited to one year, absent of any servicing. Maybe you'd have a renewable service contract, yearly, and one of the requirements is local inspection.

>force the owner to get the train serviced.

This is coercion, implemented via subterfuge. It's no different than if they sent guys with sacks full of door handles or whatever to take control of the trains, to accomplish the same result. Or if the client hired staff to crack the provider's server, and either installed ransomware, or stole the solution. Ridiculous. The way to operate is through the laws in question, rather than removing agency from the client.

If the client does not want to pay, you warn them of possible consequences, ask them if they would like to purchase an extra diagnostics package, and remind them that after a year, maintenance of the product is required to bring you back into the picture formally.

Not bad, can you steel man the geo fencing too?

There are reasons you might legitimately geofence your product, to.prevent them being stoken a war spoils like the Ukrainian tractors and harvesters were by Russia last year, or to prevent them being used in a sanctioned country like Iran. I could see a upper level manager looking at a geofence to made for complying with sanctions and turning around and using it as a anticompetitive tool to prevent third party repairs without a second thought.

> If you implement a time based expiry system, and tell about it to the customers that I think is fair.

I remember a discussion of ethics on HN a few weeks or months back.

I said there are actions that promote the common good, actions that do not affect the common good, and actions that harm the common good.

I believe you view of "business" is in the category of harming the public good.

Outsource to the cheapest bidder.

You know the type, that without question mechanically implements exactly what is written in the specification, no matter how wrong it is, technically or morally. If you are lucky that is, and not only get back a steaming pile of bugs.

The bugs in the date checking hints this wasn’t written by the sharpest guy in the room, unless it was deliberately made confusing, to not be easily pattern detected.

I like how these arguments always seem to imply that the software engineers are the ones who have to care way more about morals and ethics than whoever came up with the morally questionable "feature" to implement.

The alternative is that software engineers are resolved of responsibility for only following orders.

Also, I don't understand why so many software engineers are willing to write code that is clearly adverse to users so their managers can get their performance bonuses. What's the upside? Narrowly escape the next round of layoffs so you can do it all over again?

The tech companies are not our friends any more than they are the users friends, they collude against us and treat us as disposable. This will continue as long as engineers do not take a stand.

> I don't understand why so many software engineers are willing to write code that is clearly adverse to users so their managers can get their performance bonuses.

In my experience, the young engineer is hired and bets his/her career on this one position, then they are told to do the dirty work or leave. That's why I resigned from my first software engineering job.

You might be right, and thank you for doing that.

I remember hearing that the number of software engineers doubles every 5 years, meaning that most software engineers will always have less than 5 years of experience. I think it was Bob Martin.

Nobody frames it as an evil thing. It’s spun to sound good the engineers don’t put any critical thought into it.

> The manufacturer argued that this was because of malpractice by these workshops, and that they should be serviced by them instead of third parties.

So libel on top of computer sabotage.

I am Jack's complete lack of surprise. If the Polish government doesn't make an example of this company and cripple them, it will just be a signal to every other industrial manufacturer that it's OK. This is the kind of fuckery that can endanger the continuity of a nation on a wide scale.

Actually, a one-time-only "fix" would be a huge wasted opportunity. This needs to be addressed globally for all bad actors / scalpers, including consumer device manufacturers like Apple, and other assholes like Deere tractors, etc.

We need to bring back and legislate the right to repair and ownership rights.

> a one-time-only "fix" would be a huge wasted opportunity

“One time fix” is not the correct way to see a potential prosecution here. The question is if the system is setup to punish this behaviour or not. A particular prosecution of a particular company in a particular case feels like a one time fix but it acts to demonstrate that this behaviour is not acceptable, and that we have the legal framework and the willingness to punish it. Especially of course if the prosecution is succesfull.

It is a bit like calling a vehicle barrier on the side of a mountain serpentine road a “one time fix” just because it only ever caught a single runaway bus during its lifetime.

I'm not sure a structural fix is needed here. It appears that EU law already requires unbundling service from sales in cases like this, and requires that necessary documentation is made available for third-party repair shops. Sabotaging public transit infrastructure is surely a crime in Poland, and I suspect it won't be hard for the authorities to prove that was what happened here.

Sometimes people do things even when those things are illegal. All that's left to do at that point is prosecute them.

Poland does not have the authority or power to address this “globally”. They do have the authority and power to address the case of this specific train company.

I'm not sure if this is a 'I can't believe they actually did that' moment, or an 'I told you so' moment, because even though it is the inevitable result of tolerating such technology in the consumer sphere, by using it against an entire nation's transportation system I am in awe of how incredibly brazen it is. You don't cut open the golden goose! You had it great being able to screw over consumers but now you have gotten governments and infrastructure involved, and you will reap the consequences.

This might be a defining case that sets the power relation between governments and the comapnies that supply all the stuff they need.

Whatever the criminal filings are, hopefully they get what they deserve.

One can only hope that it will be the outcome... Won't keep my hopes up

If checking if 21 <= day AND 11 <= month AND 21 <= year is the actual logic they used to check if a date is after Nov 21 2021, I'm not sure they should be writing code that runs on a train.

(Edit: Fixed inequality direction.)

This is due to IEC 61131-3 environment - they are getting date value as 3 separate variables from system.

Similar stuff is common in PLCs all across the world.

Sure, but regardless of input format, the test in GP's comment would indicate that October 22, 2021 is not before November 21, 2021.

The right test is something like "year < 21 || year == 21 && (month < 11 || month == 11 && (day <= 21))". And yes, probably refactor that logic for clarity, but from this form you can see that it is not equivalent to the test above.

The code as deployed has errors (the link actually shows ghidra decompile of relevant portion). Months & days are counted from 1.

The error is what makes it so that instead of failing "unfixably" once 2021-11-21 happens, it instead simulates failure between nov. 21 and dec. 1, and dec. 1 and january 1, every year starting with 2021.

Yes. Its that second part that makes me unsure. Not that they're checking month and day and year.

DAY >= 21 AND MONTH >= 11 AND YEAR_2digit >= 21

The condition yields true from 2021 on for:

nov. 21 - nov. 30

dec. 21 - dec. 31

This might be on purpose if they’re merely trying to make the third party maintenance company look unreliable.

Just in time for the Christmas rush. This calls for criminal prosecution.

I reversed the direction of the tests, as pointed out, but the original logic would fail to detect that say June 22, 2022 is after 21 November 2021.

(you have swapped the direction of the inequalities -- not that it makes the code any more correct)

They shouldn't, but it's an industry with heavy code quality regulation, which naturally ensures much worse code quality.

I would think any competent software engineer would refuse to write code that could cripple their own nation's civilian transport because of their boss's greed, and thus the company had someone do it who was idiotic enough to try.

There are plenty of competent software engineers that do extremely unethical things for money, look at the defense industry.

Context: https://www.theregister.com/2023/12/08/polish_trains_geofenc...

They also claimed to have found an undocumented key combination in the cabin controls that would unlock the trains.

I hope everyone who operates these trains now knows what that is.

The combo was removed in software update, unfortunately.

I wonder what necessitated the update; or was it a forced one, as seems typical these days?

The other question this begs is, what other "interesting" hidden features did they add in the update?

To summarise from what Dragon Sector published:

- After one of the trains was "restarted" with the unlock code, some of the trains that went to manufacturer for various reasons returned with software that had the unlock code patched out

- Some trains that have locked up outside of Maintenance halls and which were sent to NEWAG returned with added GPS geofencing for the lockup logic and changed time period (previously the lockup engaged if train didn't run for 10 days)

- More cases to be presented at 37c3

I really really hope it was the Konami cheat code.

Is the manufacturer still disputing that they're the ones who installed that logic bomb?

In public, yes, also their probable paid PR sock puppets on social media.

Is this considered a legal mechanism for a public service? If this is the code it has to be in some documents as legal binding already right otherwise they just wouldn't have been able to do this.

Legal mechanisms seem moot in a situation of sabotage against the national infrastructure of a state. The word that's in th back of my mind right now is, at a stretch, "terrorism".

Nah, it’s clearly extortion, and clearly not terrorism. In wartime it could also be sedition. But the intent is to extract euros, not terrify a populace into political submisssion.

Fair enough. I dislike misuse of the "terrorism" word too. But if someone else had done exactly this, for slightly different motives...

I would say your original choice is better. Sabotage. And for the purpose of extortion.

Such are some of the few times when "making an example" is both necessary and justified.

Will be curious to see what happens.

whether or not it's terrorism, at an absolute minimum it's fraud

Thomas had never seen such bullshit before

I wish these neighbourly hackers could go after Apple and see if they can get to the bottom of this: https://news.ycombinator.com/item?id=36926276

What bottom is there to get to?

It’s due to calibration data.

If the parts match up it uses the available calibration data, if not it ignores calibration data and reverts to uncalibrated state.

IMHO that's just the "plausible deniability" excuse --- look at the evidence presented there in a bit more detail, and you'll see it starts leaning towards being deliberate. The only "calibration" is the serial number. If these Polish hackers find code that effectively does "if serial doesn't match, slightly perturb the points", it would be certain.

I’ve looked at “the evidence presented” in great detail when it made the news cycle in tech circles, I’m also pretty familiar with Apple’s calibration process and have gone through what information is publicly available via websites that document more intricate details like The Apple Wiki (they’re surprisingly accurate by the way).

I’ve touched upon some of the details on calibration 4 months ago here on HN when it was a trending topic[0].

0: https://news.ycombinator.com/item?id=36936697

This is just the point of the iceberg for Apple.... Someone just defeated one of their DRM for the sensor that detects when your close the lid on your laptop (if you used a replacement part that was not bought by apple)

Wild. Although I've heard Apple have a hardware shutoff for the microphone when the lid closes so perhaps DRM on the sensor is less insane from a security perspective than it might first appear.

Here's the video about it: https://m.youtube.com/watch?v=Lxo6l_whDeE

Where can I learn more about this?

A good start is https://social.hackerspace.pl/@q3k/111528162462505087 and https://badcyber.com/dieselgate-but-for-trains-some-heavywei...

There were also multiple threads on earlier parts of the story, but those are the "origin" english language versions.

Next week, it will be presented at this year's 37C7 at Chaos Computer Club. Get your popcorn ready.

<https://news.ycombinator.com/item?id=38730275>

Richard M. Stallman has a proposed fix to this through 4 software freedoms.

Maybe they were just giving trains Christmas off?

My people call that “a load of horseshit”

"you wouldn't download a train would you?"

Hope they get the book thrown at them for such blatant attempts at ripping off the communities they serve. Greedy af people

Reply on news.ycombinator.com