Hacker News Reader

Comment by phil21 | Hacker News Reader

> I am a software developer and if I write some piece of medical software with bugs that are so blatant in it, I'd see hell up to potentially being sued into oblivion

Based on watching the medical software field as a consumer (patient) and friends who are doctors, this is a fantasy. The quality of software in this field is abysmal and there seems to be almost no repercussions to those who develop or sell it.

Which is precisely why this sort of thing can be rolled out without much fear by those pushing it.

View on HN

That's besides point, the point is that classical software bugs are contained and we have process and understanding allowing us to discover them, triqge and fix. For AI based sw you can guess and try another prompt.

Why is that "the point"? You are the second person in this thread to insist on owning it.

I get why it's problematic, obviously, but if it produces statistically better results (which I have no idea of), I don't think it's right to just write it off because of this.

Because you cannot hold people accountable, since it isn't "people". We humans are emotional creatures, and we will always prefer systems where someone can be held accountable for mistakes. The developer, the doctor. Whoever.

I do not oppose AI integration; I'm not a Luddite. But having a "move fast, who cares if a couple die" isn't the way to go with sensitive fields, like the medical field.

I suppose we will come up with proper responsibility-hierarchies and guardrails around AI, but until then, people have a right to complain about the lack of them.

The case of Therac-25 is required reading for a computer science ethics course. (How requisite a CS ethics course is a different story entirely though.) It's not a fantasy for more regulated medical devices eg pacemakers.

The software is heavily regulated for medical devices. Saying an MRI machine has bad software seems highly unlikely to me. This is of course different from Epic, but even then as a patient MyChart is really not that bad

Isn’t medical software highly regulated ? You sure there is no fear ?

Reply on news.ycombinator.com