Human-in-the-Loop Is Not the Same as Judgment-in-the-Loop | Hacker News Reader

Human-in-the-Loop Is Not the Same as Judgment-in-the-Loop

kunyuan.substack.com

16 pointshufdr5 comments

My main worry about AI is people misunderstanding its limitations and then deferring their subject matter expertise judgement to a hallucinating, amnesiac used car salesman bot. Or taking humans out of the loop such as orders for lethal systems to kill humans without rigorous intelligence.. a decade ago, US mil drones absolutely required an officer to give the command. Ukraine, modern NATO/US.. I'm not so sure that boundary is respected anymore. Judgment and responsibility cannot be outsourced morally/ethically in a multitude of situations. Perhaps it's fine if AI manages meeting schedules or decides what you're having for lunch.

Historically, tools extended human capability. The expectation was always that humans remained responsible for the outcome. What's new is not that AI helps us think. It's that many people seem increasingly comfortable letting it think on their behalf.

Tools can also dull human capability. Calculators made it less important to be able to do math in your head. But how many people are pulling out their calculator at the grocery store or in other everyday situations where it would be helpful but not crucial? Now imagine this effect but for general thinking and creating.

That's what worries me about AI. Not that it will replace human judgment, but that it may gradually remove the friction that judgment depends on.

I'm astonished just how quickly LLMs have eroded critical thinking, perhaps exposing my own biases and assumptions about just how much I believed or understood others critically evaluated and critically judged something.

I also wonder how much of the population is truly introspective. Maybe that's some of the root cause here?

I am deeply, deeply concerned that LLMs are creating a generation... no... zombifying several successive existing generations into intellectually-devoid vassals completely unable to think for themselves, or approach novel and difficult problems without hallucinated assistance.

I'm also astonished, maybe I shouldn't be at this point, how a significant portion of people I know - some of them I would have expected to know better - don't realise that an LLM is a statistical word generator that is in no way equitable to the intelligence of a human being.

Then again, human intelligence exists on a bell curve; I really start to wonder exactly where the peak of that curve actually lies...

Reply on news.ycombinator.com