My Thoughts on ChatGPT | Hacker News Reader

My Thoughts on ChatGPT

calnewport.com

5 points0x54MUR4111 comments

I strongly disagree with Cal on most of his points here; Cal wrote an article in the New Yorker yesterday that effectively recapitulates the "Its just guessing the next token" argument. In his example, he explains that ChatGPT couldn't possibly understand how to remove a PB&J from a VCR, but asking GPT what to do discredits this analogy almost entirely. It is true that GPT doesn't have servos with feedback or other hardware to remove a sandwich itself, but it can certainly explain the process including how to clean the leftover residue and follows the details by suggestion you seek out a professional technician.

In the same article he admits ChatGPT is immensely complex and that "any number of humans" could not reproduce the rules by which it works; but then goes on to assure the reader that is naive and is not a threat to their jobs or safety.

This seems really disingenuous to me given he's a professor of computer science at a respected university. This kind of cynicism, that you might call naive, leads readers to put aside concerns about safety.

In my opinion, 'It's just guessing the next token' is true.

I agree, but I'm also resigned that these mainstream venues like New Yorker have a speed limit on how fast they can push the window.

For example they have moved in the last two months from "ChatGPT Is a Blurry JPEG of the Web" to "What Kind of Mind Does ChatGPT Have?" so that's some kind of progress. In two more months maybe they will put another take that's even better.

The author might not fully grasp how many well-paid white-collar jobs, for example in the medical sector, involve tasks that GPT-4 can handle quite easily. Now imagine GPT-Medical, a digital doctor sidekick, conducting patient interviews and figuring out diagnoses way faster and with more patience than a human physician. The catch? You still need a physical exam! Combine that with GPT-Medical's consultation, and there you have a solid treatment plan. The physical exam part is likely the saving grace for doctors in the near future. But, mark my words, it's just a matter of time before the medical world starts embracing systems like GPT-4, trained on medical interviews, notes, and case studies. Increased productivity will be demanded by penny pinching administration.

This is close to absolute euphoric nonsense.

Who is going to be held liable when GPT-4 misdiagnoses a patient or hallucinates the wrong advice and risks the patients life? I don't think you would solely trust a black-box AI to fully replace a medical professional or legal professional with the absence of liability and regulations.

I'll go a step further, I also guarantee that you would not trust medical or legal advice from GPT-4 or any black-box AI to inject yourself or take a certain amount of pills without going to a human doctor for a second / third opinion.

Fundamentally these systems are unexplainable neural networks which cannot reason about their own decisions transparently and regurgitate back whatever it has been trained on. Furthermore, such systems like LLMs are completely untrustworthy for this serious high risk applications other than summarization of text.

It's going to be baby steps of course. For the specialties that I am most familiar with, note writing will probably be the first thing to be automated. I can feed GPT-4 a garbled mess of a note now and it will make a pretty good note today (a made up visit, with well written examples in the prompt). After Epic gets permission to use real notes, it will be just a stack of dominoes that falls down after that. And it's already happening [1].

Once you can throw crap against a screen and epic makes a decent note out it, history gathering will come next. "GPT-Medical" will go through prior notes and summarize them for the current visit. And after that is fixed, pts will be required to do an interview with "GPT-Medical" before the visit. Of course there will be real physicians working with these systems as the care is advanced.

[1] https://www.beckershospitalreview.com/ehrs/epic-to-use-micro...

I see that there is a lot of content created by ChatGPT. In the future, it may become difficult to find high-quality content

You think it may become difficult to find content created by ChatGPT?

No I mean content hight quality created by human instead chatGPT

Imagine after GPT-5 and it makes the highest quality content, and we do a search and find one written by human and we are like 'what is this trash was it made by human or something?'

hacker news thread on his associated new yorker article: https://news.ycombinator.com/item?id=35560019

Reply on news.ycombinator.com