I see a lot of people getting confused, and the contention here is not that ChatGPT helped prepare for martial law in any way, but the fact that someone knew about it happening before it happened. Not really related to ChatGPT IMO.
> At 8:20 p.m. on December 3rd last year, when Chief Lee searched for the word, the State Council members had not yet arrived at the Presidential Office. The first State Council member to arrive, Minister of Justice Park Sung-jae, arrived at 8:30 p.m. It is being raised that Chief Lee may have been aware of the martial law plan before them. Martial law was declared at 10:30 p.m. that night.
[0] https://www-hani-co-kr.translate.goog/arti/society/society_g...
Considering that the whole affair is considered treason now and we now know of memos talking about "collecting persons of interest, put them in a ship and explode it" (no, seriously) --- there's a very good chance that the inner cabal who planned the coup would get life sentences or worse.
(I'm not sure how important was the person mentioned in the article - there are just too many bastards. It does seem like a random article to show up on HN.)
That’s the dangerous part.
ChatGPT doesn't get nuances. It doesn't get subtle differences. It also gets large amounts of information wrong.
The machine seems to be unable to say or even detect that it does not know. At the same time, it communicates in flawless English (or whatever the current setting is), which is a trait we tend to associate with highly educated people from the real world. This short-circuits our bullshit detectors a bit.
You might, and I try to. Humanity as a whole? In practice, highly confident people who are totally sure but wrong, still get listened to over people who are humble and aware of their limits.
Humans also short-circuit each other's BS detectors.
People intuit that Wikipedia is written by people, so they can apply that knowledge appropriately.
For some reason, most people have a knee jerk reaction to a fully synthetic statement that biases them strongly towards the assumption of veracity.
I always think of LLMs as “my functioning alcoholic veteran friend bob, who has several PHDs and was blown up a couple of times in Iraq”. That seems to be a good framework in order to intuit the usefulness of llm generated output.
This. We know that computers are very good at actual computation, and we don't expect them to go completely haywire in conversations either.
Though this is beginning to change, with the observation of just how blatant some of the hallucations are, accusing random people of serious crimes etc. But the pro-computer bias is still strong.
There was an awful case of a system in the UK which accused postal officers of defraudation. The software malfunctioned, but people were indicted and punished by the courts relying on infallibility of computers, and some of the innocent victims committed suicide out of shame.
1. LLMs are put in a position where everything they say is clearly based on encyclopedic knowledge of absolutely everything
2. LLMs try to use language that is very general, helpful and friendly, and as a result end up not properly portraying nuances, like "sometimes", "in this case", "not always", etc.
3. Humans are capable of saying "I don't know", or "I think XYZ but I'm not sure"
4. Humans convey that they aren't sure by lack of nonverbal confidence
These are differing sets of skills and issues. LLMs dont behave like humans, they don't solve things like humans, and people take what they say at face value by default.
This is true, if you decide to take a ChatGPT answer at face value without any further work. Personally I find it useful sometimes to ask an LLM a question, get an answer and the verify that answer for myself. Doing web searches and pulling together relevant information to get the answer for a question can be harder than getting an answer and then looking to verify it. Perhaps something like that was going on here, impossible to know of course.
plus, now i've been biased by the immediate response. if it says "these CVEs don't have vulnerabilities" then I'm now thinking they're probably okay and just need to validate, instead of starting from zero and doing due diligence. this will lead to confirmation biases or laziness.
you aren't gonna look up if that little detail is right; you're gonna slowly absorb more and more subtly false info.
Actually, there is an incentive to remove edits in Wikipedia if you want to be part of the ego-fueled bureaucracy that considers WP as their property.
ChatGPT is consistently lying (hallucinating), sometimes in small ways and sometimes in not so small ways.
The reason I do it in combination with normal search is that normal search will often get clogged up by 3rd party websites and at best lead you to only the main legislation. The LLM is likely to name you the main legislation so you can search for it directly by name and also mention other major related pieces.
We can't infer any amount of trust from this episode except the trust to put the data into ChatGPT in the first place, and let's be honest: that ship sailed long ago and has nothing to do with ChatGPT.