"Cognitive surrender" leads AI users to abandon logical thinking, research finds
arstechnica.com
arstechnica.com
Then, we have meetings where we will ask a basic but specific question about what they want us to make, and we're just met with blank stares. They have no answers, because they've never actually thought about it.
And then everyone else needs to do the thinking for them.
That said, it can be very hard to be a good client. Writing requirements (whether for art or engineering) is something that on average, people are very bad at. And often you will only find out you cared about something after you see it (oh god I am so bad at this, especially because it's often delayed, so I will go 'looks good, no notes', then like a day later go 'oh wait, actually...'), which is why having a healthy dialogue and rapid feedback loop is so valuable to any project.
On soft topics like politics models say something different depending on the prompt or the latest fine tuning. As Microslop say in its TOS, AI is for entertainment only.
Software is unfortunately dominated by fakers. Paul Graham said in one of his essays that the C students command the A students. Back then it meant MBA > software engineer. Now it means that the bullshitters in software command the intelligent ones.
You have to resist daily and expose the frauds if this profession is to be saved.
The cognitive surrender is the most predictable outcome. Many here will claim they'll rise above the path of least resistance and use AI responsibly, and even if that is true for many here, think about the most typical worker. Those who only want to go home at 5 after putting the least amount of effort into their job. Our society is about to be rewritten by them.
In general, I believe the problem of our time (sociopolitical divide, echo chambers, propaganda, people getting pulled to extreme viewpoints ...) isn't so much about the difficulty to access truthful information (most people would know how to fact check their beliefs and assumptions given enough time and motivation), but about the constant information overload that makes this process impractical. You are essentially softening your brain into accepting large volumes of information as facts, unchecked, and pretend that it's a good thing. You essentially don't know anymore the extent of what you know. Worse, you no longer know how you came to know, because the underlying principles and processes of knowledge (natural laws, models, theories, ...) were not involved in the learning (you could assert that day means light, and that night brings darkness because you have seen it repeated extensively and convincingly, but you wouldn't internalise this knowledge through the model of the earth orbiting the sun, and so you wouldn't know how to generalize from this knowledge into thinking about seasons or do any abstract reasoning on your own).
That is to say, we should be much much more cautious about what and how we read and learn about stuff.
This is most definitely the issue, and I’d would say you can go a step further. There are groups of people who don’t know how to stay safe in the information environment, while others who understand how to shape the environment.
The latter group is able to shape the content available to the former.
https://news.harvard.edu/gazette/story/2018/10/network-propa...
If you stop doing long division by hand and use a calculator is that cognitive surrender or just normal life? And if the calculator has the wrong answer and you accept that is that that surprising?
In terms of the danger of trusting stuff without double checking there seem more problems with Fox News etc. than AI which tends to be fairly neutral if sometimes wrong.
yes, you have surrendered to the assumption that the calculator can calculate quicker and more accurately than you can, almost invariably correct.
Now people are surrendering to the assumption that the AI knows more and can reason from what it knows better than they can.
What you have discovered is a recipe not for memory, but for reminder. And it is no true wisdom that you offer your disciples, but only the semblance of wisdom, for by telling them of many things without teaching them you will make them seem to know much while for the most part they know nothing. And as men filled not with wisdom but with the conceit of wisdom they will be a burden to their fellows."
- Plato
I think no reasonable person would be against literacy in the modern world and similarly we will continue to adapt to new technology and be the better for it.
I suppose part of the reason so many people are so dangerously trustful of LLMs is because they assume that if the LLM was put out there by decently responsible humans (doubtful, but understandable), then so too should the LLM be decently responsible? The analogy does break down there.
In other, moderate older news, cognitive surrender leads TV users abandon logical thinking, research finds.
In other, even older news, cognitive surrender leads newspaper readers abandon logical thinking, research found.
Shall I go on with how cognitive surrender leading to abandon logical thinking spreads out in history, AI being nothing special in this regard?
Here the user is actively generating
Cite your sources?
You never heard of newspaper reporters? TV shows producers? Writers? These are the actively generating ones. You will also have consumers. Not looking to generate but looking to... consume.
The same with AI. AI is just an evolution of production and consumption. Not all use AI to create. Most use AI to consume what others created: models, harnesses, knowledge.
AI's existence is like the mental equivalent of a heavy weighted barbell that also happens to be edible and tastes delicious. You could use it in a way to get in great shape, you could also use it in a way where you get type 2 diabetes.
It is up to you and your own experiences to decide how that is likely to go for most people.
I mean, you could read the papers themselves, they aren’t inimical to your position by nature.
For example, one of the more salient results is that the more confident you are in AI, the less likely you are to check the output.
When a new invention arrives on the scene, its properties need to be mapped.
Step 1) read the article
Step 2) have a frontier AI model summarize the article
Step 3) find any 'bullshit' that it comes up with
Step 4) do it 5 more times in a row because obviously the non-determinism will make it say something different each time and it should bs you at least once since it's simply a bs machine anyways
Step 5) report your findings