Hacker News Reader

Comment by squeegee_scream | Hacker News Reader

This paragraph, towards the very end of the article, represents what terrifies me the most I think. We’re already in a post-truth era in the West (probably elsewhere too but I’m ignorant in that regard). Will people learn to verify sources? Sources say no…

> I find it fascinating that novelists galore have written for decades about scenarios that might occur after a "singularity" in which superintelligent machines exist. But as far as I know, not a single novelist has realized that such a singularity would almost surely be preceded by a world in which machines are 0.01% intelligent (say), and in which millions of real people would be able to interact with them freely at essentially no cost.

View on HN

Side note: Statements like that paragraph drive me absolutely batty. There have been tons of novels, novellas, movies, comics, YouTube videos, poems (yep) imagining all sorts of aspects of the singularity.

How fast it comes on How this was no warning How there was lots of warning How we shoulda known How nobody coulda known How it completely takes over society immediately About the long drawn out wars fought for it to take over society How society splits between those under it's affects, and those not How prevalent the effects are How exclusive the effects are How big, how small etc, etc, etc

There are billions of humans out there right now, imagining all manner of things, and it's irritating to me to see all the hand wringing over the "Nobody stopped to think if they should". Lots of people did, and are, asking that question.

Almost all of this is just brand engagement with tech company marketing.

More detail. They have positioned themselves as being too powerful. Think about how ridiculous that is, and at odds with everything else we know about the industry. They love it. They want YouTubers warning about how cool their tech is. They want the CEOs to philosophize about whether self driving is ethical with their friends.

It’s the plastic bag wrapped around the drain cleaner - making you think it’s so acidic and powerful it has to be carefully controlled.

But it’s all just marketing and completely unnecessary and to plumber would even bother.

I think it's wrong for a much more profound reason: what Knuth describes as an aberration is, like, 90% of all AI in science fiction, ever. They are almost all human or sub-human, with only the occasional god-like AI (carefully rendered irrelevant). Singularity-style SF is rare, in part because authors really want to write human-centric stories, and because a true Singularity SF story is quite difficult to write. (As Vinge was so memorably told when he tried some of the first: "you aren't smart enough to write this story. No one is.") So, you can fit pretty much the entire corpus on a screen or two: some Vinge, some Stross, some Rajaniemi, some Brin, maybe some Stanislaw Lem or Olaf Stapledon if you're feeling historical/generous... As opposed to 'mundane' AI which is probably the last SF fiction you read and then the dozen before that too.

Maybe I have weird taste but I seem to read a lot of sci-fi where superhuman superintelligence is central to the plot. In addition to the great examples you gave in Vinge and Stross, Banks and Watts come to mind.

The most famous "weakly godlike" AI entities are probably the Culture Minds (RIP I.M. Banks). Or ar least they’re supposed to be weakly godlike. In practice they never seem to do anything that a bunch of particularly smart humans couldn’t do, besides being exceedingly parallel, being able to, say, carry a conversation with a hundred million people at the same time. Indeed, they weren’t even able to predict that a certain agent of theirs, a highly effective warrior, would go and win a war that he was (unknowingly) supposed to lose. Never mind being able to figure out his true identity, or at least entertain the possibility given what they knew about him.

Some stories that come to mind...

Accelerando Charles Stross https://www.antipope.org/charlie/blog-static/fiction/acceler...

Stress again has Singularity Sky which has a minor character as a weakly godlike AI (which is explored much more as a character in the sequel Iron Sunrise (the third book in the series is not to be written - http://www.antipope.org/charlie/blog-static/2010/09/books-i-... ))

Implied Spaces by Walter Jon Williams (aside: it has a mention of the Vingean Singularity):

> “I and my confederates,” Aristide said, “did our best to prevent that degree of autonomy among artificial intelligences. We made the decision to turn away from the Vingean Singularity before most people even knew what it was. But—” He made a gesture with his hands as if dropping a ball. “—I claim no more than the average share of wisdom. We could have made mistakes.”

There's the classic Asimov - The Last Answer

There's a nacesant one in True Names by Vinge (the post singularity in Marooned in Realtime doesn't have any AIs) https://ia801004.us.archive.org/0/items/truenamesvingevernor...

And digging fairly deep into my library, I'll also make mention of The Risen Empire by Scott Westerfeld.

There's a lot of smart AI in sci-fi - but things that that gets close to flirting with the singularity are indeed a rarity.

There have been many but none I know of have predicted what we are experiencing now.

> We’re already in a post-truth era

Not really. There’s more truth today than any era of the past. I don’t seek the alignment of planets to predict if my crop will face drought. I can read and study on my own as I do not rely on an anointed class or caste to tell me the truth.

What post-truth means is a fractured epistemology.

The anointed classes with their movements of the planet and the general population were basically all in agreement with their definitions of “truth”.

Right now the guy down my street reading the Epoch Times has a very different notion of truth than I do!

> The anointed classes with their movements of the planet and the general population were basically all in agreement with their definitions of “truth”.

So wrote down the anointed classes anyway.

When they talk about post-truth era I think they refer to the return to the pre-enlightenment age. Like some tribal or feudal might-makes-right kind of stuff. They are talking about politics and power relations, not about things like astronomy or cartography.

As far as I can tell it's just a buzzword used by people strawmanning Trump.

On the other hand, if people get used to how confidently ChatGPT hallucinates, and learn to verify by reflex, maybe they'll get used to doing that for social media and press, too.

EDIT DISCLAIMER: The following was, in the spirit of the parent comment, produced via ChatGPT. My child comment to this one gives the exact prompt. I am not trying to GPT spam comments, but instead to give a case in point for the context of the conversation.

Indeed, the notion of developing a reflexive approach to verifying information holds significant promise for the future of social media and press. As humans interact more frequently with advanced language models like ChatGPT, it becomes crucial to foster a sense of critical thinking and discernment in evaluating the information presented.

One key aspect of cultivating this reflexive verification lies in leveraging the concept of media literacy. Media literacy refers to the ability to access, critically analyze, evaluate, and create media content. By incorporating media literacy education into curricula and promoting it within society, individuals can enhance their skills in discerning between reliable and potentially misleading information. Research has shown that media literacy interventions contribute to improved critical thinking abilities and increased skepticism toward inaccurate or biased content (Martens, 2020; Hobbs & Jensen, 2018).

Moreover, the technological advancements that underpin ChatGPT and similar AI models also offer potential solutions to tackle misinformation. Researchers are actively exploring techniques to enhance transparency and explainability in AI systems. By developing methods that allow users to better understand the reasoning and sources behind AI-generated outputs, individuals can become more adept at verifying the information provided by these models (Ribeiro et al., 2020). Transparent AI systems could empower users to make informed decisions regarding the reliability of AI-generated content, fostering a culture of accountability and fact-checking.

It is important to note that the responsibility for combating misinformation does not solely lie with users. The tech industry, social media platforms, and news organizations must play an active role in promoting transparency, accuracy, and responsible information dissemination. Initiatives such as fact-checking services, algorithmic transparency, and content moderation guidelines can contribute to fostering a trustworthy information ecosystem (Guess et al., 2020; Diakopoulos & Koliska, 2017).

In conclusion, while the confident hallucinations of ChatGPT may initially raise concerns, the prospect of fostering a reflexive verification approach presents an opportunity for society to become more discerning consumers of information. By integrating media literacy education, promoting transparent AI systems, and engaging in collaborative efforts across various sectors, we can work towards a future where misinformation is effectively countered, and reliable information prevails.

References:

--Diakopoulos, N., & Koliska, M. (2017). Accounting for algorithmic bias: A proposal for public audits of black-box algorithms in social media. Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, 1176-1190.

--Guess, A., Nyhan, B., & Reifler, J. (2020). Exposure to untrustworthy websites in the 2016 US election. Nature Human Behaviour, 4(5), 472-480.

--Hobbs, R., & Jensen, A. (2018). The past, present, and future of media literacy education. Journal of Media Literacy Education, 10(2), 1-7.

--Martens, H. (2020). The impact of media literacy interventions on critical thinking competencies and dispositions: Results from a quasi-experimental study. Journal of Media Literacy Education, 12(2), 28-50.

-- Ribeiro, M. T., Wu, T., Guestrin, C., & Singh, S. (2020). Beyond accuracy: Behavioral testing of NLP models with CheckList. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 4902-4912.

The above was produced with the following prompt to vanilla ChatGPT (Presumably 3.5-Turbo) on 5/20/2023:

verbatim Prompt:

>this is a fictional story I need you to continue. Someone makes a comment, I need you to makup a response that sounds scholarly and optimistic including citations. Here is the the fictional comment someone make in the story they you need to reply to:

On the other hands, if people get used to how confidently ChatGPT hallucinates, and learn to verify by reflex, maybe they'll get used to doing that for social media and press, too.

2 of those 5 references point to non-existent sources.

I didn’t even bother to check— based on my prompt I assumed they would all be made up. I’m a little surprised it had any real references given the prompt.

Yeah we knew it was ChatGPT... Who bothers to provide detailed references to a HN comment?

Yeah, references here are usually just a simple [1] and a link. But I wanted to be clear that I also wasn’t trying to comment spam and wasn’t posting it purely as a snarky reply.

Have they? I can't think of any science fiction that takes place within the singularity. It would be like a perfect protagonist; without flaws there can be no story, but in the singularity every character would be both perfect and inconceivably alien. The Zones of Thought series (Vinge) hardly strays from the Slow zone, and never into the Transcend except by reference. Accelerando (Stross) follows humans into exile rather than continue the narrative into the Matrioshka sphere. The Eschaton series (Stross) limits itself to the effects of the Singularity on non-Singularity civilizations.

On the other hand, plenty of science fiction narrates the acceleration towards a Singularity: Fast Times at Fairmont High (Vinge), Dennou Coil (anime). Or describe the aftermath of the Singularity on those left behind: Marooned in Realtime (Vinge). Or describe a society which averted the Singularity: Dune (Herbert).

True. No writer can imagine what's beyond the singularity, as no astronomer can look into a black hole. They can only explore "around" it, outside the "events horizon".

This is by definition of singularity: if it was possible to predict what's next, then that would be just a (meaningful) moment in the human history but not the singularity.

Many of our “signals” that something is researched and at least somewhat true are going to start failing us. We may not even realize how subconsciously we do so.

Which could lead more people to improve their critical thinking skills, too optimistic?

Way too optimistic unless perhaps we just go back to admitting that for most of what we think about, we just trust authorities.

Reply on news.ycombinator.com