List of Citogenesis Incidents
en.wikipedia.org
en.wikipedia.org
Dystopia? Idiocracy? I don't know, but I don't like it.
Our species has obviously managed to make it pretty far without facts for the longest time. But we've comfortably lived with easily verified facts for 20-30 years and are now faced with a return to uncertainty.
If I had to guess, we'll see stricter controls on institutions such as Wikipedia that rely on credentialism and frequent auditing as a means to counter the new at-volume information creation capacity. But I don't really have the faintest idea of how this will turn out yet. It's wild to think about how much things are changing.
They're better than Wikipedia... but only barely.
In the end you use Wikipedia and an encyclopedia the same way: to get a broad understanding of a topic as a mental framework, then look at the article's citations as a starting point to find actual, citable primary sources. (Plus the rest of the library's catalog/databases.)
As far as I am concerned AI responses will never be reliable without verification. Same as any human responses, but there you can at least verify credentials.
But that statement is neither, so it must be false…
1: How A.I Will Self Destruct The Human Race (Camera Conspiracies channel)
Pretty sure we just called it the 90s.
I really don't understand how anyone can have such a positive impression. I refuse to register an account just to try it out myself, but that isn't necessary to form an opinion when people are spamming ChatGPT output which they think is impressive all over the Internet.
The best of that output might not always be possible to distinguish from what a human could write, but not the kind of human I'd like to spend time with. It has a certain style that - for me - evokes instant distrust and dislike for the "person" behind it. Something about the bland, corporate tone of helpfulness and political correctness. The complete absence of reflection, nuance, doubt, or curiosity with which it delivers "facts". Its refusal to consider any contradictions feels aggressive to me even - or especially - when delivered in the most non-judgemental kind of language.
It is like the text equivalent of nails on a chalkboard!
English speaks will probably never realize this, that most kids need to say learn English first, then programming.
When I was homeless, I asked around on the internet for a source for who said that. There is a real incident where a British official said something like "We shoot people who do that" but it wasn't about cannibalism.
But the toxic classist forum where I asked initially replied to me with basically "You're just a stupid homeless person misremembering that." No, I read it in my teens when I had a near photographic memory and was one of the top students in my high school class and everyone respected me as one of the smart people, long before the world decided I was some loser making things up. I'm quite clear the anecdote in the book was about cannibalism.
There is a real historical incident similar to it, but the book got the details wrong.
I have also read some crazy accounts of how Einstein's Theory was proved because the solar eclipse bent light so much, you could see a star that our Sun should have been obscuring.
Humans tend to believe things we read. If it's in writing, it has some kind of authority in our minds.
This is often not the case and we need to get better about recognizing that a lot of "writing" on the internet is just modern chit chat and not reliable.
(inappropriate and maybe even criminal in the presense of children, unless you're straight & monogamous, in that case it's fine)
https://www.oxfordbibliographies.com/display/document/obo-97...
FWIW, this was the observations during the solar eclipse of 1919, by Eddington and a bunch of less famous people. It apparently made headlines at the time.
The British officer story happened in Korea IIRC, the custom at the time wasn't cannibalism, but that women who lost their husbands would be killed to join them in the afterlife.
The discrepancy between Mercury's perihelion precession and Newtonian gravity had been observed long before Einstein developed general relativity. Calculations under general relativity correctly determined Mercury's perihelion advance, but general relativity did not predict the perihelion advance, since it was already known.
https://en.wikipedia.org/wiki/Perihelion_precession_of_Mercu...
All I'm saying is some accounts seem to really exaggerate how much this effect was.
In what sense do you use the word "fabricated"? In the sense that it invented a falsehood with an intent to deceive, or in that it says things based upon prior exposure?
There is information stored in its model. That information might not be correct.
There is no way even at any level of abstraction and squinting just right to use a term like that for what chatgpt is or does. It's fancy auto-complete. Literally matching and mashing up patterns against other writings by probability. That's it. Auto-complete is neither understanding nor believing.
I'm not sure how that's productive, but feel free.
That's the harm. "quacks like a duck" is not good enough to just let people operate as though it's a duck, and they are, and it's not ok and it's not harmless and it's not their own fault, it's yours and mine.
If you have to use anthropomorphic terms, then there's only one that applies: it lies.
Nothing it tells you is in any way shape or form "true", it's only ever plausible, and if it's true, that's still just a coincidence because it has no concept of data validation against reality. It's just an autocompleter; it's algorithmically incredibly simple software that's been written exclusively for the purpose of "finishing a story, given a prompt" and the fact that the prompt can be in the form of a question makes zero difference for that, so that's the part that ChatGPT leaned into hard.
Using anthropomorphic terms is a bit cringy at best, but in general, they actively interfere with both people's understanding of what these things are, and their ability to talk about them based on, ironically, a true understanding of them, rather than people's hallucinations about what this current generation of LLM autocompleters is.
No, you're pretending something is settled as "incorrect", when it's not, trying to unilaterally force one viewpoint on the issue. "It's just an automcomplete and cannot believe anything" is not something agreed upon by all experts/philosophers of LLMs/consciousness. Some "behaviourist" philosopher might easily agree that ChatGPT does indeed believe it, for example.
> But quite often in the AI literature the distinction is blurred in ways that would in the long run prove disastrous to the claim that AI is a cognitive inquiry. McCarthy, for example, writes, '-Machines as simple as thermostats can be said to have beliefs, and having beliefs seems to be a characteristic of most machines capable of problem solving performance" (McCarthy 1979).
Here's[4] an article talking about ChatGPT specifically, asserting that philosophers like Gilbert Ryle[5], who coined the phrase "ghost in the machine, agree that "ChatGPT has beliefs":
> What would Ryle or Skinner make of a system like ChatGPT — and the claim that it is mere pattern recognition, with no true understanding?
> It is able not just to respond to questions but to respond in the way you’d expect if it did indeed understand what was being asked. And, to take the viewpoint of Ryle, it genuinely does understand — perhaps not as adroitly as a person, but with exactly the kind of true intelligence we attribute to one.
[1]https://www-formal.stanford.edu/jmc/ascribing/node4.html
[2]https://en.wikipedia.org/wiki/John_McCarthy_(computer_scient...
[3]https://www.cs.tufts.edu/comp/50cog/readings/searle.html
As for the Chinese Room argument: that's literally the argument against programs having beliefs. It's McArthy argument for demonstrating that even if the black box algorithmic system seems to outwardly be intelligent, it has demonstrably nothing to do with intelligence.
There is no such festival. An anonymous Wikipedia editor had made it up and inserted it into Wikipedia's list of harvest festivals in 2012. Someone at NASA used the Wikipedia list for naming features on Ceres. (Ceres was the Roman goddess of agriculture.)
I wrote to the US geological survey to point this out. They changed the name of the mountain.
Full story on my blog: https://blog.plover.com/wikipedia/ysolo.html
The success of an LLM is quite subjective. We have metrics that try to quantitatively measure the performance of an LLM, but the "real" test are the users that the LLM does work for. Those users are ultimately human, even if there are layers and layers of LLMs collaborating under a human interface.
I think what ultimately matters is that the output is considered high quality by the end user. I don't think that it actually matters if an input is AI generated or human generated when training a model, as long as the LLM continues producing high quality results. I think implicit in your argument is that the _quality_ of the _training set_ is going to deteriorate due to LLM generated content. But:
1) I don't know how much quality of the input actually impacts the outcome. Almost certainly an entire corpus of noise isn't going to generate signal when passed through an LLM, but what an acceptable signal/noise ratio is seems to be an unanswered question.
2) AI generated content doesn't necessarily mean it is low quality content. In fact, if we find a high quality training set yields substantially better AI, I'd rather have a training set of 100% AI generated content that is human reviewed to be high quality vs. one that is 100% human generated content but unfiltered for quality.
I don't necessarily think this feedback loop, of LLM outputs feeding LLM inputs, is necessarily the problem people say it is. But might be wrong!
With small models, at least, you can watch LLM output degrade in real time as more text is generated, because the ratio of prompt to output in the context gets smaller with each new token. So the LLM is trying to imitate itself, more than it is trying to imitate the prompt. Bigger models can't fix this problem, they can just slow down the rate of degradation.
It's bad enough when the model is stuck trying to imitate its output in the current context, but it'll be much worse if it's actually fed back in as training data. In that scenario, the bad data poisons all future output from the model, not just the current context.
Imposter sophistication levels might be the way we rank these in the future.
> The expression "Dunning–Kruger effect" was created on Wikipedia in May 2006, in this edit.[1] The article had been created in July 2005 as Dunning-Kruger Syndrome. Neither of these terms appeared at that time in scientific literature; the "syndrome" name was created to summarise the findings of one 1999 paper by David Dunning and Justin Kruger. The change to "effect" was not prompted by any sources, but by a concern that "syndrome" would falsely imply a medical condition. By the time the article name was criticised as original research in 2008, Google Scholar was showing a number of academic sources describing the Dunning–Kruger effect using explanations similar to the Wikipedia article.[2]
[1] https://en.wikipedia.org/w/index.php?diff=55273744&diffmode=...
[2] https://en.wikipedia.org/wiki/Wikipedia:List_of_citogenesis_...
Probably not no basis, Dunning and Krueger really did so research & found [retracted] a negative correlation between self-rated ability and performance on an aptitude test afaik [/retracted]. But it's often overgeneralized or taken to be some kind of law rather than an observation.
No, they didn't.
They found a positive linear relation with between actual and self-assessed relative performance, with the intersection point at around the 70th percentile. (That is, people on average report themselves closer to the 70th percentile than they are, those below erring higher and those above erring lower.)
The (self-rated rank) - (actual rank) difference goes up as actual rank goes down, but that's not self-rated ability going up with reduced ability.
Anyway, much as I do it, it annoys me too.
Related peeve, though as far as I know this is still restricted to gamers... How do you feel about "akimbo" meaning "wielding two guns, one in each hand", I believe that's from CounterStrike.
Or perhaps the word "glaive", to mean a thrown multi-bladed spinning weapon? I believe from Warcraft.
No no, Krull (1983):
Colwyn is found and nursed by Ynyr, the Old One. Ynyr tells Colwyn that the Beast can be defeated with the Glaive, an ancient, magical, five-pointed throwing star.
https://en.wikipedia.org/wiki/Krull_(film)
Incidentally, Krull is like many other fantasy films of that era (there was a bit of a trend at the time, it appears) that are very much like (some) D&D scenarios: the plot is essentially a string of little vignettes in each of which the good guys confront some terrible enemy and defeat it, culminating to a big boss fight at the end. Frex, Conan the Barbarian (1982) is very much like that, as is The Beastmaster (1982).
(sed /indian/native american/g)
But via citogenesis, the coati really became also known as the Brazilian aardvark. So the original claim is true, and this wasn't really citogenesis after all. More like self fulfilling prophecy.
That doesn't really fit here, but it's a similar idea.
"It is well known that Zimbabwe experienced severe hyperinflation in 2008..."
Very interesting article as a whole, of course.
That's what everyone else is saying already. Not sure what exactly you are arguing against.
I suppose that was inevitable.