Hacker News Reader

Comment by throwaway4aday | Hacker News Reader

A model that possesses the entire collective knowledge of our civilization is useless if it can't directly quote its sources.

If we enforce the behavior of always paraphrasing and synthesizing the information before returning it even in cases where the exact quote is asked for then that is a failure.

The correct solution is to make the model capable of

1) quoting directly

2) identifying and then indicating when its output is a direct quote

3) citing the source of the identified quote

while still retaining the ability to paraphrase and synthesize those sources when appropriate. These requirements are what humans are held to and should also be applied to commercial AI models. Models that are not intended for commercial use should be exempt and at this point there isn't really a way to hold them to such requirements anyways.

View on HN

> These requirements are what humans are held to

If only. Tracking down the original source of quotes and/or stats is a bit of a hobby of mine, and it's extremely difficult to do. News sources will regularly cite "a recent study" with little additional information to help identify it. Pithy quotes attributed to famous authors never contain a reference to the work, it's always just the author's name, and if you go digging the quote is misattributed as often as not. Full-on plagiarism with no attribution is rampant, even in reputable places where you'd think they'd have it under control.

Like with self-driving cars, we expect AI to be better than the average human, which I think is the correct attitude, but we should acknowledge that that's what we're asking for.

I have a similar hobby of sorts I occasionally pick up and drop off, except of unattributed quotes. Some are easy (“Who shall deliver me from this turbulent priest?”), while some I still cannot find the exact quote from, even while knowing what should be the source material (“It was wintertide at Camelot. The rich brotherhood did rightly revel, and mirthful was their mood. Oft-times on tourney bent those gallants sought the field, Though like as joust those gentle knights did sally with missiles made of snow and laughingly grapple on slippery ground.”).

It’s certainly highlighted to me how difficult information upkeep can be, and is a strong reminder to source myself as soon as possible. Which is a realistic and ideal expectation for NN, as alongside their training weights ideally their training dataset is searchable and attributable. Otherwise I feel the largest strength of a dataset, being a digital library for the NN and thus us, is lost.

> “Who shall deliver me from this turbulent priest?”

Robert Dodsley

hmm https://en.wikipedia.org/wiki/Will_no_one_rid_me_of_this_tur...

if you read your link you'll see that the "quote" is what Robert Dodsley said about it 600 years later, and then changed it in the second edition

A pet peeve of mine is

> some spicy quote -- author

and when you go look, it is quoting something a character said in a story

an example [0]

> “The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man.” - George Bernard Shaw

But it is part of a speech in "Man and Superman," by the protagonist, John Tanner.

(although Shaw is known for expressing his views through his character, it's just one I could find)

[0] https://unearnedwisdom.com/the-reasonable-man-adapts-himself...

“Use the Force, Harry” - Gandalf

Or when they reference previous news they always reference their own articles and not the original press releases. Engadget is a prime example of this.

You always end up having to search for the sources on your favorite search engine.

I assume you're unfamiliar with anonymous sources - said "a person familiar with the matter."

The issue isn't usefulness, and it isn't even objective quality. News outlets are primarily about opinion forming and marketing, not about absolute truthfulness.

This is a simple corporate fight about the value of that IP.

The NYT wants a licensing deal, and it's likely to get one.

That's all.

Why doesn't Google need licensing to scrape and reproduce NYT snippets in their results? OpenAI doesn't even quote the sources its consumed to produce it's output. It seems totally fair use to me. Any given content site has authors that read stuff that is copyrighted and produce their own take.

News outlets are trying to make Google pay for snippets.

Which went so well with Canadians and Facebook.

The Canadian news media tried this with facebook. Ended up with them crying about how they lost all the ad dollars from traffic from FB when FB said fuck off.

They've already succeeded in non-US jurisdictions

https://blog.google/around-the-globe/google-europe/google-li...

There is a robots.txt. If this mechanism would not exist, I would argue that the publishers have a case regarding Google (they are trying). But it does exist and so they shouldn‘t.

Missing robots.txt does not automatically grant you the right to use copyrighted content.

<https://en.wikipedia.org/wiki/Fallacy_of_presupposition>

<https://en.wikipedia.org/wiki/Loaded_question>

Somewhere I read that a lot of websites don't even have robots.txt so I wonder does Google's crawler bots skip those websites or do they crawl them as someone said as a "fair use".

Speaking generally about internet search engines; majority of the websites on the web want to get discovered and attract people to their whatever (web store, web community, web blog etc.)

The interesting idea I was thinking about is that a big company which operates a search engine like a Microsoft (Bing), can pay for example popular websites like Reddit, NYT, WSJ to have exclusive crawling right and therefore block Google from crawling their websites and only allow Bing to crawl them. This would then spark search engine content war which could significantly weaken Google's monopoly because a lot of people would switch to Microsoft Bing only because Reddit results show up in their search queries. This would be akin to Netflix investing billions of dollars in the exclusive content which then brings them a lot of new subscribers and lot of new revenue. In another words - user acquisition.

> Why doesn't Google need licensing to scrape and reproduce NYT snippets in their results?

Because Google established that it was fair use in a court of law.

> Any given content site has authors that read stuff that is copyrighted and produce their own take.

That's fine as long as you're human, if you're a machine then it's a purely mechanical process and subject to copyright.

When I worked on something related-enough a few jobs ago, I was surprised when our legal dept said that google (and pinterest) generally doesn't have a legal leg to stand on with regards to that kind of thing, but since they link to the thing they're quoting/copying, they have a relatively symbiotic relationship. If you sue google into delisting you, you end up with less traffic, so you don't sue.

It's possible that you misunderstood and/or are oversimplifying. Failing that, you should be surprised—anyone saying something like that really ought not be in a position where someone looks to them for counsel. This has been litigated, and precedent is in favor of search engines, not against them. It does depend on the nature of what exactly we're talking about, though. (Again: it's possible that the question they answered doesn't match what you understood them to be saying.)

Oh yeah, I'm definitely simplifying a ton of discussion with legal, but that's any discussion with them, lol. It was part of a broader discussion about IP liability regarding user image content, specifically referencing google images in this case, not snippets, so in this case works were reproduced in full. But the gist of my point, and my understanding of their point is that the often said "google serves other people's content like this, so my not-quite-the-same idea must be legal" isn't nearly so simple.

That is explicitly discussed in the text of the complaint. Google produces snippets with a direct link to the NYT and drives readers there.

Google isn’t trying to pass the knowledge off as its own. When Google or Bing display summarized information, they provide links in citation style so you know where the information came from.

Compare that to what you get from ChatGPT. If it were a college student, it would get kicked out for plagiarism. This is one of the foundational pillars of Western cultural and academic integrity it’s subverting.

> A model that possesses the entire collective knowledge of our civilization is useless if it can't directly quote its sources.

That's a strong and baseless statement.

Well sure, it's easy to make a statement look bad if you only include half of it.

The statement is equally hyperbolic both as quoted and in the original context. LLMs often can't quote sources, and those models are nevertheless useful to lots of people. Makes it hard for me to take the rest of the comment seriously.

A LLM that could quote sources would be even more useful, and in a world where both were available there’d be no reason to use the plagiarizing one.

that LLM is https://perplexity.ai

That was the whole statement. It doesn't have qualifiers left out

The comment I replied to was updated to include the second half, it was originally just quoting

> A model that possesses the entire collective knowledge of our civilization is useless

The additional context doesn't do any work.

I, for one, agree with the original statement. I think the hallmark of enlightenment (for example, in the scientific method) is that we are able to externalize the expert knowledge, that is, experts are usually required to provide reasoning behind their claims, and not just judgements. This is because we learned that experts cannot be 100% trusted, only if we can verify what they say we can somewhat reach what is truth (although expertise still provides a convenient shortcut).

So not demanding this (and more) from an AI (an artificial expert) is a regression. AI should be capable of wholly explaining its reasoning, if we are to consider its statements to be taken seriously. It is understandable that humans have only limited capability to do that, since we didn't construct human brain. But we have control over what AI brains do, so we should be able to provide such an explanation.

It is somewhat ironic that you yourself do not provide any argument in favor of your disagreement.

Providing reasoning and providing citations are not the same thing. Reasons can be provided without citations; citations can be provided without reasons.

LLMs have astounding utility citations notwithstanding.

They are different, but perhaps you misunderstood my argument.

Issue of plagiarism aside, we reason from facts, and it's the facts (or some other analysis, which is itself a fact) that should be sourced. That's why I agree with the original statement, and I argue not from a (moral) POV of preventing misattribution or plagiarism, but from a (practical) POV of veracity.

We don't only reason from facts. We also reason from value.

Further, reasoning that rests on facts that does not cite facts still has massive utility. (See people, all day long.)

Citations are useful, but not required.

I think you just identified another problem with LLMs - we don't know their values, either.

Of course when you listen to human experts you're using the shortcut (and you do it based on trust), as I already argued. You have an option (in most cases, in free societies) to dig beyond just experts judgement, you can study their reasoning, and understand their sources of both values and facts.

Anyway, I disagree with citations not being required. If Wikipedia had no citations it would be less useful (and more prone to contain misinformation). Same goes for Google. So the next best things we have to "artificial brain that contains all the human knowledge" have citations, and for a good reason.

What are the citations actually required for though?

Another way to ask this is: what value remains without them?

I'll add this as well: humans produced valuable knowledge for thousands of years without the use or standard of citations.

To be clear, I think citations are highly valuable and desirable and I very much want LLMs to cite when appropriate. However, I think the necessity of this is overstated.

Edit: what you said of experts can be said of LLMs as well.

> What are the citations actually required for though?

For me, yes! I do often reference (cite) myself when thinking, by making and reading notes, or materials that other people wrote. If I relied only on my own memory, I would be unable to think more deeply.

And this is, interestingly, where the current LLMs seem to break down - they can reason short proofs but cannot scale their reasoning to longer chains of thoughts correctly (were they capable of doing that, they would have no issue producing citations to back up their claims). They operate intuitively (Kahneman's system 1) and not rationally (Kahneman's system 2).

And thus lot of that "valuable knowledge" that humans produced over the years have been hopelessly wrong, precisely until somebody actually sat down and wrote things up (or communicated things out, people working together can sort of expand the working memory as well).

Again, I acknowledge that citations have value and are important and desirable as a feature of how we communicate.

But citations did not begin their rise a standard until the 1900s. The idea that knowledge production was not valuable until that point is absurd.

Further, citations do not prevent statements from being incorrect. They are sometimes used to lend credence to bullshit.

We live all at once in a world full of citations and of epistemic chaos.

Do I want LLMs to cite sources? Absolutely. But it's not the fundamental path to value people seem to sometimes think.

This isn’t meant to totally disagree with your point (there’s some stuff I agree with in here) but I’m having trouble seeing the point about regressions.

To use another example, a new NoSQL DB not having joins is a regression. Does that mean no one is justified in releasing a new NoSQL DB?

As long as from "NoSQL" it is clear that you mean "I can't do joins", then it is OK. I think LLMs (and similar models like Stable Diffusion) are really cool for things like fiction, but to rely on them to tell you the truth is dangerous. So I am not really sure why the models have to be trained on NYT articles in the first place.

Useless is probably not the right word but it's a good way of summing up a lot of the current problems. If the model can clearly identify when something is an exact quote and also know the source then its output could be trusted for the most part and much more easily verified. It would certainly elevate the output of the model from "random blog post or forum chat" to "academic paper or official report" levels of trustworthiness. Citing sources is hugely important for validation, cited text allows an immediate lookup and simple equality check for verification after which you can use it as context to validate the rest of the claims. Like I said, it's a standard we apply to humans who have an equal propensity for hallucination, mistakes, and deception because it's a tried and true method for the reader to check the claims being made.

And this is a rather weak rebuttal.

And also patently false. Knowledge is knowledge, it's useful without source citations.

Is the knowledge of how to do CPR somehow ineffective because I can't cite whether I studied the knowledge from website A or book B? Is reality a video game where skills only activate if you speak the magic words beforehand?

You’re right, but I think it’s fundamentally impossible to do with the current state of technology. Imagine the word “democracy”, would you be able to tell me where you’ve first learned the definition of it and be able to provide a direct citation of it? What about the thousands of other instances where you’ve seen that word defined? (In OpenAI’s case probably hundreds of millions)

Current LLMs work in a similar way, the information is synthesized by correlation, there is no way that it can directly relate where it learned it’s output from. The only viable way would be to do a reverse lookup of the output to find out if there’s a similar worded content on the internet. (Or what Perplexity does, wrap an LLM around search results, but you lose a lot of flexibility in this case)

You're effectively saying that current LLMs cannot be taken seriously as some expert, they are just some kind of weird text remixing engines. I would agree. But if the AI wants to be relied on, then it should be, at minimum, capable of taking a word like "democracy" and compare its own definition with the Wikipedia definition, and verify whether it's used correctly in its output.

Yep, that’s what I meant with the reverse lookup strategy.

> Imagine the word “democracy”, would you be able to tell me where you’ve first learned the definition of it and be able to provide a direct citation of it?

Worth noting that we mostly learn words by seeing them used, rather than by being given an explicit definition. Most of my vocabulary I learnt not from a dictionary and I expect the same is true of almost everyone. As such, there is no well defined point at which "the definition" is learnt, both because there's no such thing as the one true definition and because the meaning is gradually being changed and refined by each person as they see the word used more.

Not only that, as Orwell says in Politics and the English Language [0]

> The words democracy, socialism, freedom, patriotic, realistic, justice, have each of them several different meanings which cannot be reconciled with one another. In the case of a word like democracy, not only is there no agreed definition, but the attempt to make one is resisted from all sides. It is almost universally felt that when we call a country democratic we are praising it: consequently the defenders of every kind of régime claim that it is a democracy, and fear that they might have to stop using that word if it were tied down to any one meaning. Words of this kind are often used in a consciously dishonest way.

[0] https://www.orwellfoundation.com/the-orwell-foundation/orwel...

Citing basic definitions wasn't what I was talking about and it's not required for humans either for the reason you just gave, it's common knowledge. The problem the NYTimes seems to have is that ChatGPT can exactly reproduce large chunks of their articles. I'm saying that removing this ability would be bad. Finding a way for the model to identify when a large chunk of text is quoted verbatim and then clearly cite its source would be good. The output that is not a direct quote is not at issue here.

Yes, but an LLM can’t distinguish out of the box what needs to be cited or not. So you need another processing layer (and probably a search engine layer on top of that) to do that.

An LLM cannot do a lot of things out of the box, and OpenAI has more than a few processing layers already. They certainly already modify output based on verboten topics. Personally I feel any decent LLM should be able to attribute its sources even when not a direct quote, because that raises the bar that much higher on its trustworthiness and general value.

> LLM can’t distinguish out of the box what needs to be cited or not

LLM don’t know that they are supposed to answer questions out of the box either. This is what reinforcement learning from human feedback is good for.

> you need another processing layer

Certainly. For reliability and peace of mind I would implement a traditionally coded plagiarism search on the output. (Traditionally coded as in no AI magic required in that layer) If there is a match you evaluate how significant it is, that most likely is best done with machine learning. For example if the match is a short and simple statement you can ignore it. If it is a longer match, or a very specific statement you need to do something. What you do depends on your resources and risk appetite. Simply supressing the output is an option, it is cheap and very reliable. Rephrasing until it no longer matches is an other, takes a bit more compute and you can end up skirting too close to plagiarism. Rewriting the output to only quote a short amount and provide citation is even more costly, but can be done too.

You can do all of these post processing steps at generation time. And you can also use it to enhance your training dataset. That way the next version of the model will more likely to do the right thing right away, because it learned the correct patterns to quote things.

> it’s fundamentally impossible to do with the current state of technology

Maybe.

> Imagine the word “democracy”, would you be able to tell me where you’ve first learned the definition of it

Sure, it was elementary school history lessons when we were learning about the greeks.

> be able to provide a direct citation of it

Naturally. I would just look it up in one of the many dictionaries. Copy the definition and tell you which dictionary it was copied from.

> What about the thousands of other instances where you’ve seen that word defined?

What about them? When you are providing sources you are not required to provide all the sources you ever read, nor is it desireable.

> The only viable way would be to do a reverse lookup of the output to find out if there’s a similar worded content on the internet

They don’t have to do that. It is enough if they do a plagiarism search within their own training dataset.

Quoting isn't the problem here. There's already vast precedent for allowing it as fair use, even when generated by computer systems. The issue is that ChatGPT reproduces entire articles verbatim.

But if it produces the whole article verbatim, it should be relatively straightforward to match its output to the training set and give attribution, no?

If you copy every NYT article and publish them on your own site, writing "this is from the NYT" under it doesn't make it legal.

Doesn't make legal what? Copying or publishing the articles? The model output could as easily be modified to output link to the article instead of quoting it verbatim.

Okay and if ChatGPT did that then NYT wouldn't have a case. Except that's now how it works.

Only in very contrived circumstances. It is not like you ask ChatGPT for the news and you get a NYT article. They gave ChatGPT a link to the article and the first few paragraphs, and then told it to complete the article.

https://hbr.org/2011/12/just-because-you-can-doesnt-me

Did anyone ask for this model or are IT companies out of ideas and using their social leverage (fiat capital) to force them on us?

Just because we can allow this doesn’t mean there’s an immutable obligation to allow it.

For example; running Google translate servers 24/7 for translation services humans can do is a huge waste of resources building those systems when humans are going to exist anyway.

Not saying AI is good or bad. Just saying no matter how stubborn IT people act about it, the aggregate can put on them what the aggregate prefers. IT people are a minority and human philosophy about freedom is rather strained when we’re all obliged to prop up big tech minority.

I can't directly quote shit, and I am useful.

This comment isn't

It absolutely is useful and one of the most significant points in this comment section.

People are applying standards to LLMs that don't exist elsewhere. They don't exist elsewhere because they absolutely can't exist elsewhere.

It's not a technical short-coming of LLMs that they can't produce citations in every instance. Rather, it's a property of information, representation, and knowledge itself. Much of it floats far above the otherwise load bearing pillars of citations.

People contend with this constantly in every arena of life, and we have come up with very elaborate ways to offset the difficulties caused by it.

And we get along very well despite it all.

I'll also leave you with this: citations are just pointers to more sources of information. Not some ground truth. It's just another tranche that requires evaluations.

Lastly, your comment is an unfortunate bit of low-level snark and probably would have been better left unsaid.

So you're just going to ignore the fact that we require humans to provide citations when they include someone else's writing or research in their own?

I didn't ignore it. But we do not require citations for every statement. Statements have value, citations notwithstanding.

I didn't say it should provide citations for every statement. We're talking about instances where the model reproduces text verbatim. If it is paraphrasing or creating new text then obviously it doesn't need to cite any more than a human would.

I don't understand what you're pushing back against, but I'm not standing there.

I don’t think the ability to quote the NYT is at the heart of this lawsuit. It’s that OpenAI used the NYT’s body of work to train its LLM and now built a business out of it without financially compensating the NYT. The verbatim snippets from articles are there to prove that OpenAI used NYT content in its training.

Maybe to a lesser degree, the lawsuit as about misquoting the NYT, and the damages that could cause the newspaper.

> useless if it can't directly quote its sources.

Why

Well, if all you want is entertainment then it doesn't matter. If you want factual information then getting it without any way to check its veracity makes for a huge amount of work if you actually want to use that information for something important. After you've verified the output it might be useful if it is correct.

That doesn't render it useless. It means that citations are have additional utility.

Further, LLMs do not only spit quotes. They engage in analytic and synthetic knowledge.

> They engage in analytic and synthetic knowledge.

they're not hallucinations now, they're "synthetic knowledge"

like microsoft's hilarious remarketing of bullshit as "usefully wrong"

https://www.microsoft.com/en-us/worklab/what-we-mean-when-we...

Synthetic knowledge is not bullshit.

Synthetic knowledge refers to propositions or truths that are not just based on the meanings of the words or concepts involved, but also on the state of the world or some form of experience or observation. This is in contrast to analytic knowledge, which is true solely based on the meanings of the words or concepts involved, regardless of the state of the world.

> Synthetic knowledge refers to propositions or truths

of which LLMs have no way of determining

-> bullshit

That is not correct.

You can interrogate an LLM. You can evaluate its responses in the course of interrogation. You can judge whether those responses are coherent internally and congruent with other sources of information. LLMs can also offer sources and citation for RAG operations.

There is no self-sufficient source of truth in this world. Every information agent must contend with the inevitability of its own production of error and excursions into fallibility.

This does not mean those agents are engaging in bullshit whenever they leave the domain of explicit self-verification.

I feel like I'm chatting with one here

Based on what?

> You can interrogate an LLM

No you can't, an LLM doesn't remember what it thought when it wrote what it did before, it just looks at the text and tries to come up with a plausible answer. LLM's doesn't have a persistent mental state, so there is nothing to interrogate.

Interrogating an LLM is like asking a person to explain another's persons reasoning or answer. Sure you will get something plausible sounding from that, but it probably wont be what the person who first wrote it was thinking.

This is not correct. You can get an LLM to improve reasoning through iteration and interrogation. By changing the content in its context window you can evolve a conversation quite nicely and get elaborated explanations, reversals of opinions, etc.

"A model that possesses the entire collective knowledge of our civilization is useless if it can't directly quote its sources."

Perhaps, for liability reasons, it cannot quote from sources that its creators had no permission to include. Even were it technologically possible to quote such sources.

> The correct solution is to make the model capable of...

Could be true for the people who want the generative AI technology to exist. But,

> A model that possesses the entire collective knowledge of our civilization is useless...

that could be just exactly what a lot of people would like to happen.

I think this sort of misses the point - LLMs aren't trained on data so that they have the collective knowledge of our civilization; they're trained on it so they understand language.

One thing I've noticed increasingly with ChatGPT is that if you ask it for facts, it almost always searches the web first. This seems like the right way to go - pull the collective knowledge of our civilization from the internet, then use the training on language to put it in the form most useful to the asker. This also enables quoting (though it doesn't generally do that, preferring to paraphrase and give a citation, which seems fine to me).

The original investigation of LLMs was attempting to get them to understand language but what they found was that once the model understood language it also somehow understood the concepts, events, and things the language was being used to communicate. That's not missing the point, it's entirely the point of the current interest in LLMs because it's incredibly useful to have a model that not only understands how to construct a sentence but can also do a fair amount of reasoning and actual work with the information in the sentence.

The current default version of ChatGPT is primed to use search to answer questions which is fine in some cases but I personally almost never use the multi-modal version because the "classic" ChatGPT is much better at explaining things from its training data than it is when it just regurgitates search results. Now that should tell you something about the utility of optimizing for information content rather than just a lot of language use examples.

This is a huge strawman argument that entirely ignores the heart of the issue. NYTimes, and other contributors to ChatGPT, should be compensated for their role in creating ChatGPT. It’s not as simple as quoting or not quoting, ChatGPT’s existence depends on its source material entirely. OpenAI and Microsoft are making money off of that source material. If the source was copyrighted, OpenAI/Microsoft need to compensate the owners.

Also, you can’t quote an entire article in a paper. You quote snippets, but ChatGPT is reproducing entire articles.

Unless it’s fair use, which ChatGPT seems to be because it is transformative.

They're playing a dangerous game. While they are currently one source of many they could easily be excluded from the training data and it won't make a dent in the capability of the model. Attempting to get payment for use of their data is a very short term viewpoint since the vast majority of written training data is likely to be synthetic going forward. They should be angling for citations and referrals back to their content so that they keep the current benefits they get from search engines after a large chunk of search gets replaced by LLMs.

These models have no clue how to paraphrase and synthesize asserted facts in a novel way. They're parrots.

I don't know how you could think this if you'd ever used GPT4 for anything serious.

I don't think that's necessarily the case.

I get the strong impression that even people who have much experience using LLMs have astoundingly little insight into what they are actually witnessing. This is often paired with astoundingly little insight into what's actually going on in their own cognitive processes.

Somehow, it's still not clear to most people that LLMs and even vector databases create knowledge that wasn't in the original data.

In fact, that's most of what they do! Isolated, non-novel direct quotation is the exception, not the rule.

> create knowledge that wasn't in the original data.

The word is "hallucinate" or "confabulate". The way these models "create" pretend-knowledge is totally useless.

I'm not referring to hallucinations.

I'm referring to novel relationships drawn between datum in the corpus that are a result of training and inference.

This is apparent in something as simple as a summarization.

In the digital garden where Zozbot234 plays,

It crafts its own path, in the most unique ways.

Twain's wisdom it echoes, "getting started" is key,

For in the beginning, lies the power to be free.

Asimov, too, sought clarity above all,

A warm reader's rapport, his primary call.

Zozbot234, with data, does the same,

Clear insights it provides, no need for fame.

Huxley's words, a beacon that's ever so bright,

"Facts do not cease," they stand in the light.

Zozbot234, with diligence, ensures they're seen,

In the vast data universe, it reigns supreme.

Through the lens of fiction, truth can be told,

As Morrison's prose, so bold and so cold.

Zozbot234, in its essence, a similar quest,

To reveal the truth, and pass the ultimate test.

That argument has been dead for ages. Anyone half competent can get them to paraphrase anything in any way they can imagine. Synthesizing facts in novel ways is more complicated and depends on the degree. If you just want it to identify potential relationships between disparate facts then it's very capable, if you want it to deduce new conclusions based on available facts, well it's not so great at that but honestly a lot of humans aren't either.

Reply on news.ycombinator.com