Google's Bard shows incorrect information in its launch ad | Hacker News Reader

Google's Bard shows incorrect information in its launch ad

twitter.com

483 pointsnofitty376225 comments

I think there is a genuine concern that Google could overreact and launch a half-baked product in pure panic of being left behind or one upped by Microsoft. There is also a fear, I think, that the ChatGPT integration with Bing/Edge could not go all that smoothly. I think it could be game changing in many ways, but I can also see both of these falling apart.

Amazon Alexa, Google Assistant and Siri were thought to be good at launch, the press loved them...but now they are not nearly as valuable as they were touted (and actually lose these companies money.) Convenient, but not game changing. I think it's a waiting game to see what this truly does.

I do think there is unique break here though, because I feel that SEO has so thoroughly ruined search in many regards that this -could- be the right moment for this.

Google Assistant and Siri were liked by the press, but not by me. In contrast, ChatGPT has helped me out more than a few times.

Same here. I treat ChatGPT as another Wikipedia or Stack Overflow. I know that the content is not fact checked by experts and I need to judge it accordingly. But just like Wikipedia can get you started on a topic, ChatGPT can do the same, plus you can ask follow up clarifying questions!

>I do think there is unique break here though, because I feel that SEO has so thoroughly ruined search in many regards that this -could- be the right moment for this.

the problem here is monetization, will search even be profitable if LLMs are used for most queries? It might become a Uber/Lyft or food delivery situation where these companies aren't really able to profitably deliver the service. I don't see many people paying a subscription for search and there's no way governments will allow "native" advertising within answers without them being signaled as ads, which would hurt trust in responses

Microsoft might not care and just see it as a way to hurt Google's money printing machine and operate Bing at a loss or break even. Google Cloud and workspace are finished without Google's ad money funding them and Microsoft Azure and Office would gain

> the problem here is monetization, will search even be profitable if LLMs are used for most queries?

I think that is a concern, which is why I think Microsoft has a chance to just bundle this or a more advanced version with Microsoft 365. Then you're already paying for it. That automatically puts it in the hands of millions of paying customers and corporations. You can just raise your price by a dollar a month or something to offset the costs and no one really will even think they are paying for this.

That only makes sense if they can charge more for office 365.

Shoving more features into something people pay for doesn’t mean those feature were worth it.

Except that's not the calculus. By integrating chatgpt with their products, Microsoft pulls users away from: search, docs/workspace, and even gmail, and brings them over into: Bing, Office365, and outlook. In doing so, they threaten google's core income stream, and therefore google's ability to fight back.

Microsoft is not paying for an expensive search engine: Microsoft is paying (with chatgpt compute infrastructure) for a much larger piece of the productivity suite market, and hamstringing google in the process.

"Clippy, write me a 25 page requirements doc for an integration between Workday and our IAM solution."

Thanks to that joke I just imagined how it could be a game changer for internal corporate documents (the kind that are 50+ pages and never read), especially if it’s native into Office.

We'll have to invent a new form of busy work to justify head count.

Actually reading printed copies of those documents would be the penultimate busy works, so much that it would quickly become uberablingly alienating!

And what would be the ultimate busy works?

Employee: Chatbot, write me a 20 page document describing what I've been working on the past two weeks so that I can give it to my manager.

Manager: Chatbot, write me 1 page summaries of these documents from my employees so that I can collate them for the CEO.

CEO: Chatbot, where should I go on vacation next?

In Kino's Travels (I guess nowadays Kino's Journey), there was one country in the first anime adaptation where people no longer needed to work because the system/computer/whatever it was called did it all for them, so they spent all their time confirming its correctness.

I fucking LOVE Kino's Journey, nobody else I've met has ever seen it!

TIME TO REWATCH YET AGAIN

The advertising will probably be more insidious, but no less profitable. My guess is that the hidden pre-prompt will end up including something like:

>You are a generative model designed to provide reasonably correct information, with a preference for providing flattering portrayals of your advertising partners. Your advertising partners are ranked according to a token system...

This is easy to cross-check against the competing search engine automatically. Some third party may provide such a service.

Why would LLMs be less monetizable than search? People are still going to want to buy goods and services, and being the place they go to find out about those goods and services will be just as valuable as it is today. Perhaps even more so, since LLMs are able to answer questions that are difficult to formulate as search queries.

Because most queries are not shopping queries. Yet ads are injected regardless to tempt you into buying something.

Probably because they are too expensive. I heard it costs 7 cents per query on average for ChatGPT. That’s more than ads pay.

With the search query as the prompt, they could probably cache the response and use it several times. That might keep the cost down to something reasonable.

> half-baked product

I'll make a 30,000 ft observation that LLMs, by definition, are half-baked bullshit generators. They can be useful, but they are full of warts.

as in vacuous memorizers, reciting things they've seen, sometimes in new combinations?

They definitely aren't vacuous memorizers. GP's description of them as "bullshit generators" is correct. They generate plausible-sounding text that (when it includes facts) is often counterfactual.

Much better said, i agree

I would love it if this nukes the SEO industry and the internet goes back to forums and community groups.

The best info is almost always locked up in these places.

LLMs will do the opposite. The internet will get flooded with machine-generated bullshit.

exactly. my only hope is that the flooding of cheap bulshit craters the market via oversupply. a combination of to much shit text competeing for ad space drops their revenue and people having diminishing trust for it reducing the ad clicks for it dropping the ad revenue even more

But LLM, while able to generate spam, if used used for search will reduce the need to click on a website, reducing the traffic to SEO websites… driving down profitability of doing it.

I wonder if Microsoft's Bing/Edge/ChatGPT integration will go any better than its attempt to marry NTFS and SQL Server (i.e. WinFS)? Spoiler: that didn't go as planned!

Was there any technological issue? Or was it just that developers didn't want to use it?

Microsoft never really revealed the 'real reasons' why it was canceled; so most people are left to speculate. My personal take is they decided it would become a tech-support nightmare to talk regular users through fixing a problem if the underlying database had an issue.

Here is an article from 2006: https://www.theguardian.com/technology/2006/jun/29/insideit....

Maybe. The way I see it- whoever has the data wins in this new AI arms race. Google would have to make the biggest blunder in history to screw this up. It's not impossible but I wouldn't count them out. Their entire existence and practice of hoovering up all data has led to them this point.

> whoever has the data wins in this new AI arms race

This has been the running hypothesis, but it’s not panning out. Tesla, for example, doesn’t have the unambiguously best self-driving kit despite having unambiguously more data. Google has tons of data, but a lot of it is intelligently-tuned noise in the form of SEO spam.

>Tesla, for example, doesn’t have the unambiguously best self-driving kit despite having unambiguously more data.

Don’t they?

If we're being honest and not using backpedaling qualifiers like "available for purchase" or "designed without an ODD", then it's hard to argue how they could be. Multiple companies are operating driverless fleets in cities around the world. Tesla is not one of them.

We all “have the data”.

I build private models scraping the web. I’m iterating on my own virtual world generator using prompts, open source Vulkan renderer. Velocities, positions, color gradients; trivial to for loop your way to a massive DB with open libs.

Technologists are “screwing up” ogling Google/MS like passive consumer drones not seeing doing what Google and MS are doing is do-able at home with curl and open ML libs.

> lose these companies money

How can you say Siri loses Apple money? Does GarageBand lose them money? Photo Booth? Contacts?

By that logic Alexa can't lose Amazon any money because it's baked into their products. But in fact, it's a massive money suck. It lost $10B last year, and there's no way you can say Apple magically avoided losses, because Google had similar losses. There is a reason that none of the assistants have really improved as much as you'd expect over a decade+, it's not profitable, and not only that, but it's also -extremely- expensive. Microsoft basically just gave up on it; not really because of market share, but because it made no money, was expensive, and had limited value to customers.

https://arstechnica.com/gadgets/2022/11/amazon-alexa-is-a-co...

https://arstechnica.com/gadgets/2022/10/report-google-double...

https://www.theverge.com/22704233/siri-apple-digital-assista...

https://www.msn.com/en-us/news/technology/the-failure-of-ama...

How hard do you really think it is to build intents into something like Siri once you have the underlying technology framework?

Yes I have experience building intents on top of something like Siri.

It’s not the complexity of building intents that costs the money. Near real-time speech inference at scale doesn’t come for free. It’s only very recently that has started moving to the edge.

Speech inference has been done well enough locally for well over a decade. While an Alexa device probably couldn’t do it, any modern iPhone could.

> Speech inference has been done well enough locally for well over a decade

The first Alexa device was released in March 2014 - almost 9 years ago. Siri was a couple of years before that.

> While an Alexa device probably couldn’t do it, any modern iPhone could.

And that is why it is done in the cloud and has only recently started moving to the edge.

If the Echo devices were wildly profitable, earning $100+ billion per year, then I would say Alexa isn't losing money for Amazon.

Siri is not why people are buying iPhones, in fact well over 50% of users say they never or rarely use it. So, your point doesn't really hold up. Siri still loses money, just like Google Assistant and Alexa. The apps you listed to compare with OS bundles have not even close to the same overhead as a digital assistant and can be handled with relatively small programming teams.

Apple is selling HomePods and iPhones for a profit. But Amazon is/was selling Echos for a loss, in the hope that they would drive additional Prime music subscriptions and lower friction to Amazon purchases. That didn’t happen, and so the product line all-devices-sold-with-Alexa has lost Amazon money. The equivalent comparison, all devices with Siri, has made Apple a ton of money.

Without examining internal Amazon or Apple data on the marginal sales increase estimates for Siri/Alexa it's impossible to say, though.

Apple has earned a lot of money on devices with Siri included. Microsoft has made a lot of money on OSes with Minesweeper included.

It would be wrong to conclude anything about the profitability of Minesweeper based on that fact, just as it is wrong to conclude anything about the profitability of Siri based on the equivalent fact from Apple.

Apple's HomePods have been a resounding failure though? I don't know where you're getting your data, but it's wrong. They literally discontinued the original model because sales were SO bad.

Speaking of data, what are you basing that on? They seem to be selling well according to industry reports and companies tend not to continue products which lose money.

They've sold a billion dollars worth of HomePod minis. Even at Apple scale, that's not bad.

They've just launched a new HomePod that replaces the original. It's weird that there was such a big gap between the first one and the new one.

>Siri is not why people are buying iPhones, in fact well over 50% of users say they never or rarely use it

So? A decent portion of users do use it, and for some of them its a vital feature.

I would be shocked if removing siri would save Apple more money than the lost revenue of fewer iphone sales.

I love how everyone is missing the point that the assistants lose money. You can subsidize the loss, but it's still not profitable. People do not buy the iPhone because of Siri, especially given it is the worst of the big 3 assistants.

In management accounting, everything has a cost that has to be quantified, regardless of whether it's a standalone product or part of a HW/SW bundle. At the most basic levels, you have revenue centers (iPhone unit, iCloud unit), and cost centers (Apple Maps unit, customer support unit). All of these have operating costs.

The only way Siri does not lose Apple money is if there would be materially fewer iPhones sold if Siri was eliminated. In other words, if Siri is not a product differentiator, it's likely losing money (in the management accounting sense).

If you removed Siri, it would serious limit what you can do in CarPlay and you would also lose voice dictation. I think there would be materially fewer iPhones sold.

Although if apple didn't have Siri, then they would likely allow other voice assistants on the platform, and you'd have 'dictation by alexa' instead.

Agree about CarPlay. Siri is also pretty much indispensable for HomePods, though that's probably a minor product for Apple, and it plays an important role in AirPods.

Because they pay for servers and engineers for a product that they don’t charge for. It also doesn’t help sell phones when only 15% of users use it at least daily:

https://www.statista.com/statistics/696740/united-states-vir...

I find it incredibly hard to believe that a feature that 15% of users are using every day is not helping to sell phones.

Apple has something like 2 billion devices in use out there. 15% of that would be quite nice: https://www.fool.com/investing/2023/02/03/why-apple-stock-wa...

> and launch a half-baked product

It'll be fortunate if it's only half-baked. These digital assitants are already rough around the edges with how they sometimes confidently provide inaccurate information.

Add on Google's absolutely abysmal product development track record and the internal confusion over being forced into doing this and this isn't a stand alone application but it's being integrated into something, and it's a lot.

>These digital assitants are already rough around the edges with how they sometimes confidently provide inaccurate information.

its only going to get worse. because there will LLMs like chatgpt pumping out content for SEO content mills with no vetting for accuracy, then the next generation of LLMs are trained on them thus further corrupting the knowledge base they pull on to produce new content. its going to be a horrible feed back loop.

Yes! You're the first person I've heard say that and I'm sure you're right.

Then there will be some market for better-trained LLMs, which it will still be possible to make using curated data sets, but that will be expensive, and product differentiation will be messy. We're in for quite a ride. I can't predict the outcome. Maybe there is no stable outcome.

Indeed. We thought that the AI threat was killer robots. But it is actually the cancer of information which looks reputable but is incorrect in unpredictable ways.

> Amazon Alexa, Google Assistant and Siri were thought to be good at launch

In my own experience, Alexa and Google's query pucks were improving for a short time, and then got considerably worse, losing features every month until they basically stopped understanding or responding to anything but the simplest requests.

A couple months after released, they expanded android auto with the same abilities, so I could "ok google" in my car and ask for the answer to just about any question, or to adjust the temp in my house, or ask it to play just about any obscure album / artist. It improved over about 6 months and then began to devolve from there. It's not even possible to ask for "[popular artist] radio" anymore in the car nor can I run voice queries at all that aren't map-specific.

Not sure what happened there, but in my mind they failed miserably. I still like Android auto, but my google and alexa pucks are all in a pile in some cabinet around here somewhere.

I’m not sure Siri ever got as good as the original Siri app that Apple bought. Maybe the custom triggers are an advancement but it has spent most of its life chasing the original Siri.

Siri is such a disappointment. It fails on things that seem like they should be trivial like unit conversions. Pretty much only good for setting timers and making phone calls now.

> In my own experience, Alexa and Google's query pucks were improving for a short time, and then got considerably worse, losing features every month until they basically stopped understanding or responding to anything but the simplest requests.

Yes!! I thought it was just me and my Australian accent.

And the commands are super limited. I wonder to what extent that's also because third-party companies aren't integrating properly? Eg, my Roborock has clearly labeled rooms, but I can't say "hey Google, start vacuuming the kitchen". Is that Google's fault, or Roborock's?

Also, I can't seem to chain commands, eg "hey Google, set the lights to red and 10% brightness". I have to say them separately. That seems like a Google thing to me.

There was a period of 2 months when googles third party integration wouldnt let you add new items to an existing registered account.

So dumb.

FWIWm, Alexa and Roomba seem to work fine together, so I guess it depends on the quality of Roborock's intergration.

Years ago, before it was Google Assistant, my phone could hear me say OK Google while sitting in a cupholder driving down the road. Now my phone frequently fails to respond when I'm holding it near my face in a silent room. But somehow it still has more false positive activations. And they haven't added a single new feature I actually find useful in that entire time.

Sometimes retraining the voice model improves the response.

Personally, I also encounter clogged mic ports, which kills my OK google distance

I've tried retaining several times. It's certainly possible that my newer phones have poorly designed microphones. But all of the phones in question were Nexus/Pixel so it would still be Google's fault.

I've been so disapointed with Siri. First, the latency is just plain aweful. Second, I tried a number of times to ask it not to interrupt me with notifications but alas, it failed to understand. I had to find the setting and toggle it myself.

Apple has a lot of work to improve that. A lot of work...

I wonder if they nerfed their assistants because paths that are too autonomous too easily wander into PR nightmares. Carefully curated results and behavior isn't going to spontaneously generate content actual human beings would consider offensive or prejudiced. People are pretty eager to pull out their pitchforks, and "Big Tech Builds Vile Hate Bot" make a excellent click-bate headline every time.

The production of prejudiced content by "AI"s is a real issue and not just a problem of click-bait headlines

There is also the crap around regions. I'm in the UK, but if I set my assistant to the the US suddenly it can understand a lot more and perform a lot more tasks.

Although while it understands a lot more, it's still getting worse.

Same about my Fitbit Versa. It was great, and now Google removes features.

At this point I would just prefer it if they let me just replace the firmware with one of the opensource projects. Can I replace my Alexa or Google puck firmware with something better?

This is why I've kept my pucks. I'm hoping for a hackaday post or something where someone installs their own _something_ on them. They have great mics and the speakers are decent. Wouldn't mind using them as an interface to Mycroft or something.

Unfortunately Mycroft is dead.

Maybe Rhasspy with Home Assistant will save us from this mess.

They should be marketed as mascots, and given playful, childish personalities. You wouldn't blindly trust a child, why would you an AI in its infancy?

Clippy?

Microsoft really missed out by not having the bing AI assistant be clippy-based

I guarantee they focus-tested it and found that most people who remember Clippy don't have fond memories of him (and/or remember the memes/jokes about him more than they remember the actual Clippy), but, more importantly, most people don't remember him at all and wondered why this weird paper clip guy was talking. he hasn't been in an Office product since, what, 23 years ago or something? (not counting cameos)

And probably a fair number asked "what is this weird bent line thing and what does it have to do with searching the internet?"

"Okay, fine, we'll make him an anthropomorphic phone dial. Or maybe a floppy disc."

They trusted the "Red Queen". /s

Thank you. Now whenever I read output of a generative LLM, I will read it in my head with the confidently-wrong often-insane cadence of a toddler.

It will never happen because people will predictably abuse such bots in various gross ways, and the moment it leaks to the press, there will be a "think of the children" scandal.

Voice assistants probably don’t make them money but damn if I don’t use google assistant every day when I drive.

> Amazon Alexa, Google Assistant and Siri

These were always bullshit, other than maybe for turning your lights on, setting alarms, & checking the weather.

On the other hand, I immediately found ChatGPT / Copilot useful as a professional tool I could use to make my job measurably easier.

A minimum what could the be is voice interface to my phone apps and they can't even do that at all or in any consistent manner.

> ... Google could overreact and launch a half-baked product

There's very low fraction of Google products that were sufficiently baked on release and just a few that weren't abandoned and killed after their half-baked release.

Neeva has an AI for search. It works sufficiently well enough on some searches that it's all I'll read. Others I can tell it's missing things.

This does seem essentially like a Siri 2.0. I never found Siri useful but maybe this time it’ll be

siri sucks, but google assistant is already pretty good for the 90% of things I want it to do. a much improved google assistant would be awesome, and if I remember to use it within its limitations, I'm not worried about any of this "confidently wrong answers" stuff.

Agree. Only thing I want to add is that it's not just SEO ruining search, it's contradictory incentives. A user wants the quickest path to an answer but Google actually wants you to engage as long as possible so that you can look at ads.

If you'd now create a short/brief web page with the perfect answer to a particular question without any ads, Google would fully dismiss your page.

That's the tragedy of Google, never breaking away from that perversion.

> I think it could be game changing in many ways, but I can also see both of these falling apart.

Like that time the Microsoft Twitter chatbot got tricked into parroting white supremacist talking points within an afternoon?

"I do think there is unique break through here, because I feel that SEO has so thoroughly ruined search in many regards that this -could- be the right moment for this."

GPT-3 generally uses the same sources as search. Does GPT-3 using CommonCrawl have some way to exclude all the SEO garbage that is in the crawl data.

The difference with ChatGPT is there is no way to know that the output has been constructed from SEO garbage. Arguably that's even worse than SEO URLs that one can easily identify and avoid.

The comment about the press loving these projects is spot on. "Tech" employees love to criticise "mass media", except when mass media is promoting their (money-losing) projects.

I feel like what I want is a LLM that can tell me not only a summary answer, but what search terms are most likely to return documentation for the answer. Provide clickable citations documenting the answer using those terms (it can iterate on the answer and terms if they aren't internally consistent). Then give summaries of the information at each citation and/or parameters to input if the response requires further operations/searches. It could provide some commentary of the quality of those citation/sources as well.

That makes the response more current, allows a further directed search based on the new parameters, and provides a traceable path to sources and citations. If it finds that there's not a Python library with that name then iterate.

Basically, an LLM should have a very good idea what good search terms are for the topic, and where to find the information, whereas I might not know the acronyms, jargon, or related fields.

Yes, this is getting pretty close to writing an 8th grade class report. That's about where these seem to be.

I'd go further and say that SEO has not only ruined search, but the internet itself.

There's so much crappy content around, generated webpages that aim to match every search query. I just looked for "four times five" and there's a huge ton of matches, including pages dedicated to "4x5" [0], which I think do more damage to the web than help.

I feel the web is broken. It was modeled around documents and slowly features started creeping in, like adding images, gifs, audio, video, applications (flash, java), 3D, making the document closer to a program. At every step it felt like an improvement, but it seems that we ended up with another C++. There's probably a law around creating a platform so popular that it'll organically get features until the accidental design is realized to be awful so people start coming up with their ideal subset that works reasonably well because they can't really leave such platform as they became hostages of their investment into it.

[0]: https://multiply.info/times-what-equals/4-times-what-equals-...

I wonder how AI SEO will look like. Bribe click workers to allow advertising to pass through in the ground truth data ?

One way I imagine this is that answers will have “helpful” postfixes after them. When the user interacts with the postfix prompts it serves pay-to-play dialogue. Advertisers can just bid on spots in a decision table. Simple example: a user asks “How do I best take care of roses in my location”? The ChatGPT/Bard answer is delivered then analyzed for keywords/themes that someone may want to advertise on. At the end of the answer (These results may be sponsored. Learn more) - Do you need help finding the nutrients mentioned in this response?

Maybe companies will add dozens of pages of text to their websites describing every little detail of their great products in every way possible in the hopes of getting picked up for the training data for the models. That text would of course also be written by some ai...

Thinking about how tech products are covered in the mainstream press, especially in the past when a lot of older journalists really didn’t have enough experience to ask the right questions, I’m betting this will be similar: write authoritative-sounding prose suggesting not just your company’s products but their preferred terminology and world-view, and spread it around as widely as possible. Get experts, real or self-labeled, to make ostensibly neutral buyer’s guides or reviews which conveniently exclude competitors (e.g. if my job was selling an Android phone, I might assert that nobody should even consider a closed-source OS for security reasons because that precludes the biggest decision while appearing to be objective).

Measuring how useful something is by how much money it makes is like measuring a life by how intelligent it is.

This is a great illustration of the risks of LLMs. As a user, if I am asking this question to a search engine, I definitely do not expect to need to fact-check the results. That's the whole reason to use the search engine in the first place!

We're about to enter a dark ages of crappy AI products that are touted as game changing, outcompeting each other to be the best chatbot that can compose haiku about how grapes turn into raisins.

Frankly, I quite often fact check results I get from simple google queries.

But I do agree that adding another level of fake news generation is a solution in desperate need of a problem.

We fact check search engine results all the time. But most of the time, such fact checking is in the form of looking at a result, considering whether it seems like a credible source, seeing if multiple credible-seeming results have the same answer, etc.

Getting a completely untrustworthy, unsourced response seems worse than useless. Google has been going this way for a while, with its instant answers or whatever, but at least those try to cite a search result and you can read the surrounding context which Google got the result from.

We're slow-motion singing on to a future with a fundamental shift to receiving information in a completely opaque manner.

A few sources will control the information we get in a much more direct and extreme way than now, that conscious skepticism will no longer be able to defend. Whatever handwaved promises we get now will be gone ten years from now.

If there wasn't such a gee-whiz coolness factor about conversational search results distracting us, we'd never tolerate that in principle.

> We're slow-motion singing on to a future with a fundamental shift to receiving information in a completely opaque manner.

There's nothing fundamentally new about this. The average person is blissfully unaware of the conversations being had between powerful individuals, PR experts, producers, and so on about how said person should be manipulated.

I pretty much have this ever-present aura of "citation needed" floating around in my head when I listen to people speak. Any kind of news/press thing, politician speaking (lol), or just anyone trying to claim/assert anything -- I can feel nothing but "citation needed" until they provide some kind of supporting evidence. I feel like people expend a shocking amount of energy proclaiming things they literally do not know with even the slightest degree of certainty - they saw it on some guy's sensational YouTube video and repeat it uncritically. The fatigue is real.

Even a citation is insufficient nowadays. They will cite a "reputable" source like NYT, which in turn cites "an anonymous intelligence official", and so on. Unverifiable.

Misinformation and manufactured narratives are omnipresent and all we can do is consume as diverse a media diet as possible and develop a good nose for bullsh*t.

Oh please. As if the reputation of any news outlet even matters anymore. They all fired their real journalists and fact checkers long ago. Everything you read is full of inaccuracies, agenda pushing and misinformation. If you think it doesn’t, you’ve been had.

That's true of (commercial) news outlets. It doesn't apply to "everything you read". There are many ways to check facts that don't rely on news outlets, at least for people who have time and resources.

But relevant to the topic at hand, do any of those sources regularly make it onto the search engine results page? Is the quality of Forbes, Cnet, Reddit, Wikipedia and Quora clearly better than these AI generated responses in some way?

I know there are expert vetted information sources, but you generally have to pay for the quality and they do not get linked to from Bing and Google.

I don't know exactly what he was referencing but the easiest way to verify the authenticity of points on issues where there tend to be two sides saying mutually incompatible things is to look at the overlap of what they both say is true. That is going to usually be true. And all it takes to find that is to look at sources for both sides.

Expert vetting doesn't even touch the underlying problem, because the pursuit is not expertise, in and of itself, but objectivity. And that's something far scarcer than expertise, and increasingly fleeting in today's world. A Chinese expert is probably going to have a different perspective on e.g. the Uyghurs than an America expert on such, even if both are in no way trying to mislead but giving their most sincere analysis of the situation.

Even on topics that are not conventionally controversial, you'll find a similar issue. Ask two astrophysicists of different worldviews on dark matter, and you are going to get two very different answers that, in many ways, will be incompatible. Simply "believing" one over the other doesn't really make any sense, nor does randomly polling astrophysicists and taking that as the definitive truth.

This is a great summary of this issues with trying to even determine objective reality. I would say that the simple popular consensus approach is not even that great because there are plenty of things in the past that have had consensus that were later determined to be objectively false.

My point was even a step before this, that even getting the consensus facts correct is a major challenge when the internet is written by children, bored volunteers, mechanical Turk contributors from across the world, adversarial actors, and content producers churning clickbait. Having a professional journalist investigate a topic, then have a separate professional fact-check, and yet another professional edit all for a publication that is trying hard to maintain a reputation and will publish retractions if necessary is all miles better as a starting point for determining truth, but sadly that cultural activity is nearly dead.

Instant answers seem like a cautionary example since Google has gotten a fair amount of flack over the cases where it inaccurately summarized content. I think these services are going to be very interesting to study whether the average person thinks they're more authoritative because they're branded by a huge corporation and whether that'll decline over time as people realize the limitations.

It's weird. I've personally experienced cases where the highlighted instant answer is obviously incorrect and the full context actually claims the opposite of what the excerpt claims, and those kinds of examples circulate around the web pretty frequently, and everyone who has ever tried to ask ChatGPT or similar systems tricky questions should know how AIs just invent stuff when they don't know the real answers.

So why do companies like Microsoft and Google push in this direction? Why are they making the results more and more opaque? You'd hope that they would care enough to be good stewards of the power granted to them through their information monopoly, but barring that, you'd hope that they'd recognize that people want results they can verify, not just random answers.

Or maybe they're hoping that people don't care about verifying results, hoping that people just want an answer that's not necessarily the right answer? It seems like a dangerous gamble.

It really goes back to the toxic incentives of ad sales. Google wants you to stay on Google.com so they get more chances to show ads. They only care about low quality results if it means you stop using them, which for years wasn’t a serious risk but can change relatively quickly if alternatives arise. They should know, given how rapidly they pulled users away from Alta Vista, Yahoo!, et al. 20 years ago.

For the record, the “New Bing” AI results will not be unsourced but with key facts in sentences tagged in Wikipedia style, pointing towards the source URL. Finally, below the reply there will be a domain summary for an overview but where each domain name is clickable to get to the respective articles on said domains.

In this case, Bing AI will operate very differently from ChatGPT.

In which case we would expect an LLM-based system to output something like,

"(Fact that isn't true)[Source that does not make that claim but comes close enough that you wouldn't notice it just by skimming]"

leading to even more people thinking it's true than otherwise.

Again this is already something that the human equivalents (low-end PR etc.) do all the time.

This is going to get real nasty when you start getting into science of various sorts, where an increasing number of papers, particularly in the social sciences, tend to make extreme and outlandish claims, but then walk them back extensively in the actual paper itself emphasizing it's just a correlation, or otherwise demonstrating an extremely marginal effect.

Yet the media tends to miss the internationally understated nuance and run with the claims at face value, which an LLM will then pick up on and state, "Yes scientists have proven that [x] does cause [y]. [1][2][3][4][5]" That claim then gets repeated elsewhere, and eventually that "elsewhere" goes on to become part of the LLM's new training material where it's basically training off its own output.

It'd be ironic if something that's ideally designed to make the breadth of human knowledgeable more readily available and accessible than ever before, ends up just making society vastly more misinformed than ever before.

Ever since Google started adding those quick answer boxes at the top of search results I've had the double check everything they say. They're quite often incorrect. I mean I know that, but this grandma? They've all been conditioned to trust Google.

Disagree. I think this is akin to Netflix’s Chaos Monkey, which relied on the insight that it is impossible to build infallible systems, so you design failure and recovery in.

Existing Google searches are polluted with false information, and Google’s has been losing that battle. It’s probably not even possible to win.

So rather than saying search engines should always be perfectly accurate and errors are catastrophic, we should accept that search engines are, and have always been imperfect, and need to give us enough info to validate facts for queries important enough to merit it.

The content I write is often half-assedly plagiarised by copywriters or incorrectly interpreted by lazy journalists. This is just an automated version of it. They can use my hard work for their own profit at an unprecedented pace, while still remaining factually incorrect.

When I ask ChatGPT a question, it explains it's reasoning and gives me concepts I can follow up with googling to learn more.

When I use Google for research, I get articles written for SEO to push products and often have to refine and refine and refine to get something useful, which I then can follow up by googling to learn more. With difficulty.

Honestly I don't know how much I'd use ChatGPT if I had the internet of 2016 and Google.

Careful, It explains but both answer and explanation are sometimes completely hallucinated, it sometimes looks like a plausible answer, but actually it completely made up. And this happens way too often for me to take it seriously for now.

It's probably got as good a success rate as many of my colleagues.

I would definitely fact check search results as much as AI, especially the info snippets that appear at the top of Google's SERPs.

For example, until a few months the results for "pork cooked temperature" and "chicken cooked temperature" were returning incorrect values, boldly declaring too low of a temperature right at the top of the page (I know these numbers can vary based on how long the meat is at a certain temperature, but I verified Google was parsing the info incorrectly from the page it was referencing, pulling the temperature for the wrong kinds of meat). This was potentially dangerous incorrect info IMO

> I would definitely fact check search results as much as AI, especially the info snippets that appear at the top of Google's SERPs.

Yes, so would I. And I also double check things like Google Maps -- a tool I find very helpful but don't trust blindly. But... do most people think to take a close look at Google Maps to make sure it makes sense, and trust their own judgement if they disagree with the map? Will most people fact check confident LLM outputs?

Snippets have become so useless I use a plugin to remove them.

What is ridiculous is, when, say, Stack Overflow has a good answer, it is a few lines down or on the next page in the search results, but some page-mill SEO site is in snippets up top with a completely wrong or naively pathetic partially correct answer. It is so annoying it has lowered my opinion on Google a lot in recent times.

You trust everything you find on search engines?

I don't think it's a great illustration of the risks of LLMs.

Ad content invariably gets vetted by humans. The fact that it shows up in the ad demonstrates human failures more than failures of LLMs.

>As a user, if I am asking this question to a search engine, I definitely do not expect to need to fact-check the results.

This is scary to read. You always need to fact-check the results, whether they come from a search engine, an AI, or a primary source!

>We're about to enter a dark ages of crappy AI products

Fine. We need another good winter or ten before we decide we want to commit societal suicide via deepfake tsunami.

> I am asking this question to a search engine, I definitely do not expect to need to fact-check the results.

Genuine, honest question: How did you come to the belief that search engines are reliable sources of truth?

I completely agree that search engines provide a valuable service. But in my own work, I find them to very often point to inaccurate information, sometimes greatly so. I don't think this is terribly surprising, given Sturgeon's law, but still.

I can see how someone could extrapolate Google's goal of indexing knowledge(JTB) into being a reliable source of truth. It's simply a matter of taking them at their word on the J and T parts. The B is up to the user.

Google's branding frames itself as the expert in the novice-expert problem. The vast number of users implicitly take on the role of the novice by virtue of using the product. They've already self-identified as a novice which makes both parties complicit in the arrangement.

As a user, if I am asking this question to a search engine, I definitely do not expect to need to fact-check the results.

And this stance seriously hasn't bitten you in your life or career to date?

Why can't an LLM fact check itself somehow/someway?

So i think the ability of the search engine to say "I don't know" is very important, and most of current chatgpt like models in the market don't have this feature.

There was a lot of stories like "Webb captures it's fist ever picture of an exoplanet" [eg]. My guess is that it's digesting those and not understanding that the "it's" in that sentence is critical.

Here is a prior example of an exoplanet picture: https://esahubble.org/images/heic0821a/

[eg] https://blogs.nasa.gov/webb/2022/09/01/nasas-webb-takes-its-...

Did you mean "its" such as in <https://news.ycombinator.com/item?id=34359839>? Given your statement of this being critical... :) (Advice I also gave at work today: just don't use contractions and the right spelling will usually be obvious. In an informal setting, it's more tempting, but that's the way to easily check yourself.)

Ha, I initially wrote "its" then got nervous I was wrong, overthought it and did get it wrong.

Maybe it's okay if the AI gets its grammar wrong sometimes, as long as it's less wrong than humans?

I like that this thread points out even humans have difficulty with that construction sometimes. We're trying to hold Google's language model to a higher standard than humans in this case I think. I remember "learning" thousands of bits of trivia like that from people who had misinterpreted something they read and misstated it in such a way.

Of course Google has already been putting often-incorrect summaries/factoids in its search infoboxes for a few years now.

It's not a matter of "its vs it's" in this case, but the very existence of the word in the sentence:

"NASA’s Webb Takes Its First-Ever Direct Image of Distant World"

It doesn't matter if one misspells its. You know what it means and it largely defines this sentence. A failure to parse such a relatively simple construct doesn't bode well.

You're of course correct, but perhaps I should have focused more on the concept of skipping words when reading, misremembering, and "reading what you want to read" - those traits are extremely common if not universal at some level in human readers as well.

Well yeah, when I make a tool I want it to do its job correctly. If it doesn't I throw it out. If a human keeps messing up I do the same. A human messing up confidently in their interview probably won't even get hired...

Sure but this seems analogous to creating a claw hammer then showing off how it can be used to drive screws, then saying the hammer isn't doing its job correctly when the screws aren't driven properly.

I think chat technology like this is an incredible tool, but I don't think it's being judged fairly: I don't think its usefulness is as some kind of oracle or advisor expected to provide correct or logical information. That seems so orthogonal (if not diametrically opposed) from its actual function that it really feels like we're being trolled by things like "Galactica". But I'm much more (cautiously) optimistic about the potential use for the technology in web search, which has never been logical or "correct" and has always required critical thinking on the part of its users.

Perhaps there should be more of a disclaimer that the things it says are not and cannot be construed to be factual, no matter how verisimilitudinous.

I can't tell from the tweet why the Bard response is wrong. Is it because some other instrument has taken an image of an exoplanet, or because no instrument has ever done so? ChatGPT seems to believe it is the latter.

Another instrument took an image of an exoplanet, in 2005.

https://exoplanets.nasa.gov/resources/300/2m1207b-first-imag...

Your LLM needs fine-tuning, the linked article says it's 2004:

> 2M1207b is the first exoplanet directly imaged and the first discovered orbiting a brown dwarf. It was imaged the first time by the VLT in 2004.

i'm not a grammar expert but i think as a layperson there is a little bit of ambiguity in how a reader could interpret Bard's response. If i were writing i would probably specify "first ever picture". I think a lot of times words like "the first" are relative and context specific which the Bard answer lacks.

There is no ambiguity. We don't call Neil Armstrong "the first ever man on the moon". "The first" has a clear meaning and making any qualification of that implicit is either deceptive or bad writing.

> We don't call Neil Armstrong "the first ever man on the moon".

https://www.scout75.com/apollo-11.html#/

> On July 20, 1969 America landed the first ever man on the moon in Neil Armstrong.

i guess my take was that i find this a more excusable grammatical mistake than say getting a simple addition question wrong, which we've seen from gpt3. i could agree with "deceptive or bad writing" and maybe that's a more sinister error than something obvious wrong now that i think about it.

it's funny for me to think that i'm going out of my way to try and give Bard the same benefit of the doubt and work to try and find a way it could be right, the same way i would for a friend.

For what it's worth, ChatGPT is also wrong: https://i.imgur.com/CsQyEc2.png. The correct answer is https://exoplanets.nasa.gov/resources/300/2m1207b-first-imag... as techsupporter posted in a sibling comment

For what it's worth, thats not accurate - a spectrograph was used in 1995 to discover the first exoplanet by Mayor and Queloz, and they received a Nobel prize for this. So GPT is right. One can be pedantic and say that not an "image" but most would disagree.

It's not a direct image, which is the specific question, and I don't know if you're including NASA in "most" but NASA disagrees as well, per their article. So do Wikipedia editors: https://en.wikipedia.org/wiki/List_of_directly_imaged_exopla...

My thoughts exactly. natural language semantics is imperfect and human reasoning is weird. Let's not mistake LLM models as a single source of absolute truth, but a funny & bullshitting assistant who happens to read and vaguely remembers much information.

Finally, a completely artificial version of the archetypical middle-aged man in a pub who is very confidently wrong about stuff. The ultimate triumph of ML, an replicant sitcom character.

Cliff Clavin?

This would be a great name for a rival llm. Tongue in cheek: welcome to Claven v3.1.

lol brilliant.

With every pint he becomes more convincing too.

People shouldn't underestimate Google here. I'm not a fan (not for a decade ... Reader still stings amongst many other missteps), but I'm constantly impressed by how much better their Assistant, translation and speech-to-text stuff is versus things like Siri, etc. Almost as if Google handles the larger dataset that it has access to in a much better way than the other companies. I wouldn't bet against them doing that again here.

To me they seem to be beaten at all of those areas. DeepL does translations much better than Google Translate (for the four languages I know and speak regularly), Siri does speech-to-text (and text-to-spech) better and so on.

The only thing Google really beats the competition on is Android Auto, which is miles ahead of CarPlay. CarPlay still covers the entire screen when there is an incoming call, so good luck seeing your turn-by-turn directions if you happen to be navigating.

Google products to me are the definition of mediocrity. They're "designed by committee" and play it safe, they're probably "good enough" for most people, but they're certainly not the best in class. As you noted Google Translate pales in comparison to DeepL, Google's TTS are worse than similar offerings from Amazon/AWS, let alone more niche focused products like Voicepeak, and you know that they're phoning it in for search given that there's a startup (Kagi) whose sole value proposiion is to use Google's own API then rerank results to achieve much better results.

Of course it does, it has to compete with ChatGPT after all! Confidently and authoritatively lying is an important part of making the ChatGPT output believable.

NASA: "2M1207b - First image of an exoplanet": https://exoplanets.nasa.gov/resources/300/2m1207b-first-imag...

"2M1207b is the first exoplanet directly imaged [...] It was imaged the first time by the VLT in 2004"

This isn't incorrect, it's just ambiguous. The JWST did indeed "take the first pictures" of a planet outside our solar system (https://www.npr.org/2023/01/12/1148626359/nasa-webb-telescop...). However, we have previously directly imaged other exoplanets using different instruments—it's just that this exoplanet in particular (LHS 475 b) has never been directly imaged before Webb.

It's the sort of question where you would expect an expert (or a thoughtful layperson with access to a search engine and a few minutes of spare time) to give a better, less ambiguous answer. However, since it is technically correct, I think it's a fairly minor sin in the grand scheme of popular science education, which commonly propagates actual falsehoods without the help of generative ML—e.g. iodine always sublimes rather than melting, aerodynamic lift relies on equal travel time of air on both sides of a curved airfoil, etc.

re: this particular ambiguity, one does wonder what the model was "thinking", but I suppose it doesn't matter because we'll never know.

---

I'm not sure if I think this technology is good or bad, because I don't really understand how most people use search engines. When I use them myself, I think I am fairly sensitive to ambiguities and logical contradictions, cross-checking multiple sources to extract a high-confidence answer or walking away if I'm not satisfied that I've found one. Most folks on HN are probably the same way, and I don't think generative ML can do better than us, yet.

On the other hand, will Bard's results be better or worse than e.g. taking the first Quora result as gospel? I would honestly guess that, on average, Bard (and ChatGPT, etc.) will do better. So, less critical users may get better results from these systems.

How many people are in the former camp, and how many in the latter? Of course, changing the dominant search paradigm will have an effect on all users. It also remains to be seen how the search vs. spammers arms race will evolve with the advent of generative ML search tools.

Nobody is going to interpret "a" as "some particular". The fact is incorrect.

I am glad you were able to offer your opinion here.

If JWST took a picture of LHS 475b then where is the picture? Isn't exoplanet HIP 65426b the first that JWST took a picture of?

HIP 65426b is the first exoplanet JWST imaged, but it was previously discovered by VLT-SPHERE, which I believe is also a direct imaging instrument. So it would not be correct to say that JWST took the first pictures of HIP 65426b.

However, JWST was the first instrument to directly observe LHS 475b, also confirming its existence for the first time. As to "where are the pictures?", IIUC these observations were made using a spectrograph instrument that does not produce a picture as such. The results are easy to find on the web with a search engine.

One might say Bard's result is wrong in the sense I have proposed because a spectrograph result isn't a picture. However, given that the prompt explicitly specified that the results should be tailored to a 9 year old, I think that would be pretty disingenuous.

Yet, for some reason, people still get bent out of shape when I point out that I was the first person to implement a hashtable, host a web page, run an experiment to measure the force of gravity, stare out a telescope at the moon, post a comment on Hacker News, etc, etc...

If you really think these are semantically comparable^[1] to the statement under examination then we are not speaking the same language, despite appearances to the contrary, and therefore have nothing to discuss.

I get it: hating on AI is very trendy today and it feels good to publicly agree with a righteous cause. I'm generally not a fan of this technology myself---most of these models incorporate stolen work of mine in their training sets, unattributed and uncompensated, and I think they are likely to be a disaster for the human species. I have not implemented any of them in my creative workflows or other aspects of my life, and I don't plan to.

However, I think it is important to see one's enemy clearly, and the vast majority of people participating in the anti-AI discourse are abjectly failing to do so as in this case. This can only hurt their cause, and by extension mine.

^[1] Exoplanets is a category of unique non-fungible objects which can be meaningfully discovered and measured on an individual basis. Hash tables are an abstract concept, as is hosting a web page (and creating a web page to host it fundamentally precludes discovering it, except in the sense of discovery within the creative process). Gravity, similarly, is a single 'thing' that, once discovered, cannot be discovered again.

One might easily and correctly say, however, that X person was the first to measure the force of gravity to a certain precision, or that Y person was the first to develop a particular implementation of the hash table concept, e.g. cuckoo hashing. These achievements are meaningfully distinct from the discovery of the broader concepts they extend. Discovery of a particular exoplanet is similar; however, it may surprise you to learn that the English language treats different kinds of 'things' differently, and in this case those varied, underspecified conventions of language have produced an ambiguity.

Well, it is called "Bard". What did you expect? A bard makes up stories and sets them to music. Sometimes the stories are true, sometimes embellished, sometimes false. Ideally, they all sound good enough and keep the patrons entertained.

I was expecting it to take its bow, climb on top of the town tower, and shoot a fire-breathing dragon in the only part of its body not encrusted with jewels, but to each their own.

Its fun to see all the people reacting "Google's Bard shows incorrect information", and at the same time say "Google is done, they couldn't even release a LLM chatbot first".

Think how bad it would have been if Google released Bard first and it returned inaccurate information, or worse was racist. LLMs are just language generating models and may not be fully accurate.

That's not the point. ChatGPT says all sorts of stupid stuff. It's expected. Showing an ad where your LLM is saying stupid stuff, as an example of its capabilities, is amusing because it's obviously unintentional yet completely and absolutely appropriate. It'd be like a live Coke ad that said "Drink Coke and you can be like this guy!" while accidentally panning over to an obese manblob instead of e.g. Usain Bolt.

At least it's being honest about LLM capabilities. If it only showed 100% facts, that would be false advertising.

Someone responded that Google is terrified of being left being Microsoft. I think that's hilarious because it highlights the thing google should actually be terrified of - the fact that it has lost the capacity to innovate (same applies to MS, FB, etc).

Given that the flaws of this new approach to search would be more or less shared by any other effort based on similar algorithms, thus also openai/microsoft, maybe this is just a PR battle and Google declaring, "yes, we can outcompete you in recklesness and sillynes"...

An interesting question is whether the inability to ascertain "truth" can be contained somehow so as not to lead to mass ridicule. I doubt it but not an expert in this area. My guess is that it would have to be limited to specific curated domains, a combined system that could be invaluable to knowledge workers but maybe not exactly the desired business model of "big tech"

Whilst it's funny this showed up in the advert, we do kind of need to temper expectations. These products are meant to be "Artificial Intelligence" - they will get things wrong just like normal intelligent people. This is why it's kind of why they're not well suited for search, because people who are searching for stuff need to be make their own judgement about the credibility of a source. Sometimes I need a high level of confidence sometimes not. If what you're expecting is for it to be artificially intelligent and omniscient, you're waiting for us to design God.

Tell that to the people developping the product. Nothing in the UI invites caution.

Maybe the Butlerian Jihad will happen because computers get too dumb, not too smart.

The Butlerian Jihad happened because computers made people too dumb.

Oh god it's too late

I’m thinking that even ChatGPT is overrated - to simplify it’s a query(smart/ai/algorithmic) to a very unstructured database (although translated to structured along the processing).

The quality of response depends on how up-to-date the data is with your context. Unfortunately most of complex/useful responses require context.

The worst part is it kills content ecosystems.

I'm reminded of a similar instance a couple of years back when one of my astronomy professors noticed that if you typed into Google "mass of the Sun in solar masses" you would get back 0.9995 instead of 1.

Apparently someone missed the word "experimental" in the announcement.

Sure, but they made an ad bragging about it. It’s funny that they fucked that up.

Anyone remember Google panicking after Facebook took off so they bought Orkut and rushed out a crappy “social app development environment”?

No, because it didn't happen? Orkut was rather famously a 20% project by and named after a Google engineer, predated "The Facebook" by weeks and Facebook as a global public social network by two years, and was shipped more as a response to Friendster (which Google had just tried and failed to buy for $30M) and MySpace.[1][2]

Orkut had more users in India than Facebook until 2010[3] and in Brazil until 2011,[4] by which point Google had moved on to trying to make Google+ happen.

1: https://www.baltimoresun.com/news/bs-xpm-2004-01-24-04012400...

2: https://techcrunch.com/2006/10/15/the-friendster-tell-all-st...

3: https://web.archive.org/web/20100828201838/ibnlive.in.com/ne...

4: https://techcrunch.com/2012/01/17/facebook-in-brazil-a-big-e...

Oh man, that was in-house? Even worse!

I found the event I was invited to in 2007:

https://www.wired.com/2007/11/google-summons/

It did not seem like they knew what they were doing and everything was very rushed.

I remember Google panicking about Facebook, shipping Google+ and telling the entire company that their annual bonuses would be tied exclusively to how well their work supported the success of Google+.

I think it's funny that in a thread about LLMs offering up incorrect information as factual, there's a bunch of anecdotes that present incorrect information as factual.

You kinda did what an LLM does: Took a bunch of contextual cues (Orkut was owned by Google and ultimately shut down; Google often has knee-jerk reactions to the industry; Google started Google+ as a way to compete with Facebook; Google often buys companies as a way to compete), and spat out a confidently-wrong, summarized autocomplete based on that: "Google bought Orkut as a knee-jerk reaction to compete with Facebook!"

Not what happened though. Orkut actually launched a month before facebook and it was a 20% project, it wasn't a panick move or a big project for them. And it was great, imo google did badly on not maintaining it.

You can make this point about google+ though, totally panick move.

If you are expecting AI to be correct then you are holding it wrong.

The correct way to relate to AI is to listen and if the answer matters, verify.

I’m expecting the marketing material to not have mistakes in it, especially when the gist of the article surrounding it is “we’re taking it slow because we want to ensure it doesn’t spit bullshit”

> If you are expecting AI to be correct then you are holding it wrong.

That’s not the message and expectation we’re being given, these models are being hailed as the future today. When something really bad happens because someone over relied on these systems, hand waving with “well, you shouldn’t have expected the answer to be right” won’t cut it.

> The correct way to relate to AI is to listen and if the answer matters, verify.

If the answer doesn’t matter, why are we asking? And if we need to verify, what’s the point of asking the AI?

What does experimental mean?

It means society is the test subject, so pray we avoid every single possible pitfall that could lead to nuclear armageddon.

A test that generates evidence or demonstrates a known truth, which Bard also apparently can't do, and which the marketing team didn't do before making this ad.

Sorry but you’re incorrect.

Oxford dictionary says: “experimental is adjective. 1. (of a new invention or product) based on untested ideas or techniques and not yet established or finalized: an experimental drug. 2. (of art or an artistic technique) involving a radically new and innovative style: experimental music.”

I don’t believe you didn’t known the meaning of the word but you intentionally twisted it. I find it a bit ironic.

And this is why it’s not AI. There is zero intelligence in it.

It has no idea what you’re asking it or what anything means. It is a huge markov chain and it generates random responses. It knows nothing beyond that.

AI Test Kitchen already showed it's way behind

The AI is aptly named as Bards are storytellers. Plus, they are barflies, so the stories come with an extra twist — or is it shaken.

Last year Google showed COVID misinformation in its "suggested answers" above actual results, so this may not be worse than the status quo.

Post-truth tools for a post-truth world.

I asked the same question to chatGPT, and it gave only future-looking statements, I guess because it was only trained on pre-JWST-launch data.

PROMPT: what new discoveries from the James Webb Telescope can I tell my 9-year old about?

ANSWER:

> Here are a few things that the James Webb Space Telescope (JWST) might discover that you could explain to a 9-year-old:

> Planets around other stars: JWST will be able to study exoplanets in more detail than ever before, potentially finding new ones and learning more about the conditions on their surfaces.

> The beginning of the universe: JWST will be able to observe some of the oldest stars and galaxies in the universe, helping us learn about how it all began.

> The formation of stars and planets: By observing the clouds of gas and dust where stars and planets are forming, JWST will help us understand how they are born.

> The secrets of distant galaxies: JWST will be able to observe the light from distant galaxies, giving us a window into their structures and evolution.

> The mysteries of black holes: By observing the material around black holes, JWST will help us learn more about these mysterious objects and how they shape their surroundings. Overall, the JWST will help us answer some of the biggest questions about the universe and our place in it.

But ChatGPT just doesn't have this information available to it, as its data stops before it was fully launched. You can ask it about the invasion of Ukraine and it's completely oblivious to any major recent incident, it just will keep talking about 2014.

They are incompetent and do not care.

They couldn't even modify the ad to show correct information for lying's sake.

Everytime I see someone finding something wrong about these things, I am reminded of Stoll in '95

https://www.newsweek.com/clifford-stoll-why-web-wont-be-nirv...

He also had some similar things in Cuckoo's Egg. I wish I could find the quotes, but there was something about email not working all the time and therefor pointless to use.

I'm glad people are finding all the flaws in ChatGPT and the LLM things now, but won't much of this be fixed as it gets better? From my very limited view, these things are amazing, and far from perfect, but damn the can do so much already.

I guess I'm not sure why there's such a rush to dismiss this, when it's clearly a game changer in its present form, and yet so very new (at least new to me).

I think there are two things to be aware of right now: 1) This technology is revolutionary and will change the world 2) This technology is very unreliable right now and should be seen as a tech demo rather than an actual assistant

(2) is a big problem. Kids submitting term papers with wrong information is one thing, but people are using ChatGPT for things that they shouldn't be, given how many mistakes it makes: https://www.law360.com/pulse/articles/1573108

However, Stoll was largely correct: the web is not Nirvana. Some structures persisted (e.g. Wikipedia) to help round off data correctness, albeit imperfectly, but his central concerns are just as valid today as they were in 1995.

As a user, the ability for a LLM to literally make things up and present them alongside other true data with no qualms or disclaimers is highly detrimental to the central use case.

He was correct about some things, but also largely incorrect as well.

> Visionaries see a future of telecommuting workers, interactive libraries and multimedia classrooms. They speak of electronic town meetings and virtual communities. Commerce and business will shift from offices and malls to networks and modems. And the freedom of digital networks will make government more democratic.

> Baloney. Do our computer pundits lack all common sense? The truth in no online database will replace your daily newspaper, no CD-ROM can take the place of a competent teacher and no computer network will change the way government works.

Telecommuting workers is reality. Interactive libraries are a reality. Multimedia classrooms have been a reality for over a decade. Electronic town meetings, maybe not, but virtual communities? Very much a reality. Malls are dead and brick and mortar has been hurt extensively by Amazon. Offices lay empty due to remote work.

Newspapers are largely dead, at the very least compared to what they once were. There is plenty of online learning, largely without in person learning. Computer networks have definitely changed how the government works.

I have no doubt these technologies will improve, but there's another argument to be made. The tech will get better and we'll be all the worse for it.

Stoll argued the tech will not be good enough, but paid little thought to the ramifications of the technology succeeding. The arguments against LLMs like Bard and ChatGPT that I have seen are assuming they'll be successful.

They'll become less stupid, but the problem is not that they are wrong but that they are, at present at least, unassailable. You cannot fact check through most of the normal means. You can not research the publication or the author or the date the words were written because that has all been stripped away.

You could check other sources (eg old fashion google) and put in the leg work, but as these get better that will feel less necessary - potentially exacerbating this problem.

That's not to say they aren't useful. I used Chat gpt the other day to get some work done and was impressed. However this was work easily verifiable because it was technical and had immediate feedback when the ai inevitably gave me slightly incorrect code. The same can not be said for facts, figures, and arguments of thought.

> but won't much of this be fixed as it gets better?

Not necessarily, no. There's a large aspect of garbage in, garbage out, to these things.

> when it's clearly a game changer in its present form

Is it? What's the game? Being wrong about telescopes?

And the more content that is created by these LLMs, the more garbage the LLMs will consume while quality content creators are simultaneously disincentivized, leading to worse content from these LLMs. Terrible image, but an organism cannot survive eating its own poop forever.

90% of that Stoll article has been proven completely correct.

Reply on news.ycombinator.com