Comment by goranmoomin | Hacker News Reader

Comment by goranmoomin | Hacker News Reader

After asking a few questions, I’m very disappointed with its quality compared to Bing Chat.

My current AI chat benchmark is to ask about LLVM APIs. I’ve asked to both Bing Chat and Bard three questions:

- What info can the LLVM lazy-value-info analysis pass analyze?

- Can you give me code samples to use the info?

- Can you give me a list on important LVI methods?

Both fared well on the first question, but on the second question, Bard either didn’t produce LLVM code at all (showing examples of LLVM IR), or hallucinate non-existent methods not even close to what existed.

Bing chat, in comparison produced correct answers that are very helpful. In my experience Bing chat almost always produces something that is both coherent and useful; Asking LLVM questions, Bing chat can search the Doxygen documentation, find out common LLVM patterns on its own (like using a Worklist while iterating), and I’ve succeeded in writing whole LLVM passes that compiled in one go, from a very light description of an algorithm.

Maybe I’m spoiled with Bing chat, but I don’t think I’ll be using Bard that much if it’s quality doesn’t improve. Very disappointed personally. (I’d have expected Bard to work much better mostly just because Google searches better than Bing.)

Do we know if the comparable here is Bing or chatgpt4?

By which I mean, does bard have internet access?

Bard says it doesn't:

> I do not have internet access in the traditional sense. I am a large language model [blah blah]

> However, I do not have access to the real-time internet. I cannot search the web or access current events. I can only access the information that was used to train me, which is a massive dataset of text and code. This dataset is constantly being updated, so I am always learning new things.

Bard isn’t self-aware and can’t know what it’s capable of except for what’s in an initial prompt.

It doesn't have to be to answer this correctly. ChatGPT for example had a pre-coded response since they predicted that this would be a common question, and it seems plausible (if not certain) that Google did the same for Bard.

It treated you better than it treated me when I asked that a few weeks ago. The response it gave me was some smartass bullshit about how it wasn't going to do my homework for me (and no, neither homework nor school in general had anything to do with anything we were talking about).

Too much training from StackOverflow comments.

It's appears to be sort of hooked up to the internet.

For example, I asked it the weather in one location and it provided accurate current weather, but in another location it responded with an "as of 10AM" answer.

It definitely has much newer information than OpenAI models, but also hallucinates a fair bit.

I asked Bard for the weather in my locale and it came back with a wildly inaccurate prediction. I then asked it where it got the information and it said from "Google Search."

I checked weather.gov and told Bard the prediction was wrong and it came back and said yes it was incorrect and now agreed with me based on the weather.gov prediction.

I then tried different locale to see if it's smart enough to use a better data source on the second try. I asked about another city my family lives in and it once again came back with an inaccurate prediction. I asked where it got the prediction from and it gave me a link to the weather.gov website. I checked and the prediction Bard was giving me for the high temperature was actually the current temperature.

I can't say for sure, but it does seem like Bard has access to these websites but is struggling to properly interpret them.

Bard seems to be able to access indexed pages, at least. For example, two hours ago, I asked for a recap of https://www.reddit.com/r/cpp/comments/13bw8ud/the_future_of_..., and Bard did a good job with it. And, from today, Bard supports my native language, Korean.

GPT-4 with web browsing is also available on chat.openai.com for subscribers.

Not for everyone unfortunately :(

I never get good results with bing, not nearly as good as phind.com but to each their own. Soon I'll just be telling my autogpt go research something and report back and it'll talk to the Google/Bing ai's and go back and forth until its satisfied and come back and give me a run down.

Reply on news.ycombinator.com