Hacker News Reader

Hacker News

Comment by babuskov | Hacker News Reader

Parent Full thread

babuskov

> ChatGPT will be the google killer

This was my prompt: What's the relative distance between Sun and its planets compared to the size of Sun?

ChatGPT answer:

> The average distance between the Sun and its planets is approximately 5% of the Sun's diameter.

If you run into bunch of such answers, you will soon stop trusting it for anything.

As for Google - it doesn't even try to answer the question. But no answer is often better than a wrong one. Google at least gives me link to websites that have enough data for me to calculate it myself.

View on HN

Google, Wikipedia, and historically the internet had been unreliable for many of the queries. More so in the beginning. But over time most of the errors are ironed out. ChatGPT is a very promising start I would say.

In the end convenience wins out at cost of correctness. The proportion matters, convenience::correctness.

How so? Anyone who sincerely asks a question wants the answer to be 100% correct, or the person answering to make it clear that they have a low confidence or don't really know. If the answer is false but stated with certainty, it is not only useless, it can even be worse than useless. There are other quality dimensions for answers, of course, such as the right level of detail and comprehensibility for the person asking, but these are all useless if the answer is false in the first place.

> ChatGPT is a very promising start

It is not the start. It is at least the third iteration (if not more) of this tech.

> In the end convenience wins out at cost of correctness.

There's nothing convenient about a wrong answer. Wrong answers have no value. I could very easily build you a convenient service that is rarely correct but nobody would care.

For the same question I got:

> The distances between the Sun and its planets vary greatly. For example, the distance from the Sun to Mercury is about 36 million miles, while the distance from the Sun to Pluto is about 3.6 billion miles. The size of the Sun is about 864,000 miles in diameter, which means that the distance from the Sun to its planets is much greater than the size of the Sun itself.

Then it says something that doesn't follow, probably answering the more frequent question of relative sizes of the planets compared to the Sun

> To put it into perspective, if the Sun were the size of a basketball, the nearest planet, Mercury, would be about the size of a grain of sand, and Pluto would be about the size of a golf ball.

Interesting. I got a totally different answer:

>The distance between the Sun and its planets is relatively small compared to the size of the Sun. The Sun is a massive object, with a diameter of about 1.4 million kilometers, while the distance between the Sun and its planets can vary greatly. For example, the distance between the Sun and Earth is about 150 million kilometers, which is about 100 times the diameter of the Sun. Similarly, the distance between the Sun and the outermost planet in the solar system, Neptune, is about 4.5 billion kilometers, which is more than 3,000 times the diameter of the Sun. So while the distances between the Sun and its planets are not insignificant, they are still relatively small compared to the size of the Sun itself.

I wonder if it chooses units based on where you are located, or if it's just arbitrary. Also, it seems clear from this example that is has no context for the answer because it doesn't see that its first statement is contradicting its next statements.

If you use the Q&A preset in the playground it will give Unknown if it doesn't know. You can also set Show Probabilities to Least Likely and see which parts of the result are guesses.

I also changed the Temperature from 0 to 0.5, and it gave the right answer:

Q: What's the relative distance between Sun and its planets compared to the size of Sun? Show your math.

A: The relative distance between the Sun and its planets is approximately 1/100th the size of the Sun. This can be shown mathematically by calculating the ratio of the radius of the Sun (6.96x10^8 m) to the average distance of the planets from the Sun (1.5x10^11 m), which gives a ratio of 1/100th.

Reply on news.ycombinator.com