Comment by underyx | Hacker News Reader

Comment by underyx | Hacker News Reader

Parent Full thread

As always, I tried my favorite prompt:

> I'm playing assetto corsa competizione, and I need you to tell me how many liters of fuel to take in a race. The qualifying time was 2:04.317, the race is 20 minutes long, and the car uses 2.73 liters per lap.

Claude 2 got really close to the right answer on the first try. The only thing it didn't consider is that time for 9.6 laps means driving 10 laps total, since incomplete laps must be finished after time is up. For the record, GPT-4 does take even this catch into account and gives a perfect answer, while Bard outputs results that are off by orders of magnitude.

Claude 2's response was:

> Okay, let's calculate this:

> * Your qualifying time of 2:04.317 equates to an average lap time of 124.317 seconds

> * A 20 minute race = 20 * 60 = 1200 seconds

> * So in a 20 minute race you can do roughly 1200 / 124.317 = 9.67 laps

> * At 2.73 liters per lap, that's 9.67 * 2.73 = 26.38 liters of fuel needed for the race

> So I would recommend taking on at least 27-28 liters of fuel to be safe for a 20 minute race, given your average lap times and fuel consumption. It's always better to take on a bit more fuel in case there are any safety cars or variations in your lap times during the race. Let me know if you have any other questions!

> The only thing it didn't consider is that time for 9.6 laps means driving 10 laps total, since incomplete laps must be finished after time is up.

I would be hesitant to judge an LLM on the various ground rules of the real world that it happens to have internalized or not, that aren't given in the prompt.

For my uses I'd want to have prompts that provide necessary context anyway - and would actually favor ones that can do it from the prompt without needing to already know - so that it is not limited when it encounters a novel problem domain.

Eventually one of these comment threads is going to be included in the training set invalidating this as a test.

Which is why knowledge cut off date is important. I prefer if it is frozen to pre-ChatGPT-3.5. Anything post-ChatGPT-3.5 release date should be considered tainted - imagine the sheer number of articles generated by spammers who used ChatGPT.

Knowledge cut-off date doesn't prevent your model from getting tainted though - if you're doing any kind of RLHF, unless all your human reviewers were kept isolated from the world since ${knowledge-cutoff-date}, they will inadvertently give the model glimpses into the future.

It's not immediately apparent to people just how much leakage can happen this way. Up to a year ago, I'd probably give people this story[0] to ponder on, but now it's no longer a hypothetical - GPT-3.5 and GPT-4 are clear, practical demonstrations of just how much knowledge is implicitly encoded in what we say or write, and how this knowledge can be teased out of the input data without any prior context, completely unsupervised, given sufficient time and effort (which in silico translates to "sufficient compute", which we already have).

--

[0] - https://www.lesswrong.com/posts/5wMcKNAwB6X4mp9og/that-alien...

That might be fair in the short term. However it's not a workable option long-term, or all such models will be very limited in their knowledge as humanity advances technologically and culturally.

If you want me to be honest with you, LLMs are themselves a short term approach and can get us to, at max, AGI levels (for this current era). I don't see us getting to ASI with just LLMs. For the sort of "emergent ability" that ASI requires it has to be something more "simpler" and the learning be more "virulent" / "instantaneous" (not sure if these words convey what I really want to convey). Otherwise, LLMs will always have a "maxima" at which point it fails. And that maxima is collective intelligence of all of humanity in the current epoch. If you go back a 1000 years, the collective intelligence of all humanity would be completely different (primitive even). Would LLMs trained on that data have produced Knowledge that we know today? I don't think so. It could still, theoretically, reach AGI for that era and accelerate pace of learning by 50-100 years at a time. LLMs will surely accelerate pace of learning (as tools) even now but by themselves won't reach ASI levels. For ASI, we really need something more simpler/fundamental that is yet to be discovered. I don't feel LLMs are the way to ASI. AGI? Yeah possible.

Same is true for humans - a scientist inventing everything from their head would not achieve much, but if they can conduct experiments, and if they persevere, they eventually make discoveries. A pure LLM is like the first case, a LLM with tools or part of a larger system is the second.

Forgot the final one liter required for the FIA sample, gottem.

When you test this prompt, do you do multiple separate queries to see how much variance there is in the answers?

Google should train bard on their own interview questions. Maybe it will get this right after that

Interesting that is mentions the safety car making you need more fuel, when really you need less because of the reduced speed under it.

Reply on news.ycombinator.com