Proof of Gemini cheating: https://i.imgur.com/eYJDFjS.png
Answer about cherries falling from the sky...
(there is no question or context beforehand, this is the first question of the chat)
Answer about cherries falling from the sky...
(there is no question or context beforehand, this is the first question of the chat)
https://i.imgur.com/3sNr3LW.png https://i.imgur.com/EIj0nZg.png
Edit: When I did your prompt it got it right on the first try fwiwi https://i.imgur.com/E3zYEca.png
------
Let's break down the information given:
The school is two blocks north of the pool. This means if you are at the pool and move two blocks north, you'll arrive at the school.
The convenience store is one block south of the school. Therefore, if you start at the school and move one block south, you'll reach the convenience store.
Based on the first two statements, the relative positions from north to south are: School, Convenience Store, Pool.
Thus, the third statement saying "The pool is south of the convenience store" is indeed true, based on the positions provided by the first two statements.
-----
But it starts weirdly messing up, when I asked it to explain it using coordinates.
---
Let's use a simple coordinate system to represent the positions of the school, the pool, and the convenience store. Let's assume each block corresponds to one unit in this system. We'll start with the pool as our reference point at coordinates (0,0). Remember, moving north increases the y-coordinate, while moving south decreases it.
The school is two blocks north of the pool. If the pool is at (0,0), moving two blocks north puts the school at (0, +2).
The convenience store is one block south of the school.
If the school is at (0, +2), moving one block south puts the convenience store at (0, +1).
Given these positions:
The pool is at (0, 0).
The school is at (0, +2).
The convenience store is at (0, +1).
Therefore, instead of the pool being south of the convenience store, the pool is actually located to the north of the convenience store if you consider "north" to mean increasing y-coordinates. The convenience store, being at (0, +1), is southward relative to the school but not to the pool. The pool, being at the origin point (0,0), is south of both the school and the convenience store in terms of their y-coordinates. Thus, the third statement, "The pool is south of the convenience store," is false according to the given coordinates.
---
It gets coordinates and everything right, but still somehow messes up North and South.
So much money being poured into AI startups just for them to have to resort to cheating to improve their results a tiny bit and claim advancements.
and not "cherries" and "pears" like the example found on the internet.
In some ways, this is proof that Gemini isn't cheating... It is just doing typical LLM hallucination
Do LLMs have logical rules built in? What makes them different to a very advanced Markov chain?
Are there any models out there that start from logical principles and train on top of that?
(Apologies for poor understanding of the field)
Really nothing. There's some feedback structure in the layers of the model, it's not just one big probability table. But the technique is fundamentally the same, it's Markov, just with the whole conversation as input and with billions of parameters.
Handcrafted by humans ? No
But it's still possible to learn such rules from the data in an effort to complete the primary objective (predicting the next token)
Edit: Not native. I'm not sure 'rules of engagement' is the correct english term here.
Thank you for the explanation. It seems like the LLM "plays" to learn? That's very cool, thank you again.
If you're trying to make the case that it's cheating because it is already in the training set, then you probably should come up with different questions. This is machine learning 101.