Comment by michaelbuckbee | Hacker News Reader

Comment by michaelbuckbee | Hacker News Reader

I learned a new term in the context of AI recently: "Specification Gaming".

There's a big list here: https://t.co/OqoYN8MvMN

But it's stuff like:

- Evolved algorithm for landing aircraft exploited overflow errors in the physics simulator by creating large forces that were estimated to be zero, resulting in a perfect score

- A cooperative GAN architecture for converting images from one genre to another (eg horses<->zebras) has a loss function that rewards accurate reconstruction of images from its transformed version; CycleGAN turns out to partially solve the task by, in addition to the cross-domain analogies it learns, steganographically hiding autoencoder-style data about the original image invisibly inside the transformed image to assist the reconstruction of details.

- Simulated pancake making robot learned to throw the pancake as high in the air as possible in order to maximize time away from the ground

- Robot hand pretending to grasp an object by moving between the camera and the object

- Self-driving car rewarded for speed learns to spin in circles

All of which leads me to think that if you can't at some level explain how/what/why it's reaching a certain conclusion that it may be reaching a radically different end than you're anticipating.

Not quite the same, but equally as disastrous is data leakage. This comes in some fun and unexpected forms. One example is from the fisheries monitoring competition on Kaggle, where NNs were learning to predict fish caught using the background image of the boats rather than the fish shown on camera. i.e. different boats tend to catch different fish

inductive learning.

I have recently started learning ML and trying competitions on Kaggle. And I also seeing that in many of the competitions I have tried. The biggest predictor turns out to be the feature which wont be present when the data is created.

Great examples. I think algorithms in sites like Facebook did something similar. In order to maximize view/clicks, they themselves created problems like an echo chamber or promoting divisive articles which in turn made the whole experience worse

And Google has "simplified" search by fuzzing terms until it is somewhat possible to use even if you really cannat spel.

In the process they broke it so badly I now prefer DDG. (Who also gives me results I didn't ask for.)

Facebook algorithms making the whole experience worse is being quite charitable in describing what it did, considering it's huge role in spreading of fake news and biasing people's thinking towards the presidential election along with creating strong negative feedback loops.

There are some wonderful examples of this in tom7's videos about learnfun/playfun, an AI that (in short, very simple/stupid terms because that's all I know) in learnfun watches human gamepad inputs + the NES memory, and in playfun tries to provide its own inputs to make numbers in certain NES memory locations (those that went up for the human player) go up.

One way to not lose is to just pause the game. Minor spoiler: learnfun figures that one out :-)

http://www.cs.cmu.edu/~tom7/mario/

So "Specification Gaming" seems to be a new age The Monkey's Paw?

https://en.wikipedia.org/wiki/The_Monkey%27s_Paw

This is my favorite: "A robot arm with a purposely disabled gripper found a way to hit the box in a way that would force the gripper open"

Reply on news.ycombinator.com