Hacker News Reader

Comment by WalterBright | Hacker News Reader

If a computer program is generating the training data, aren't you just training the AI to do the same thing as the already existing computer program does?

View on HN

Not at all. The existence of the Gran Turismo game (a simulator) is not the same thing as an AI that can play Gran Turismo.

But a better analogy would be if an AI generated a computer game that another computer can learn to play.

In the end, it’s more like an anonymization layer than anything. If a computer is trained to generate input data for other computers to train with, there’s not a lot special going on.

Generating an environment and acting within it are wildly different things. Eg Tesla generates virtual camera footage of traffic situations they want their vehicles to handle correctly. The footage generator is basically a scripted video game director, while the trained AI is one of the most complex software projects ever.

  "But a better analogy would be if an AI generated a computer game that another computer can learn to play."

Doesn't the latter AI have a policy which contains novel information that does not exist in the former AI?

Even if what you say is true in some abstract information theory sense (and I would question that), there is a world of practical difference in the usefulness of a trained self-driving AI and the game engine within which that AI functions.

But.... if you were to train a model on the simulator of the game, you would expect it to pick up on the rules programmed into the simulator.

This is really no different than it picking on the rules embedded in the data gathering of real world data. Any implicit and hidden decisions in that space would be expected to find their ways into the ML.

The tricky part is that the simulator itself may not have easy to understand rules. Waymo has a Neurips talk about training world agent models that are used for car behavior in the simulation itself. Trying to make world agents that are indistinguishable from real-world vehicle behavior (e.g. minimizing jenson-shannon curvature entropy) is a completely different task than training a model to safely transport you somewhere.

Right. That was my angle. Sometimes, they are ready to see after the fact. But, just like bias can enter into data collection, so too can it enter simulation.

Yes but isn't the optimal policy novel information?

But it's it free of bias and a good execution policy in the real world?

That probably depends on how realistic the simulator is. Your trade off is more bias (due to simulation inaccuracy) in exchange for unlimited data (and hence less variance).

Agreed. I could see a good case of training heavily on simulated data, verify/validate on collected data.

My question is ultimately how this really helps with making the system ethical. Just moved the bias from collection to simulation. And... I can't see simulation being less impacted with bias.

Ethical is not the right question here, we simply want it to crash less and kill less people

The headline of the article brings in ethics.

I thought the same thing ...

Simulations are daily parts of life, in research and development from real-world simulations in research (e.g., climate) and industry (often in spreadsheets); to the theory of gravity (gravity simulated with mathematics) - and every other theory of science, social science, and humanities; to develping your iPhone app on your laptop or just reading the train schedule or using a mapping program.

So what is the difference? Those simulations were built from reality. The Theory of Gravity was built from and confirmed with empirical observations, not from someone else's simulation of gravity! That essential foundation of science, reality (it's not science otherwise), is what is missing.

Also, we already have the problem of our biases and preconceived notions infecting training data, and AI becoming a simulation of that rather than reality. By then training on 'simulated data' (yikes!), we seem to create more of a loop.

If machine learning (ML) is trained on human behavior it can never be better (for some measure of better) than people are. So if racism is widespread, it will influence decisions trained into a ML algorithm. That raises the question, can we train an ML algorithm to make the decisions we _want_ rather than base its decision making on what people currently do. Training on generated data might be one way to do that, to build in implicit biases because we want more fairness in decision making than people currently exhibit. But then who gets to choose what those "goal" biases are? Someone could add a bias that improves life for people of male gender and worsens it for everyone else for example. This seems like a really important problem that has no clear answer. It's very much related to electing politicians where the choice of politician is also a choice of future goals. We don't know of any objective way to always decide what goals are best for everyone (voting certainly is not objective since many voters are not aware of all possible policy/goal implications and can be tricked into voting against their own interests). Yet it seems certain that researchers are working to introduce selective biases into algorithms, if just to adjust for known biases that are problematic. As an opaque input into an opaque algorithm, biases become invisible in deployments and could become very difficult to reverse or fix later. Even with continuous learning, if people's behavior is the input, the result will never be better than people are currently, which can create problems for the future. Some kind of intentional bias aimed towards goals seems necessary, yet also seems very dangerous since it can introduce biases decided by a very small set of people.

Excellent points. It's human beings all the way down.

The fact that you can compute y = f(x) doesn't imply you already know x = f⁻¹(y).

If you want an AI that can recognize cats, and train it with computer generated pictures of cats, you may wind up with an AI that only recognizes cat pictures from that particular generator.

This tradeoff is known as synthetic domain shift and is still an active area of research. https://paperswithcode.com/task/synthetic-to-real-translatio...

No. We had realistic 3D graphics 20 years ago.

I'm not aware of any true 3D computer vision system that could reliably play those games (from just vision).

You can train an AI to do the inverse of the existing program (as is the case for the self-driving described in the article.) Take some input, generate output using the existing program, and then train the AI with the input/output reversed.

Not necessarily: https://en.wikipedia.org/wiki/Data_augmentation

at least in the case of reinforcement learning, no. just because you can simulate the problem doesn't mean you know how to optimally solve it - ex driving a car

An obvious counterexample. I was young in 2005 when Tesseract was open sourced. I wanted to use it to do something. But I decided to trial it out first by writing something in Notepad and then screenshotting it and trying. Synthetic data! But no, I didn’t know how to make the existing computer program turn image into text.

as a contrast to the other comments, you will still get biases from where the simulation differs from reality.

eg. if your simulated traffic lights dont blink at 60hz, the model trained on it wont know to handle it

Generating a realistic city and a self driving AI are two wildly different tasks

Even a simpler task like image classification such as: does the picture contain a lion. Imagine you have 3d model of a lion. You can render it from lots of different angles, lighting conditions, backgrounds, stretched out, curled up, etc. You know the ground truth classification on all renderings is that the picture contains a lion, but being able to generate images of lions is a very different task from recognizing lions in images.

The potential issue with using synthetic data to simulate the problem (like image classification) is that recognizing a lion in synthetic imagery and recognizing a lion in real imagery may also be very different tasks to a computer.

not that much actually because the image are handled at a lower resolution and with a smaller set of colors, to make the models train faster(a lot faster)

so they look pretty much the same to a computer vision algorithm

Reply on news.ycombinator.com