Hacker News Reader

Comment by skywhopper | Hacker News Reader

Part of the problem is... games have explicitly defined rules, start and end points, boundaries, and discrete "win" and "loss" states (and sometimes "draw"). If the game itself (ie, all the rules including the ability to judge "win", "lose", or "draw") can be easily represented in a simple computer program, we shouldn't be surprised that a complex computer program can master the game.

The real world is not a finite problem with explicit rules, obvious boundaries, well-known start conditions, or any way to judge a specific situation as "win", "lose", or "draw". But, even if you want to argue that specific tasks can be broken down this way, you still have to be able to represent this subset of reality in the computer, before AI magic can even begin to work on the problem.

View on HN

Precisely so. You can, given enough time, brute force your way to victory in any game of perfect information. This is not the case with reality, as far as we understand it so far. Thus, from a theoretical perspective, the class of problems that contains all games is strictly easier than the class of problems that exist in a less artificially constrained environment.

Perfect information is not the case when the AI is reading pixel colors off the screen and pressing keys on the keyboard. This is very like the real-world situation.

As for goals, they are indeed better defined in games. But we can create artificial goals attached to the real life: - goal: GPS sensor reports specific position - goal: the camera reports a recognised object in a specific position - goal: a button was pressed (signal reaching the AI e.g. through the cloud)

You'll find pretty quickly that the exact techniques that worked so well in learning to play Atari games very often fail spectacularly when you have to introduce goal-steering.

Reinforcement learning turns out to be fantastically clever at finding really stupid solutions if you give it the tiniest opening to do so. You put a camera in a room to provide feedback for an agent to learn to move an object closer to the camera, and it will happily learn to knock the camera over.

To be fair to our AI brethren, this is true of Humans too. I watch people game metrics every single day, and many of them fail spectacularly when presented with real world problems outside of their experience.

Gaming metrics is the very soul of corporate life

Then which is real and which illusion?

Good question. Sometimes, the metrics must be gamed to make real progress. (As opposed to the more usual gaming them for personal gain.)

Between these two, some self serving bias and delusion, mission statements etc.. what is real?!

If they meet the spec, they're not stupid solutions, it's a stupid spec.

The problem is that we generally rely on human intelligence to fill in gaps in specs that are really not stupid for a human. AI will exploit gaps that a human would basically say aren't there until they see evidence to the contrary.

So we can say that's a bad spec if we like, but what's the answer that leads to? I don't need an AI if I can just declare all users must be really good programmers spending their days writing unambiguous specs. That wasn't really the goal of the AI though.

Unfortunately, when gaming metrics has real world consequences (and this is of course the whole point of those metrics, assuming they're not gamed), then things like 2008 happens, and tens of thousands of people lose their homes.

I think you are misusing the term "perfect information." In this case, chess is a real world game where both players have perfect information. That is, neither player can keep a secret of where any of the pieces have moved.

So, the complexities of how the AI is taught to interact ultimately don't matter. It may have a lot of effort to parse the visual of the board to get the perfect information, but the game is defined as one of perfect information.

Wikipedia calls out that there does not seem to be consensus on what this term means for games with chance or concurrent play. https://en.wikipedia.org/wiki/Perfect_information

True. I think my point was more about games with non-trivial rules (or many degrees of freedom). For example going from chess/go to turn-based video games like civ, to starcraft. Usually it involves vastly more possibly positions in time and space.

Even then, it still reduces to a brute-forceable game with finite, explorable states; 'just' with an extra layer of (granted, quite interesting and technically impressive) parsing. We don't know what the case is for reality.

And yes, we can do our best to turn real life in to a game, but all such models leak pretty badly, and the leaks tend to cause much more fundamental instability.

I think the main problem is with simulating enough games/tries.

In the game, you can easily let the AI play against itself countless times. Much, much slower in real life.

Until we have good enough simulators/approximators of reality, AI can't learn fast. However, they can already learn driving in GTA, so who knows?

> However, they can already learn driving in GTA, so who knows?

Yeah, that sounds like it'll end well.

"AI plays Dwarf Fortress" is something I'd like to see.

How do you define a "win" condition in Dwarf Fortress to train the AI?

DF itself already computes the prosperity of your fortress for you. This score and the sheer number of (non-enemy) dwarfs are the way you progress through the game. So a reward function based on these two inputs, plus maybe something like "months since last major catastrophe" could work.

Dwarven Utilitarianism

"Avoid dwarven deaths"

Doesn't that get you dwarven overpopulation and sad dwarves?

"maximize happiness according to Marxist values. Allow workers to control and maximize the means of production"

No, because a lot of dwarves only have a large number of children because of the high mortality rate of infant dwarves. By decreasing the expected mortality of the baby dwarves, the dwarven families will have the opportunity to choose having less children based on their situation, turning the apparent exponential chart of population growth into a plateauing sigmoidal line.

Avoid Death doesn't mean reproduce.

Depends on whether it's the death of the society or the dwarves you're optimising for, I guess? I don't know the game well enough to say, but suicide could be the most reliable way to minimise death.

I prefer "maximise happiness" if only because the Repugnant Conclusion is more interesting than the VHEMT.

https://en.wikipedia.org/wiki/Mere_addition_paradox https://en.wikipedia.org/wiki/Voluntary_Human_Extinction_Mov...

You can aim for "maximize happiness" without assuming that maximized happiness is merely the addition of the happiness of each person.

Can you mention one or two alternatives? Summing something like log-happiness doesn't help (and really isn't much more than objecting for the sake of objecting.) Thresholds for dignity just raise the zero-bar, and inequality metrics argue _for_ the repugnant conclusion.

Maybe mean/median/minimum happiness? They don't intuitively appeal to me, but I'd like to hear an argument for them from someone who believes in them (as well as arguments for systems I haven't considered.)

Of course there are plenty of non-utilitarian systems to choose from, but I'm sure few their proponents would like to describe as maximising happiness :-)

You are retracing the arguments made by Rawls in Theory of Justice. The result he proposes is a maximin principle that gives (grossly simplified) based on the maximum of access to social goods that is compatible with similar access for everyone else (equality principle). Differences are to be favored if they advance the situation of the least well off and if they are based on contribution/merit (differences principle).

Most commonly, maximin is extended to leximin i. e. Looking at the next least welloff. There are traditions of critique both from communitarianism (Sandel et al.) and libertarianism (Nozick et al.), both worth looking into.

Maximizing the minimum value is a sound optimization method. It gets an overall sound system that doesn't get any obvious break point.

In terms of happiness, it makes sure that no one is left behind, and tends to produce a more homogeneous society than optimizing for the average.

Also, [1].

[1] https://www.smbc-comics.com/index.php?db=comics&id=2569

> doesn't get any obvious break point

Well, aside from the imperative to euthanise the least happy regardless of their absolute happiness, I guess :-).

And making most individual actions normally called moral or immoral difficult to justify or condemn except that they might indirectly affect the least happy person.

And then there's the weird idea that my local actions may have moral weight depending on the arbitrary existence of an unhappy person in a distant country.

All seems pretty bizarre to me.

Lose gloriously.

Therein lies the point. :)

It's a question of generalizability. Today's "AI" algorithms are intensely overfitted to their problem domains, even if they generalize well within those domains.

Yep, this is pretty much word for word what the article is laying out.

"Greetings, Professor Falken."

"Hello, Joshua."

"A strange game. The only winning move is not to play. How about a nice game of chess?"

The ”fux dat”, or stoic approach.

Real world is so damned complicated, full of various mini games. How about using a MMORPG as a naive start point?

What if closed systems are just that much different than open systems that it's not generalizable?

Closed systems don't have Black Swan possibilities. Black swans are sometimes super-linear, jolting results in directions and scale unimagined.

To answer briefly (and respectfully): because an MMORPG is a different type of system than reality. Comparing cars to buses.

I've been stuck in a minigame for the past couple of decades.

And all too often most of the players will try to bend/break the rules in their favor.

Because that is indeed a very effective approach. In a game where rules can be broken or warped to suit your goals, doing so is a smart move. Just like in life, where rules don't actually exist.

Limitations of AI reminds me of "Moravec's paradox" https://en.wikipedia.org/wiki/Moravec%27s_paradox As Moravec says, "it is comparatively easy to make computers exhibit adult level performance on intelligence tests or playing checkers, and difficult or impossible to give them the skills of a one-year-old when it comes to perception and mobility."

unless AI figures out the way to win life is by figuring out how to survive

I would also bet nobody publishes research on games AI didn't perform well at.

Reply on news.ycombinator.com