Hacker News Reader

Comment by t_mann | Hacker News Reader

It's not just the je ne sais quoi performance aspect that I'd be concerned about, it's something I'd call Wii-Sports-ification: the first time you try out movement-controlled games like they made the Wii console popular, you'll likely put a lot of effort into the movements as if you were wielding a real tennis racket or a golf club. After a while, you realize that the motion sensor is much more coarse, and you do much more efficient, bastardized versions of the original movements. While I acknowledge that this system is supposed to be much more sensitive, I'd expect a similar effect to be happening once athletes get their hands on those machines in training: they'll figure out how to best trigger the detection of each move, and how much flexing of the toes, bending in the back and overall effort is just enough for the machine, and it's going to change the look and feel of the sport.

View on HN

they'll figure out how to best trigger the detection of each move, and how much flexing of the toes, bending in the back and overall effort is just enough for the machine, and it's going to change the look and feel of the sport.

I remember watching Olympic diving years ago and the commentator was making a big deal out of the fact that the divers were spreading their knees apart (apparently a flaw) but that the judges weren't picking it up because their viewing angle on the dives didn't allow them to see it. So the divers were taking advantage of a literal blindspot in the human judging to make their dives easier to perform.

So really, changing to AI judging is just going to change what the competitors optimize for in an arbitrary way. It's not going to fundamentally change things.

If anything, having computers do the judging with cameras setup from all directions with human judges available to override in particularly suspect rulings is more fair to everyone and everything involved: The players, the spectators, the judges, and the game.

It will take away most of the fun(?) of arguing a judge's ruling and gaming them, but I will fundamentally always support making games fairer and objective'er.

Right, we’re heading to a world where everything is “perfect”.

> more fair to everyone and everything involved

I hope so. But I suspect a racial bias of some kind will be uncovered.

To summarize, before they were doing stuff that a human judge will notice.

After, they will do stuff that the "AI" will notice.

Goes for any physical sport that needs human judges instead of something measurable, like running speed, for their competitions.

Pretty sure if you do competitional martial arts you train a subset of techniques that will be very visible and score you points as well.

Yeah but the judges are human and are typically different humans at each event with different opinions and abilities to notice subtlties. There is no one standard human judge to abuse but with AI there is a single standard.

I wonder if having some intrinsic randomness to the AI model would help to mitigate this problem? So that each AI has a subtly different perception of the move, and in order to satisfy every possible judge, the gymnast would need to complete the move as comprehensively as possible.

> I wonder if having some intrinsic randomness to the AI model would help to mitigate this problem?

This would result in identical performances resulting in different scores with the difference being artificially arbitrary.

Which for two identical performances - an impossibility - would be an issue, but for two different performances would still yield different scores.

So just like human judges?

Sure, you can reduce all of humanity to a random number generator, if you want

Maybe a slightly more generous take isn't that humans are random, but their value systems have some sort of distribution. I'm unfamiliar with gymnastics scoring, but judges may have different perspectives on what matters most if you take a sport like boxing that applies subjective scoring. One judge may value footwork and the jab more, while another weights power punches more, while another is swayed by seeing a competitor who is bloodier by the end of the fight despite having a more technically proficient fight.

I was thinking you’d be consistent within a competition, but different between competitions, so Ana, Betty and Carola face the same judges when competing in the olympics final, but their performance cannot directly be compared to Diana, Edith and Frida who completed at the world gymnastics championship.

The "AI" already hallucinates doesn't it?

Exactly. Right now, the best an athlete can do is to just give it their all, and I guess for a lot of people that would be what gymnastics is about. The potential for optimization with such machines is just on another level. You can re-run others' routines to see how you'd have scored, you can do thousands of reps to test the limits of the algorithm, you can code ragdoll simulators to find pockets of optimization,... Gymnasts will be discussing software glitches, version differences, how to 'break the algorithm'. Not even saying that's a bad thing, maybe it'll do for gymnastics what AI did for chess, open it up to a wider audience. Just saying that it could lead to a very different gymnastics scene, that will maybe look a bit more like eg the speed running communities for video games.

It's not like chess. Chess has clear, simple, inhuman rules. Gymnastics doesn't.

Just an example of a sport that got quite reshaped by AI, but not necessarily in a bad way. The differences between the sports could hardly be greater, and the way they'd be impacted by computers would also likely be quite different.

Apparently Go has also been reshaped:

Humans have improved at Go since AIs became best in the world

https://www.newscientist.com/article/2364137-humans-have-imp...

Would be cool to see that how well you are doing bar on chess.com (on the left) continuously change during a gymnastics routine.

I don‘t know. In some gymnastics people land cleanly by absorbing the shock with their kneews, instead of naturally bending them, for aesthetic reasons. I guess it depends on what we exactly mean by inhuman.

Gymnastics, Olympics, is meant to be a celebration of the human body and spirit. The micro-optimizations ruin it, because they break the alignment between casual participants and the best.

Regular people exercising should be inspired and uplifted by the elites, but it doesn't work when they're playing a different game entirely at the top.

So is chess tbh. Even excluding computer chess. You can play it by ear at an amateur level, but if you want to compete you have to memorize those openings. Which sucks all the joy out of the game.

Maybe everything has an amateur level that's just hard enough to be challenging, but still easy enough to be played primarily by ear.

Maybe that's most people's careers.

U really gonna call me out like that in front of all these people

Is it established that an AI judge would judge everyone the same? That it would be impartial?

This is a concern I also have. Perhaps the AI underfits/overfits and then an unusually-sized athlete comes along and the AI judges very poorly.

As usual, more proper (and entirely valid) concern is whether it would be biased less/in ways we prefer compared to humans.

>There is no one standard human judge to abuse but with AI there is a single standard.

Having one and only one judging standard is a good thing.

Imagine we create an ai box and say that the box decides the results. It has been trained to judge those gymnastic acrobatics and is decision is final.

What will inevitably happen is a guy will come to the competition with a weird pattern on his t-shirt and get a perfect score without preforming any of the exercise.

The competitors also choose their judges, in aggregate. This is a bad idea to change.

A totally different but related example is pitch framing in baseball.

Basically catchers will try to catch a pitch using a motion that slightly tricks the umpires eyes into calling a marginal ball a strike.

It does have a statistically significant effect and since it has become widespread catchers are judged on how good they are at pitch framing since that is a small but significant part of the value to help the team win.

That whole concept goes out the window if MLB ever changes to just using cameras to call balls and strikes!

Exactly, its basically a variation of Goodhart's Law - “when a measure becomes a target, it ceases to be a good measure.”

Not really. It depends on the quality of the measure and any underlying assumptions. If the target is knowing who is the fastest then a simple stopwatch (measure) is 100% aligned with the target. So the measure becoming the target is completely fine.

The problem comes when one (or multiple) measure(s) are an approximation of some ideal measure for the target. For example the famous example [1] of the measuring rat tails to determine killed rats (target). The faulty assumption here was that the only way to hand in a rat's tail is by killing them.

[1] https://en.wikipedia.org/wiki/Great_Hanoi_Rat_Massacre

Unfortunately not true. It is not a single metric.

we have extra rules on top: No doping, separation of genders,...

> Not really. It depends on the quality of the measure and any underlying assumptions. If the target is knowing who is the fastest then a simple stopwatch (measure) is 100% aligned with the target. So the measure becoming the target is completely fine.

I get your point but with this case the measure is the target, so at no point does the "measurement become the target", it's aligned from the start. But then again, this is only true if your measure is of people performing the task you actually care about. If you want to know the fastest person and you measure over 100m you may end up with Bolt, which is great if you need "the fastest over 100m in a sprinting setup" but not if you need the fastest over a full marathon.

In general, it's an extremely good first approximation - many measures result in poor targets, and care needs to be taken when turning a measure into a target. Even then it usually becomes a worse measure.

> The faulty assumption here was that the only way to hand in a rat's tail is by killing them.

More than that, it's an assumption that an increase in rat tails equals a decrease in live rats. Even if you kill a rat to get a rat tail, that doesn't necessarily hold true. It rewards those that breed 5 rats, release 4 and hand in the tail of 1. Killed rats would have still been a bad target. The true target was fewer live rats. Dead rats can be a reasonable measure, but as a target it has very risky outcomes.

>Not really. It depends on the quality of the measure and any underlying assumptions. If the target is knowing who is the fastest then a simple stopwatch (measure) is 100% aligned with the target. So the measure becoming the target is completely fine.

I am thinking of how runners all lean forward just before the finish line. This must slow their overall speed but gets a part of their body to the measurement datum slightly quicker.

I admit its not a huge deviation between the measurement and the target but still it is an attempt to affect the measurement by means other than running fastest

I shouldn't be surprised given what website I'm on, but it's crazy that people will litigate the finer points of replacing human judgement of human arts rather than reject the premise outright.

Yes precisely. For reference my most AI/LLM pilled friends are also Desktop Linux enthusiasts, for whatever that's worth..

Why is this crazy? Should we toss the entire field of science and go back to judging reality based on our feeling and perceptions?

Gymnastics is not a science. Judging "good" sports is purely an aesthetic choice.

There is a science to gymnastics. However the judging is only minimally based on the science and mostly subjective.

If something is wrong, there should be an articulable reason Find it.

That gets you people doing weird tricks to game the starting block and the finish line, which are artificial distractions from "fast running".

Only accepting entire rat bodies wouldn't necessarily work either; see the "cobra effect" that supposedly happened in colonial India: https://en.wikipedia.org/wiki/Perverse_incentive

There's a target of 9.8 seconds (or whatever) for the 100m sprint, there's a bunch of rules around things like windspeed, but it's still a good measure to find the "fastest man alive" despite being a target.

I think you mean “best shoe”

You could wear the best most advanced shoes, and I bet you still couldn’t beat Usain Bolt wearing $10 sneakers.

Two years post-retirement, wearing sweatpants and sneakers, he (albeit unofficially) tied the fastest 40-yard dash time at the NFL Combine: https://bleacherreport.com/articles/2818947-watch-usain-bolt...

Isn't the measure already the target?

Another suitable maxim here is "We shape our tools and then our tools shape us"

I'd be concerned about, it's something I'd call Wii-Sports-ification

AI might be able to radically change sports like fencing and HEMA. I could see the development of match judging AIs trained by experts. Anyone who has fenced knows the frustration of the fallibility of human judges.

It's also possible that AI judging could benefit martial arts, bringing them closer to their practical roots, by counteracting the rule-gaming competition rules. ("Sport-ification" of martial arts.) Instead of allowing just any touch to count as a point, only hits with enough body structure to allow damaging force could be allowed.

Many HEMA competitions try to implement this, but expert calibrated judging AIs would make this an order of magnitude better.

I was going to mention fencing. Thats a sport where there's a massive amount of judgment required from refs which is fairly opaque to non-specialists.

At least with an AI-based system, it's presumably consistent across performances & devices and can be used during training. While not "objective" per se, it's certainly better than an human judge that is none of these things.

The concept of judging has always included some form of bias. There is no such thing as an impartial judge. AI won't change this, it will just include a different set of biases. I'd rather be judged by a human than a machine.

>it will just include a different set of biases

One that many people who would admit the possibility in a human judge will likely insist cannot be present.

> Consistent across performances and devices

LOL! I think the assumption that an ML system would be robust across these contexts is not well founded. Things as innocuous as the lighting, background, or placement of athletic tape could easily wreck a model like this.

The thing about human judges is that the modes of failure are pretty well known at this point; and human perception and visual processing is pretty robust to perturbation.

The people that play Wii sports are not the same people that play professional sports. While I agree AI will be used to optimize routines, I highly doubt it will digress to the "I can play Wii tennis while laying down and only flicking my wrist" level. Nor do I think the "AI" sensors being used are as crude as a cheap, consumer focused Wii remote released over 15 years ago.

> The people that play Wii sports are not the same people that play professional sports.

Right, professional athletes will go to much greater lengths to exploit a scoring sytem if they can get an edge from it. The AI sensors might be much better than a Wii, but that won't fix the core problem because they will be studied and gamed much more diligently.

I, for one, would quite like to see what sort of bizarre adversarial flailing technically counts as a perfect performance. If we can get one totally surreal Olympics out of it, I'd call it a win.

Watch Olympic Sabre fencing. There's all kinds of wild flailing that people do in that sport. All that really counts is that the hit registers. So bunny hopping towards your opponent flailing that sabre wildly is a perfectly valid tactic.

as a former gymnast, I'm going to say I find this concern super unlikely

Reply on news.ycombinator.com