Hacker News Reader

Comment by jasode | Hacker News Reader

Well, reading the PDF was anticlimactic. Why? Because page 2 of the PDF describes the 12 games that were used. They were games that test memorization, puzzle completion, and pattern matching.

In other words, the "commercial video games" used in their research were "brain games" and not Pac-Man, Mario Bros, and Call of Duty, etc.

If you've ever taken a real IQ test where the proxy shows you cards to match or pictures with "missing" data, the 12 video games they used are very similar in spirit. It's not surprising that there is high correlation between the scores of those particular types of video games and real IQ tests.

However, that's not to say there is zero correlation between blockbuster video games like Call of Duty and IQ. Maybe another study will pursue that.

View on HN

Thanks for the warning. I was really hoping to see Halo or Quake or something, because in my experience practicing these games in a casual-competitive way, I hit a performance ceiling. In high school I played Halo nearly every day, and a bright friend of mine, who didn't even own the game, was a surprisingly good player for his limited experience.

On a similar note, I once read that reaction time and IQ are correlated (not sure how strongly), which is interesting because you might expect motor functions like that to be orthogonal to higher-order cognitive abilities.

Speed is good, in many contexts. Say you've two people, and person A has overall cognitive speed 25% greater than the other.

That doesn't just help with Jeopardy!, it gives you 25% more time to think in normal conversations, on SAT tests, while playing video games, at work, etc.

Even a 10% slower speed differential from a baseline human is a big, big disadvantage.

Reaction time is good in general, but rather useless in the traditional IQ tests, i.e. culture-neutral multiple choice pattern-based questions. If you are 25% faster than your neighbour it gives you the benefit of the additional 25% of the reaction time, not total answering time, which is usually much larger in the context of IQ tests. Or nearly any test, for that matter.

Oh, sorry if I wasn't clear. Reaction time is strongly correlated with the overall speed of brain function. So it applies to other brain operations too.

Normal IQ tests should also contain a computerized part where you have to react and solve problems on a computer with a time limit. Last time I did that, they still used Win2000, but I can only imagine this increased.

> Even a 10% slower speed differential from a baseline human is a big, big disadvantage.

This would make sense if all thinking is equal and only the rate of thinking varies. In practice, the quality of thought processes seems much more important than the the rate at which such processes are carried out.

For example, attempting to assess "cognitive speed" can be very dangerous in interviews. It seems like such a promising metric. A candidate who answers questions 10% faster than average will generally be much more impressive than a candidate who answers questions 10% slower than average, and it's tempting to think person A will be 22% more productive. Of course, over time, it may turn out that he occasionally provides more bad solutions to problems, or can only solve superficial problems, or solves the wrong problems. And you realize the person who consistently solves problems correctly (albeit at a slower pace) is a better choice than the person who introduces new problems as quickly as he solves existing problems.

It could always be argued that person B is actually thinking faster, but thinking through the problem much more carefully, causing him to verbalize his answer later. Or perhaps both people are thinking at the same speed, but person B has simply thought more. Maybe person B is actually thinking slower but more effectively. From an outsider's perspective, we can't know what's happening under the hood, which makes "cognitive speed" a weak metric for judging effectiveness in practice.

This. You want some easy proof? Watch a video of Steve Jobs answering questions, like the WWDC 1997 Q&A video. Time how long it takes him before he starts speaking after the questions are asked, and note the quality of the answer.

Yes, but presumably in those long pauses, a faster thinker would have time to consider more possible replies, make predictions on how the audience would receive them, and generally make more adjustments and improvements than a slower thinker.

In other words, faster cognition would probably help you formulate a slow, considered reply just as much as it would help you answer more quickly. That might be especially helpful in a public speech to a group asking unplanned questions.

I agree with freyr, though, that quality of cognition/ideation matters more than speed, and doubt that 'thinking speed' is necessarily correlated to better thought output.

Like IQ itself, faster basic cognition probably means something, but I don't think anybody knows what it means or how (or even if) it relates to "intelligence" (whatever that means).

In my personal experience, I have seen more great ideas and solutions come from "weird thinkers" than "fast thinkers". I realize that sentence does reduce to the cliche, "Think outside the box, bro," but it seems true over the course of all that I have observed in my own life.

In that example, it may also simply have been that Jobs had been prepped (or prepped himself) thoroughly with answers to a lot of likely questions.

I don't put a lot of credence to the speed thing here. I know one senior executive who made a point of writing down question on cards when he was asked them live at an event. Which I always thought was a rather clever approach to 1.) Make sure he understood the question correctly and 2.) To give himself some time to formulate the best response.

In practice, the quality of thought processes seems much more important than the the rate at which such processes are carried out.

Of course the quality of thought matters.

But if you have two individuals that have the same quality, but different speeds, the faster one will seem smarter.

Think of a fast flowing conversation. A slower thinker is going to miss references and associations that a faster one will get. So the faster thinker will learn more from the conversation, and be able to participate more. This has many benefits.

Well, you could think about in in the sense that quick mental reaction time is necessary but not sufficient for a quick physical reaction time.

Processing speed is only one part of IQ tests. You can see a pattern at higher IQs where the processing speed is lower than normal, but other parts nearly max out. People with jobs where you always have to be meticulous (like software), can show this pattern. Also processing speed is weighted lower in IQ tests vs things like pattern matching or solving.

A possible explanation for the correlation with reaction time is brain damage. If you are exposed to more lead as a child, that will damage both your motor cortex and higher cognitive functions, even if those aren't otherwise connected. Same mutational load. If you got a random mutation that slightly affects your neurons slightly less efficient (everyone carries such mutations, but our genetics is redundant enough to handle it.)

But I wouldn't expect a perfectly "normal", undamaged human to be a genius, but they would have superior reflexes. And there are tons of animals that are dumb, but have amazing reflexes.

In an FPS like Halo the relevance of skill sets is much harder to measure, but I think it's a delicate balance.

If I were forced to choose, I'd say motor skills and reaction time combined with spacial intelligence are the key drivers of success. I have yet to see an FPS professional that doesn't have tremendous shot accuracy and agility, relying only on their strategic skills. But I have seen some with great shots and spacial awareness make many questionable decisions. But my exposure is limited and my perception is biased.

I'd guess general intelligence shares relevance almost equally as it affects weapon utilization, detection avoidance, enemy action prediction, strategic positioning, verbal skills (team games), random trickery, etc.

However, that's not to say there is zero correlation between blockbuster video games like Call of Duty and IQ. Maybe another study will pursue that.

There's a lot to learn and practice to achieve mastery in those games. And experience in other similar video games will strongly correlate to how quick and high the skill in one particular (popular) game will be achieved.

I was watching a video a while back with these two guys playing QuakeIII. They both knew the map intimately, and kept track of when power-ups would respawn based on when they were picked up. Then they based their strategy on what resources they had, and what resources their opponent had and where the other was likely to go next.

So yeah, we're talking hundreds, if not thousands of hours devoted to mastery in this game. Against an opponent like that, two Quake newbies would score about the same (dying almost instantly), even if one of the newbies was otherwise very smart, and the other very dumb.

My god, Did you just bring up some memories. In the 90s my buddy Morgan was head of the game lab at Intel. He was a mast of quake, but another factor was that in the 90s broadband was much more scarce. So imagine playing quake on the fastest machines Intel made on the fastest corp network likely available at the time AND being a master of quake. That was Morgan. And he got paid to do it.

I fucking hated playing quake against him. I was ok, but he was amazing. He also would taunt you.

However, my best gaming memories were when we would basically live at Intel for days at a time playing UO across six diff machines simultaneously on that same corp net. No lag. Thanks for triggering that memory.

Quake (actually Quake III) is still alive today in the form of http://quakelive.com which is free-to-play.

I think the time is ripe for Arena FPS to make a return to the mainstream (with updated graphics). Toxikk which is still Early-access seems very promising in this regard (https://www.youtube.com/watch?v=B28oQvaL7Mg).

I think CS:GO would make for a particularly interesting study. It can probably be studied just by looking at logs. You would have to consider network lag when measuring performance.

I like it because maps like Dust 2 are so well known by competitive players. The success of an individual depends on sophisticated social factors around cooperation or divergence from team members. Strong teams nearly always outperform teams with strong individuals but poor cohesion in my experience.

Intelligence could be measured across a variety of types/dimensions including motor skills/reaction time, individual tactical performance and team cohesion.

Is anybody doing work like this? Is the data available from Valve or independent server admins?

I was hoping for at least a mainstream game that was mainly about problem solving, like Portal.

Ok, we (belatedly) changed "video games" to "puzzle games" in the title here.

I had to go through an IQ test and a couple of brain teasers for a job application -- I wasn't hired even though I scored well in the IQ test and answered correctly 4 out of 5 of the brain teasers, they were expecting an outstanding result (though they would pay you about $20/hour).

I came to conclusion that IQ tests and brain teasers are just bullshit (in the context above, that is). You usually expect an exceptional result of it, because really, in day to day life you usually complete much more challenges and give good solutions than, lets say, 90% of the population.

You are not the only one criticizing IQ tests. Even Theodor Adorno had his problems with those tests. Some even claim that the whole IQ research is not very scientific at all. An IQ test for example is considered good if there is a Gaussian distribution present in the results it produces when applied to a large set of people. However, we still don't know if human intelligence is actually Gaussian distributed. The only proof you can find are such IQ tests, which actually are designed with this very pattern in mind. It's like asking a question with the answer already in mind.

I guess with the creation of phrases like "emotional IQ" and "cognitive IQ" even the research expressed its doubt that those tests can truly hold their promises. Freerk Huisken might be right in the end. He argued that it's a logical problem when one is trying to measure intelligence in artificial test as they can not retrieve the true capabilities of a human. Intelligence is a damn complicated subject and although we humans love to measure and categorize everything, we should consider that IQ tests are far from being comparable. Truth is, we still do not understand what intelligence actually is :)

This is actually a nice topic to discuss with philosophers.

Speaking as someone whose exposure to intelligence testing comes second-hand via a psychology graduate student, my understanding is that the quality of an IQ test is just based on how well it correlates with other IQ tests. The distribution always looks normal because they normalize it. https://en.wikipedia.org/wiki/G_factor_%28psychometrics%29#P...

We may not understand what intelligence is philosophically, but in the field of psychology, IQ/general intelligence/g factor is a statistical construct that isn't directly measurable but can be inferred by its correlation with measurable factors. https://en.wikipedia.org/wiki/Factor_analysis

The pro IQ argument I hear is that nothing else predicts success in life as well.

One must wonder how much that has to do with our individualist society that tries to assign rewards to individuals instead of emphasizing group cooperation and communal advancement. In a society that freely shares, would IQ be as powerful a predictor?

An intelligence test for groups of people would be a very interesting idea.

Wow, this is such an out-of-left field but excellent idea. I personally spent most of my formative years in a culture that wasn't as individualistic as american culture so the IQ test having such an implicit individualistic bias to it is fascinating to me.

Right, IMHO QI test (and the brain teasers, anyways) could be used, but as a small part of a more complex process -- like, what about giving the applicant a real problem to work on? Unless you don't have enough time to make a hire, but in this case, well, good luck then.

When we give someone an IQ test we give them a made up set of problems to solve. We hope that if someone can solve these made up problems they will be better at solving problems that real life throws at them.

When we look at the research we find that if we know someone's score on an IQ test(how well they solve made up problems) we find they tend to do a little better(on average) at life then people who aren't as good at solving made-up problems. This is not meant to be the full measure of a man, or supposed to tell us the answer to what intelligence is.(This is probably a question a bunch of tenured philosophers could argue about until the end of civilization :) ).

I suggest reading about the subject. http://www.udel.edu/educ/gottfredson/reprints/1997whygmatter...

I have more interesting subjects to read about, I will let the experts tell me their conclusion (when they get one.)

Reply on news.ycombinator.com