Hacker News Reader

Comment by weinzierl | Hacker News Reader

One could say, "This didn't age well." but I think the real point of "it can be hard to explain the difference between the easy and the virtually impossible" is only reinforced by an almost ironic twist that switched the hard and easy around. Who would have thought ten yeas ago?

View on HN

I thought it aged well. The problem really did become easy 5 years after it was posted

And it did so specifically because of the hard work of professional high-caliber research backed by huge amounts of industry funding. Munroe knew exactly what he was talking about.

We had decent enough image classification for this task a few years before this comic. While the sentiment is true the example in it had already been solved.

Well I saw plenty of commercial software failing at this kind of thing even a few years ago, when presented with real-world pictures. Seems to be solved now, though.

Here's the ImageNet recognition challenge results from 2011[1]. Bird is actually one of the easier categories they they call out, but still calling it 'solved' is a bit of an exaggeration.

The winning model is from Xerox Research Center Europe.

1. https://image-net.org/static_files/files/pascal_ilsvrc_2011....

It's closer to 50 years since it was first posted, though.

Its worth looking at the alt text of the xkcd itself: In the 60s, Marvin Minsky assigned a couple of undergrads to spend the summer programming a computer to use a camera to identify objects in a scene. He figured they'd have the problem solved by the end of the summer. Half a century later, we're still working on it.

It's a cute anecdote but of course reality is more boring. The assignment wasn't to solve human level vision in a summer, just to do quite constrained object recognition. But even that turned out to be quite hard. But Minksy wasn't that fully miscalibrated.

I think it’s worth reading the original assignment:

https://dspace.mit.edu/bitstream/handle/1721.1/6125/AIM-100....

If you want to argue that “likely objects” is weaker than Munroe intended, I think that’s a valid position but also that we’re certainly overthinking it.

It's specific-object identification, not object categorization. Also in front of a controlled background. They had a limited set of known objects. There was no generalization expected. It asked if it's looking at one specific ball or one specific hammer, not the general category of bird in the wild.

In fact, that scope was solved fairly fast, using techniques like Canny edge detection and Minkowski fractal dimension features, Hu moment features on Otsu thresholding etc.

I respectfully submit that this might be expecting too much precision from the alt text of a webcomic, xkcd.

The story is told basically in every single course on computer vision around the beginning.

I have higher standards for college courses. You have my blessing to criticize them for insufficient precision.

Even in 2014, it was apparent that you didn't need new research per se to identify a bird, only a bunch of bird training data.

https://en.wikipedia.org/wiki/DeepFace

personally i can't help but think that it was this very comic that kickstarted the AI revolution. if i remember correctly a few weeks/months later there was a paper from yahoo/flickr (?) about this problem (identifying birds) and from then on the whole thing exploded overnight (i mean, i know this isn't what really happened, but to me it looked like it).

It's more likely the reverse, that Randall Munroe was aware of this emerging field but also knew that it's hard.

edit: found the flickr link - https://code.flickr.net/2014/10/20/introducing-flickr-park-o...

Absolutely demonstrably not, the convnet paper was in 1998 http://vision.stanford.edu/cs598_spring07/papers/Lecun98.pdf & the Krichevsky/Sutskever/Hinton Convnets at scale on imagenet paper that broke everything was in 2012 (before the comic) https://proceedings.neurips.cc/paper_files/paper/2012/file/c...

LeCun already worked on essentially convnets in the late 80s, early 90s. And arguably Fukushima's Neocognitron in the 70s was a CNN, it just didnt use backprop. Btw Fukushima invented ReLU in the 60s.

I won't go full Schmidhuber, but the field didn't suddenly spring into existence with LeCun.

And in a broader way, it seems that people don't realize that the very inception of computing was intertwined with the goals of AI. It wasn't like people invented computers to plan rocket ballistic paths and manage bank transactions and make spreadsheets and then decades later someone realized that this thing could also do AI stuff.

In the first half of the 20th century, computing pioneers were all about trying to imitate/model human reasoning. Which is an endeavor grown out of computing theory and logic, Turing's story is entangled with Gödel and Hilbert. And centuries before, Leibniz equated rational human reasoning with computation.

And the complicated logic circuits built with electronics resembled nerve cell activity, most prominently realized by McCulloch and Pitts in the 1940s.

One of the key things that started the AI revolution was realizing that GPUs were really useful for Machine Learning, which happened a 2 years before the XKCD, gradually starting around 2010, but really getting popular in 2012 (12 years ago today) with "AlexNet" using multiple GPUs: "ImageNet Classification with Deep Convolutional Neural Networks".

https://www.turingpost.com/p/cvhistory6

If you look at the quantity of computation/energy spend for each task it's still true that the problem that looks simple to people is the one that is harder to solve.

I like this ironic take that even the experts's expectations got upended with new tech.

I don't think anyone with a passing understanding of computer vision seriously thought this wouldn't get solved at some point, just that is happened years/decades sooner than expected.

s/sooner/later/: https://news.ycombinator.com/item?id=41655017

But did it upended? Ponytail expected that they can solve the problem in about five years with a dedicated team. Obviously Ponytail is not a real person, so she did not get a real team and five year budget. But real people in the real world got it, done the work, and now this problem is solved and the solution is fielded.

You might split hairs on that it wasn’t five years but three or six depending on where exactly you put the threshold, but it seems to be roughly in the right ballpark of how it went in reality. At least close enough that I wouldn’t call the expectations upended.

She came into a couple billion extra dollars and finished early.

He wasn't upended, in fact he states:

> Understanding what kind of tasks LLMs can and cannot reliably solve remains incredibly difficult and unintuitive.

His main point remains.

> One could say, "This didn't age well."

It has aged well, particularly because Randall explains that LLMs have made this even worse.

His main point is this (I'm quoting him):

  The key idea still very much stands though. Understanding the difference between easy and hard challenges in software development continues to require an enormous depth of experience.

  I'd argue that LLMs have made this even worse.

  Understanding what kind of tasks LLMs can and cannot reliably solve remains incredibly difficult and unintuitive.

Those quotes are not by Randall. They are by Simon.

You are quoting Simon Wilson (his blog) not Randall Munroe.

Ah, thanks. I had a brain fart and missed this.

But still, the point of the original xkcd wasn't "computer vision" right? As the caption explains, "in CS, it can be hard to explain the difference between the easy and the virtually impossible".

The only thing that hasn't aged well is the difficulty of identifying birds in photos. The adage remains valid in general.

Reply on news.ycombinator.com