Hacker News Reader

Comment by not_a_moth | Hacker News Reader

The wild Zuckerberg quote aside, I don't believe many people expect this current wave of AI to approach general intelligence. I would be surprised if we weren't all mostly in agreement with the author.

View on HN

In my experience, domain practitioners who spend our days elbow-deep in tensors are well aware that what we call AI right now has nothing to do with AGI.

That knowledge does not seem to generalize well, though. Even technically literate people who aren't in the trenches seem to have frankly insane ideas about AGI. This phenomenon is not helped by armchair philosophy, the frothy chatter of the tech-adjunct world, or the naive idea that "well, transistors are easy, so brains should be too!" that is all too common in people who haven't stuffed electrodes into brains and then spent weeks figuring out what the hell could be going on in there.

You'd have a pretty hard time figuring out what a CPU does if all you could do is poking a small number of needles into it, cutting it into slices to study under a microscope and measuring which parts get hot while you use it.

https://journals.plos.org/ploscompbiol/article?id=10.1371/jo...

true fact.

The thing is that progress is so fast in some areas of ML, you dont really know when something could emerge. Who knows, maybe with enough training data and a deep enough network with the right supervision, something extraordinary pops out.

Google google bert nlp. Google recently released an NLP algorithm that basically beats all predecessors, but the algorithm itself is extremely simple. Just lots of data and compute, no telling what will happen in ten years when the amount of data has exploded and GPUs are cheap.

This mentality is exactly what hprotagonist is talking about. All signs currently point to AGI being far away - at least several breakthroughs in currently unknown unknowns away, in fact.

If the most you can say about the likelihood of something happening is that, "there's no telling what can happen in the next ten years!", then the only informed forecast is that it won't happen. We can likewise say we don't know if an extinction level event will happen due to a meteor (that we couldn't detect for some reason) hitting us in the next decade.

Every time someone makes a sobering statement grounded in reality about artificial intelligence, another comment pops up which reduces to, "but who knows!" It doesn't mean anything.

bert provides incremental improvements on tasks that don't really stress-test contextualized world knowledge in linguistic tasks — contrast with Winograd schemas (SOTA: https://arxiv.org/pdf/1806.02847.pdf). Corpus data and compute don't magically solve everything.

The paper you linked to shows how a simple DL brute force approach (large model trained on lots of data) "successfully discovers important features of the context that decide the correct answer, indicating a good grasp of commonsense knowledge."

Which kinda goes contrary to your last sentence. Go figure.

My point is that while the DL + large corpus approach is the best available, it's still way, way lower than human performance (Tables 4 and 5).

Sure, but we are talking about how fast it's progressing. Given that NNs for NLP field pretty much started with Bengio's paper in 2003, I'd say it's accelerating at an amazing rate.

But NNs for language processing have been around much longer than that, e.g., Elman, 1990: https://crl.ucsd.edu/~elman/Papers/fsit.pdf.

Yeah, Bert's great, but it has the Google Paper problem in that nobody else can reproduce the results because nobody else has the compute to train the dang thing. It also doesn't solve the fundamental issues - it scores a few percent higher on some standardized sets, but it gets wrong answers that are just as explainable as any other system - shrug-emoji. ‍️‍️

A couple months ago, I was on an NPR show trying very hard to refocus the discussion on real near-term threats caused by AI: possible job loss as well as enabling authoritarianism, predictive policing, and quashing of dissent. They wanted to talk about AI coming to get us all. (Evil) AGI seems to be a more exciting topic.

"...enabling authoritarianism, predictive policing, and quashing of dissent" sounds adequately evil to me, but - you know - I get what you're saying.

Can you recommend any good reading about these threats?

Do you have a link to that show? Seems like a good one to hear.

I think this is the show he was referring to: http://www.wxxinews.org/post/connections-did-2001-space-odys...

I think pop-sci fuels overzealous misinformation in even the tech-literate. A lot of innate, unconscious cognition is underestimated despite its complexity. E.g. consider the ability of humans and most other species to approach, navigate, and understand arbitrary space and its objects - you can't do them same with AI without extensive training and covering corner cases. And even then, a flying insect such as a dragonfly will outperform your model.

Well, quite a few C suite folks seem not only to believe it, but to be convinced that anyone who disagrees is effectively committing sabotage. This is because every vendor is telling them this is so, and offering to sell it to them. This is dreadfully damaging, and a time of reckoning will come, but don't expect justice.

I see plenty of ML researchers in person and online who think we are getting closer to real AI and it just requires some evolutionary improvements over current techniques.

Sure, but current techniques are a long ways from that.

Okay, fine. I wish more people talked about Jeff Hawkins work at Numenta or cortical.io or sparse distributed representations. But in particular, Jeff's vision of understanding the neocortex is really cool stuff. If I could cast off the shackles of having to work for my money like a filthy commoner I think I would lean towards learning a more biology inspired approach like Numenta is doing.

Reply on news.ycombinator.com