Hacker News Reader

Comment by 6gvONxR4sf7o | Hacker News Reader

I'd bet people outside the field are going to be disillusioned before long as the marketing BS falls apart. Even years ago, I was already getting questions from friends that surprised me in how detached their perceptions are from the reality of state of the art ML. Eventually, they'll realize that no, we're not building superhuman AI today or soon.

However, there's a ton of cool shit happening and tons of stuff already powering industry $$$. Some of this works because of mismatched expectations, like companies who are trying to sell magic. They will fail as their customers stop buying their products that fail spectacularly 2% of the time, but really can't fail 2% of the time.

But there's tons of room in the market for stuff that fails spectacularly 2% of the time. For that reason, I don't see a "real" reckoning coming. People's expectations are too high, sure. But the reality is still Good Enough™ for a lot of problems and getting better.

View on HN

>Some of this works because of mismatched expectations, like companies who are trying to sell magic.

We were asked to look into a company that had an, I kid you not, "AI-powered problem solving platform".

The suggestions it spit out in their demo for agriculture were things like "What if the earth was upside down", and I read it like "Maaaaan, what if, like, just imagine like, the earth wasn't like below, but it was like, above... Duuuuude. Just imagine.". i.e: you could get these recommendations with a few dollars worth of haschish.

The modern mechanical turk. Pay someone to be very stoned and to sit in a box made to look like a supercomputer. Have it occasionally print a prompt for the "AI" to reflect on.

Add a pitch deck and a blockchain and you have a startup with the right buzzword cloud.

Here Comes Another Bubble: https://youtube.com/watch?v=I6IQ_FOCE6I

But there's tons of room in the market for stuff that fails spectacularly 2% of the time.

That's the key insight. It's why the commercial successes of AI have been primarily in advertising, marketing, and investing.

That’s an interesting insight. What are the long term societal implications of enabling/inducing (Jevons paradox) activities which are not “important enough” to need 100% correctness?

> commercial successes of AI have been primarily in advertising, marketing, and investing.

Which is why people say that it has few useful applications. People don't care if those areas become more effective. Another important application is mass surveillance, which people would also argue isn't a good thing.

Investing?

Is there any evidence that any AI is better than index buy and hold?

I do not think there is.

The only thing consistently better than that is insider info, which is very common and quite reliable.

There was a saying before 2008: "How do you start an AI based FOREX trading fund? You pretend to be the russian prodigy developing the algo, while I do the insider trading".

Can I propose another insight? AI sometimes works and works spectacularly and does not fail in 2% of the cases, or even in 0.02%. Just look at iPhone photos. But that AI application require deep domain knowledge + deep AI (or, well, ML, whatever) knowledge. You can't simply throw sklearn at an arbitrary problem and expect to get great results.

IPhone photos do some weird shit a small portion of the time too. I've had mostly great experiences as well as the rare unrealistically colored outcomes or misplaced bokeh.

and translation, inference on text, inference on images, speech recognition, etc.

there's plenty of bullshit uses of AI out there, but i think it is silly to say there are only limited commercial successes.

Yeah, also MoE models like Wu Dao 2.0 that if combined with work at FB with hypothesis generation (causality testing) and Codex can give some interesing applications like autowriting programs to specification. There were also some progress with geometric deep learining which kinda unifies architectures AFAIK.

Disclaimer: not an expert

Amen. Translation's such a great example. I got some ingredients at a local market recently and used the google image translate function on it. I'm sure the translation was way off, and parts were entirely nonsensical, but it was enough for me to infer what the directions were. Tons of value in imperfect translation.

"AI" is not used at all in ad tech. (This is actually a problem when hiring people today. Nobody can do machine learning without some stupid big neural net framework.)

It is, but it isn't as widespread as some thinks it is. Old school techniques like decision trees perform better for almost every ad tech tasks and are much cheaper, so if you can't do those you are basically worthless in the field. But sometimes a neural network is actually useful.

This appears to be the pattern of AI development overall: booms and busts of hype and disillusionment that leave behind useful tools that most don’t consciously consider to be remnants of a previous AI bubble.

The same will happen with deep learning networks because if we're honest no serious researcher would really consider this "AI" either. It's just a big self-correcting linear algebra machine. Researchers will keep using that "AI" label for as long as it gets them research grants or selling products.

I wouldn't call it a "linear algebra machine" if the central point (of NNs) is that they are highly non-linear (the deeper the more non-linear, roughly)

When it works, it's no longer AI.

I know I'm paraphrasing someone smarter than me there.

(Edit: Larry Tesler maybe)

People believe that achieving X means the agent will be intelligent and can thus do Y, Z, A, B, C etc.

But then someone builds an agent that can do X, but it can't do anything else. So people don't view it as intelligent, since it doesn't do any of all the other things people associated with being able to do X.

So it has nothing to do with being used to computers solving new problems, it is that we now realize that solving those problems doesn't really require the level of intelligence we thought it would. Of course solving new problems is great, but it isn't nearly as exciting as being able to create intelligent agents.

Edit: And it isn't just hobbyists making this mistake, you see big AI researchers often make it as well and they have for a long time. You will find tons of articles of AI researchers saying something along the lines of "Now we solved X, therefore solving A, B, C, D, E, F is just around the corner!", but they basically never deliver on that. Intelligence was harder than they thought.

Exactly. People make the case that something along the lines of AGI is necessary and sufficient for some task, then someone comes along and solve it with something completely unlike AGI, proving that it's not a necessary condition. Or in more straightforward but less precise terms, solving this is AI and then it isn't.

So, our brains are just a little ecosystem of competing functions, and our pre-frontal cortex / prediction engine (the piece most AI researchers care about) is trained on species survival, but beholden to the "lizard brain."

I'm still on the bandwagon that AGI is already solved, and we just don't recognize it because we aren't as complex or magical as we like to pretend we are. We are just currently selected-for species survival, and not even in the present environment as the dominant species of the planet. That doesn't make us "Generally intelligent" any more than many other systems on the planet (including the computational ones). I'm not even convinced that "intelligence" is especially a thing.

If we're trying to replicate "human intelligence," then I think that's something else entirely, and would require removing a lot of capability from the AI that I don't think anybody would want to remove from AI, given that we have plenty of serviceable humans to do the work of human-intelligence style tasks.

No, when folks say AGI, they mean "cares about the things I care about, and is more accurate, but can speak in a way I understand." But how is a network of neurons going to care about what you care about if they see the world in a different way, or have been trained on solving a different set of problems than "living a life as a human?" It just doesn't make sense to expect, or even want, human intelligence from a synthetic intelligence.

Don't you think, though, that each boom and bust cycle leaves us closer to real accomplishments?

We now have protein folders and superhuman Go players -- that's new.

I agree that ML ("AI") is currently at the alchemy stage. And, guess what? A neural network isn't even Turing complete! [citation needed - correct me if I am wrong.] So ML can only compute SOME functions.

AGI, when it comes, and believe me, it will, will have ML as part of its structure, but only a small part.

I have no idea if each boom brings us closer. Scientific discovery is not exactly a linear process; we can’t observe before the fact if we’re getting closer or are on a dead end path, and that’s even assuming we can ever get there at all.

But it’s hard to deny that each boom doesn’t give us something useful. Neural Nets might not exactly make an AGI, but they do have uses.

I mean, each cycle does leave us with real accomplishments. If the question is whether it leaves us closer to AGI, then it's an open question. Like, when AGI happens, it will certainly trace its roots back to things happening in each period, but its roots will also go back to Gauss and Newton and co, so nobody knows whether it gets us closer in the way you probably mean.

ML is Turing complete in the sense that every computable function can be approximated to arbitrary precision by a 3-layer neural network. Classic result from the 90s, the paper in question (iirc) is titled something like “neural networks are universal approximators” Turing machines also can only approximate to arbitrary precision, so the computation models are equivalent.

Of course that depends on what you mean by "real accomplishments".

It seems to me that that deep nets have effectively maximized the potential of using gradient pursuit to model patterns. But if you remove gradients from your data, or shrink your data down to tens of samples, or shift the problem to logic, or need to use functions that aren't convex or differentiable, deep nets run smack into a wall.

Luckily human perception makes extensive use of gradients, as does most search, so problems in those arenas have been unsurprisingly amenable to solution using deep nets (vision, speech, game play, etc). But many of the problems that remain untouched by DL, like human cognition, are NOT driven by gradients. Will deep nets eventually fill that void? I doubt it. You can convert only so many problems with big data into gradients to pursue them efficiently with DL before that transformation trick runs out of steam.

Personally I think deep net language modeling is one of those areas, and soon we'll encounter the limit of their generalizable contextual phrase association. Then because deep nets are so difficult to selectively revise or extend the specifics that they have learned, the vanguard of ever more complex deep nets (transformers) will eventually sink beneath their own weight, taking the last best hope for DL-based general AI with them.

> But many of the problems that remain untouched by DL, like human cognition, are NOT driven by gradients.

This seems to me quite a deep insight. But how would you formally define a process that is gradient-based?

> But if you remove gradients from your data, or shrink your data down to tens of samples, or shift the problem to logic, or need to use functions that aren't convex or differentiable, deep nets run smack into a wall,

We're making some progress on the convexity bit (heat functions, RL, etc.), but yes, there are other areas of statistical research all involved in trying to solve those sorts of problems.

ML is not necessarily a panacea, but just because you can point to problems that it doesn't solve doesn't mean it has no "real accomplishments."

> DL, like human cognition, are NOT driven by gradients.

???

This round got a lot further than the last round.

I went through Stanford CS in the mid 1980s, just as it was becoming clear that expert systems didn't really do much. There had been Stanford faculty running around claiming "strong AI real soon now" to Congress. Big-time denial as the field collapsed. Almost all the 1980s AI startups went bust. The "AI winter" followed. Today, expert systems are almost forgotten.

This time around, there are large, successful industries using machine learning. Tens of thousands of people understand it. It's still not "strong AI", but it's useful and profitable. So work will continue.

We're still at least one big idea short of "strong AI". A place to look for it is in "common sense", narrowly defined as "not doing something really bad in the next 30 seconds". This just requires lower mammal level AI, not human level. Something with the motion planning and survival capabilities of a squirrel, for example, would be a big advance.

(I once had a conversation with Rod Brooks about this. He'd been building six-legged robot insects, and was giving a talk about how his group was making the jump to human level AI, with a project called "Cog". I asked why such a big jump? Why not try for a robot mouse, which might be within reach. He said "I don't want to go down in history as the man who created the world's best robot mouse". Cog was a flop, and Brooks goes down in history as the the creator of the robot vacuum cleaner.)

> There had been Stanford faculty running around claiming "strong AI real soon now" to Congress.

That would be Ed Feigenbaum, the man who (IMHO) almost single-handedly brought on the AI winter of the 80s. Because of him we had to call all the AI research we did in the 90s something other than "AI" lest it get shut down instantly.

Um, yes. His book. "The Fifth Generation - Artificial Intelligence and Japan's Computer Challenge to the World" lays out his position. For a short version, here's his testimony before a congressional committee, asking for funding.[1] "The era of reasoning machines is inevitable. It is the "manifest destiny" of computing."

He was taken seriously at the time. Chief Scientist of the USAF at one point. Turing Award.

[1] https://stacks.stanford.edu/file/druid:mv321kw4621/mv321kw46...

> The era of reasoning machines is inevitable. It is the "manifest destiny" of computing."

Isn't this pretty much the current opinion of the majority of the thousands of AI researchers and programmers today? Maybe this guy was early to the party but his vision seems in alignment with today's practitioners.

Yes; if anything the hype is even stronger today. The difference is that neural nets on modern hardware really do quite a few useful things, while expert systems on 80s hardware mostly did not.

The reason the hype of today is more dangerous is what I call the grandmother issue: In the 80s we dreamed of a program that would recognize your grandmother when she walked in the room. That was one of our holy grails.

Good news: The grandmother recognizer was finally built with deep neural nets in the 2000s and it works pretty well. It's not perfect but if you tell it to play grandma's favorite song when she walks in the room it's no big deal if it occasionally fails to recognize her, or if it plays the song when someone other than your grandma walks in the room.

Bad news: We now have people who (effectively) want to attach the grandmother recognizer to a machine gun with instructions to shoot your grandmother and only your grandmother, and never to fail to shoot her if she walks in the room. Suddenly the Type 1 and Type 2 errors are a whole lot more consequential. Modern NNs are simply not fit for that job.

We have pretty good AI for low-consequence purposes, but it cannot be used for high-consequence purposes without a lot more fundamental research. Incremental improvements to deep learning are not going to get us there.

Wellllll, um, not really.... Any NN recognizer can be fooled and your neighbor's dog may as likely be shot by the "granny gun AI". You need to read up on the validity of facial recognition software. It ain't nearly human!

"not human level. Something with the motion planning and survival capabilities of a squirrel"

I think you're underestimating how smart a squirrel is.

Ok robot, solve this: [https://youtu.be/hFZFjoX2cGg]

(John Rober’s squirrel maze, one of the most wonderful concotions I’ve found on Youtube.)

It's an indication that we're missing something. A squirrel has somewhere around 75 million to 100 million neurons. That's not all that many in terms of modern hardware. We ought to be able to build squirrel-level AI with cellphone-sized hardware. What are we missing?

There is a big jump from "let's create some algorithms and data structures inspired by a simple model of the brain and see what kind of problems these algorithms are useful for" to "why doesn't our model fully and faithfully represent an actual brain".

The first of these questions has led to real progress in things like image recognition, whereas the second of these questions has not led any real progress in digitizing squirrel brains.

OpenWorm is, after years of work, up to nematode level.[1] Their WormSim is an emulation, at the neuron level, of the simplest creature whose nervous system has been fully mapped. 302 neurons, 25 muscle cells. Runs in a browser.

So true bottom-up is not hopeless.

[1] https://openworm.org/

Animats says "Cog was a flop, and Brooks goes down in history as the the creator of the robot vacuum cleaner."

Would that be the "robot vacuum cleaner that can't see shite?":

https://www.boredpanda.com/robot-vacuum-cleaner-spreads-dog-...

If you get the upgrade, it can see the poo. https://news.ycombinator.com/item?id=28475716

that's the right answer. New technologies are always overestimated in the short term, but underestimated in the long term. The long-term usually is not a generalized solution, but is still a true solution.

speech recognition hype... now most phone trees use them.

electric car hype... now california is full of them.

self driving cars... now many new cars have driver assist.

Post must be machine-generated - it says nothing correct.

Electric cars are NOT AI!

But it is true that "california is full of them", whatever "them" may be. That may have something to do with the crowd (whores and madmen) attracted by the California Gold Rush and whose descendants now mine their wealth in VC endeavors, including deep learning.

https://en.wikipedia.org/wiki/California_Gold_Rush

"driver assist" is a long way from a "self driving car".

I was making a general statement about "technologies"...

I'm saying AI is a technology that is being overestimated now. People are trying to use it for everything now. I think in the future when the hype has worn off it will not have worked for everything hyped. But it will have worked well for a couple things to the point they are taken for granted or have become invisible.

I think there are these technological problems, many that overlap with AI, that a lot of politically powerful people want to come to fruition to enable the World Economic Forum lauded "Fourth Industrial Revolution" [1] and they are willing to invest insane amounts of money in them without expecting much of a return, thus creating really big p/e ratios. Examples include:

1. Self-driving cars. The ultimate being no steering wheel and carefully software controlled and permissioned as to where you are allowed to go.

2. AI powered Body monitoring devices (e.g Fitbit) that can be used to biologically monitor millions of consumers body states and functions.

3. Any new kind of surveillance technology. Miracle mass surveillance is something that's really interesting to this market.

4. Large centralized social media networks that aggressively moderate content with AI.

5. Various kinds of transhumanist biotech that design all sorts of biotech stuff with AI. What the heck is Calico up to anyway?

6. Blockchain stuff. AI enabled or otherwise.

They are also interested in environment tech like any kind of alternative energy, no matter how speculative or impractical and fake meat, but I digress.

So I would guess the bubble is the WEF crowd and friends with absurd amounts of money trying to make their future, for good or ill, a reality and investing without much regard for the economics of the project.

[1]https://www.weforum.org/agenda/2016/01/what-is-the-fourth-in...

It is logical. Protect the elites with automation and control the masses to prevent a "Second French Revolution". Which may be postponed but doubtfully avoided. You will own nothing and will be happy.:)

Precisely. AI outside the beaten-to-death academic problems and datasets is a vast and completely un-plowed field with enormous potential. And unlike last time, there are some sub-fields of this that do actually work this time (computer vision, signal processing, speech recognition/synthesis, some NLP tasks, etc) and beat older baselines, often by a mile. There is a lot of froth, but there's quite a bit of substance as well, and the field is moving faster than any other I have ever experienced.

I worked for a startup for a while that, late in its life, decided it needed to use AI in their product. Then that turned into needing ML in the product. Then it turned out it was just the owners trying to market the product as AI and ML powered when half of the product was the frontend used to configure the decision engine. The manually-configured decision engine was being sold as AI and ML, with the terms used interchangeably. I was actually a little surprised when it didn't work out, but not that surprised.

The main issue is that people have unrealistic expectations in domains that are personally relevant, medicine being an iconic example. So, there there might still be a 'reckoning' of sorts, even if Walmart applies AI successfully behind the scenes to power non-critical user-facing stuff.

Oh for sure. I guess I could have worded it better (not just scare quoted "real"), but it seems like a public expectations reckoning will happen, but not a reckoning that really affects practitioners who aren't 'script-kiddies' for lack of a better term.

(side note, does anyone know a less insulting way to refer to the group people call 'script-kiddies' that still gets the same point across?)

True Scotsmen. (I mean your practitioners.. everyone else is of course, not a true scotsman).

I don't believe that there is a polite way to say "These people have no idea what they're actually doing, they're applying other people's logic blindly, and if anything goes wrong they're stuck."

If that's not what you meant to communicate, you should probably explain more what you mean by 'script kiddie'.

To your side note: “technicians” is a reasonable label here. They know how to use a technique, but aren’t themselves capable of creating one.

There certainly are tasks where a 2% failure rate is fine, but even more importantly, where we see ML/DL having the biggest impact today is in regards to complex system-type problems, where there is often a less-than-discrete notion of failure.

Looking at apps we use every day, almost all of them owe some core feature to ML/DL. ETA prediction, translation, search, spam filtering, speech synthesis, autocomplete, recommendation engines, fraud detection—and that's not even touching the world of computer vision behind nearly every popular photo app.

A key understanding gap in the general public's knowledge of ML is that people think AI === Skynet, and they've therefore been lied to about the field's progress and impact, when in reality, they probably interface with a dozen pieces of technology that are built on top of recent breakthroughs in ML/DL.

Spam filtering is mostly still based on Bayesian filtering, which is simple probability theory over a set of matchers.

Human level AI in 2021 is still "tree climbing with one's eyes on the moon" (Hubert Dreyfus).

Might as well climb to the top just to be sure.

Certainly as long as there is another grant or VC investment within reach!

Is there really an AI that can climb trees? That is pretty complicated, I've just seen stuff that crawls on the ground.

ML is just as much marketing as AI. The machine isn't actually learning.

Reply on news.ycombinator.com