Hacker News Reader

Comment by mtgp1000 | Hacker News Reader

I'm not sure how anyone who's watched the exponential growth of a brand new domain can pick a point today to and say that things aren't as good as we expected. What may have happened was that some eager CEOs have overpromised on timelines and resources. But the revolution is coming, ML is already starting to change society.

We're building the tech. Right now. The author does not even realise the immeasurable potential that general purpose function approximators are having right now across all domains. The applications are not only limitless but they're being realized right now and it's insulting to have my work dismissed so shallowly by someone who speaks with such authority.

There's far more to do with ML and AI than self driving cars and shitty ad recommendations. It's just that the people who live at the intersection of mathematics, programming, and third domains are rare; give us some time. Even $16T in value by 2030 is 2-4 US GDPs from something which basically did not exist 5 years ago.

View on HN

> There's far more to do with ML and AI than self driving cars and shitty ad recommendations.

Yeah, there's also shitty sentencing recommendations[1], new-age phrenology[2], and high-tech redlining[3].

I think your entire field needs to take a year off and take some ethics and philosophy courses before going any further. Otherwise we're all going to end up much worse off.

[1] https://www.nytimes.com/2017/10/26/opinion/algorithm-compas-...

[2] https://www.faception.com/

[3] https://www.fastcompany.com/90269688/high-tech-redlining-ai-...

I agree that plenty of people have been too cavalier about slapping together some models, predicting something, and calling it a day. On the other hand, it's not like fair sentencing or fair loan recommendation is a solved problem for humans either. There is evidence that, when carefully designed, algorithms can produce more equitable outcomes than humans, for example when deciding who and how to release on bail [1].

So I think we don't want to hand over sentencing decisions to a neural network that nobody understands, but I also think careful use of (simpler) machine learning can still improve a lot of our decision-making. The question is how much these decisions improve over what we have now. There's not exactly a surfeit of wise, highly trained ethicists who are happy to make consequential decisions all day. Many human decision-makers are quite flawed too, they just get to hide behind the opacity of being a human rather than an algorithm.

To that end, a whole "fair machine learning" field has sprung up over the past few years to study this. There are like a dozen papers in the area at NeurIPS and ICML every year. There's some progress.

[1] https://www.nber.org/papers/w23180

Besides, the options are not restricted to "a machine decides everything" and "a human decides everything".

We can use machines to guide the humans, finding flaws or biases, recommending further analysis, and in a lot of other ways while still keeping the decisions made by humans.

This is a bit like telling the whole CS community in the 90s that we need a year off the internet to study ethics because people can use it for child porn, money laundering, and cybercrime.

Should we have?

Yes. The lack of ethics requirements in CS curriculums is a terrible oversight.

Would an ethics course and professional society eliminate child porn online? Of course not. Would it have shaped and possibly tempered social media giants into something less toxic? Maybe.

Parent's condescension drips a little heavy, but anyone building power tools for a species like ours ought to reflect a while on the potential consequences, imho

Do you literally think these people are creating harmful technology because they have had a deficit of ethics courses?

I'm not convinced ethics course requirements do much for a moral living. (And a peculiarly large fraction of those curricula often seems to be concerned with agonizing over whether to push various small or large groups of people on to railway tracks to be run over, rather than trying to stop the frigging train)

The examples you link don't seem very difficult to assess if they're approached with high priority placed on not causing trouble for the innocent. Also, presumably the people purchasing and deploying systems like that have usually taken plenty of formal education about ethics (and possibly philosophy), and yet there seems to be plenty of demand. Why is that you reckon?

> The applications are not only limitless

You cannot imagine how many times I've heard this line over my career. What it actually means is "we do not yet understand the limitations of this technology".

If I had a dollar for every time I heard somebody talking about unlimited growth in the dot-com era, I'd have a lot more money than my options ended up being worth.

I completely agree with this. Let me add a non-US perspective that may surprise some people.

I think a big part of what is holding back many companies from making effective, genuine, real-world use of AI is that a significant majority of the individuals involved are bad at their jobs.

On the business side, there is an widespread unwillingness to acknowledge that technical people may be better placed to make decisions than businesspeople. In my view, this includes things that would traditionally be considered business decisions. The business is unwilling to transition from "working on the solution to the problem" to "working on the AI system that solves the problem" (assuming that they even understand what "the problem" is).

On the technology side, I see a massive oversupply of badly underqualified and generally ignorant "data scientists" and "ML engineers" who have just enough bootstrapped understanding and familiarity with the plethora of (absolutely fantastic) open-source tools to fake an entry into the area, but do not have the genuine depth of background that is needed to actually plan, design, and deliver a high quality solution.

Yes, I have met people without "credentialed" backgrounds who are very good indeed, and yes I have met forward-thinking business people, but the number of people who are in these categories is a tiny fraction of the number of people who think they are in these categories.

And that's why businesses are getting less keen on AI.

EDIT: Grammar.

>On the business side, there is an widespread unwillingness to acknowledge that technical people may be better placed to make decisions than businesspeople.

Maybe acknwledging this would mean losing their jobs. Buisness people have mortgages and children and they probably don't like the idea of having their job replaced by a robot. It is important to remember that the admin/money-side people are not playing the game that they tell the operations people they are playing. They want to keep their jobs, accrue power and money, they don't really care about AI or sales, those are just tools for their own game.

“It is difficult to get a man to understand something, when his salary depends on his not understanding it.” - Upton Sinclair

They said technical people, not robots. Perhaps a subtle distinction ;)

Agree totally with the latter part though. In vast swathes of the business world, technology is not perceived as a way to gain a competitive advantage but rather something between an interesting novelty and an annoyance. The problem is there's a cultural expectation that you be "innovative" which tends to be treated as a synonym for deployment of computer-related technology upgrades (although of course it doesn't have to mean this).

But to people who have no intrinsic interest in technology, there's no real way for them to actually innovate. They just don't know where to start. So they latch on to trends they read about in the Economist or NYT on a truly massive scale. They spend days or weeks making PowerPoints about the transformative potential of blockchain, IoT or AI. They take a few programmers who are kicking around but seem bored and allocate them to an "innovation lab" where they putter around making prototypes and having fun but never impacting the business in any way.

This ticks the box labelled "we are innovative" without requiring anyone to think too hard, learn anything new or take any risks, all things that the generic unskilled graduate managerial class in our society hate doing. And of course technology upgrades are risky. When run by non-technical people they tend to go wrong in spectacularly expensive and mysterious ways, so a lot of business executives would rather pull their own teeth out than plan a major IT upgrade. This is partly why tech firms exist as a concept.

As for AI, you can't replace most of these jobs with AI because their outputs are undefined to begin with. And to be frank, my observations over the years has been that a lot of these jobs appear to be undefined in order to enable a form of cultural stuffing. If you're a non-technical executive you wouldn't want your nice business to end up filled with geeks playing board games and aggressively proving other people wrong now, would you? Better keep them diluted by hiring lots of PMs, "operations specialists", "customer analysts", "business insight teams" and so on. That way you can meet your diversity numbers and be surrounded by like-minded people.

I am an underqualified and generally ignorant data scientist.

I'm really good at understanding technology and talking to people though, so all the true experts who I know enough to talk shop with love me for keeping management off their backs.

Well hopefully startups that do so will outcompete the older models

Your argument boils down to “AI won’t be useful unless humans are twice as smart as they are” (your examples of the businesspeople and researchers), and thus doesn’t really say anything.

It is not smartness at all, but politics.

Your perspective is not just 'non-us'. The thing is since the early 80s computers can generate huge swaths of data. ML gives you a way to filter that data in a particular way. The same was true of data warehouses, smart systems, etc. The issue is not the business people vs technical people. It is an understanding of what do you want it to do. A few years ago I had a system that could generate 2k in data samples every few seconds (switches, voltages, temps, etc). What are you even looking for in that pile of stuff? You can not just feed that into a ML network and hope for the best. You have to describe what you are looking for. I had this same conversation over and over when working with data warehouse projects. A good BA on a project like that is amazing. Someone who is just kinda meh on it will kill the project dead. I do not understand your business, you do. I can apply what I have learned from other companies but only to a point. After that point I basically have to become a BA in that company just to understand what to write.

I think you've identified the basic, classic difference between artists/thinkers/scientists and everyone else: just as a musician can be given a piano and in time they create complex compositions, those that can see and operate in abstract conceptual spaces will see and use AI to create wonderfully complex compositions everyone else will be slackjawed in awe. This is human nature. AI is not an intelligence player, it's an instrument to find and tune like steroids one's own intelligence. Just like a piano, AI is useless to most people, because they do not have the artists/thinkers/scientists drive and comfort with abstract problems that are easier with even the most basic AI.

Well of course you say that, you're an ML researcher who likely went into the field expecting further steady progress comparable to what we saw between 2012 and 2016. If actual progress in the field would be slower than what is currently still expected by the majority of people this would have dramatic consequences for future research investment, which is why you see most ML researchers reinforcing the hype or at least not talking the hype down.

While the accomplishments in the last 8 years have been impressive and applications of those techniques have and will continue to have impact in the real world I think that there are extremely big obstacles in the way towards something that would be truly transformative and which would put a lot of people out of work, which the $20 trillion industry estimates assume. Unfortunately I have seen no evidence that there are sufficiently good ideas in the field to bypass the upcoming roadblocks.

I think a major problem in judging the advancement of AI is that its successes are not general purpose -- it works great for narrow tasks and not well for broad tasks that we humans do every day, especially those that require what we call "understanding" (as in modeling and employing semantics or relationships among objects like causal inference).

Deep learning's big success is its ability to attach a label to a complex signal -- an image or a sound. That's pattern recognition: speech in, speech out, and image recognition. These tasks were largely beyond computers until 2012 but something that a child or a squirrel does very well, so it's been long known that skills like these are not useful signs of intelligence or rising levels of cognition.

Other tasks DL does well are those that benefit from memorization of results form death-by-search and from vast amounts of RL simulation, like playing of board games and the transforming of patterns (images and sounds) in fun ways using GANs. But ever since Deep Blue's win over Kasparov using clever pattern matching of past chess games (by memorization), we've known that narrow skills like game play also are not useful signs of intelligence.

Yet pattern matching skills are 99% of what deep learning hath wrought. Yes, that's useful, but it's not really intelligent. It shows no signs of thinking / cognition aside from probabilistic association / clustering. So there's no reason to imagine that techniques like deep nets will take us all the way to thinking like a human. Today, because of DL, we're much better at pattern matching. But in terms of what's essential to cognition, pattern matching achieves surprisingly little if your real goal is to THINK.

As far as "thinking" tasks go, cognitive tasks like machine interpretation of intent in written text and machine translation between languages still suck, despite the impressive advances in semantic-surface associators like BERT and newer transformer-based NLP engines. To do more than answer basic questions about nouns and verbs, you need a model for deeper semantics and an understanding of logic and relations between actors. Until deep nets can model semantics that are not present in the test data, and employ logical inference, it can't be said to think, much less intelligently.

> it's insulting to have my work dismissed so shallowly

You haven't had to deal with end-users much have you? I've seen years of really excellent work (not my own) blithely dismissed with expressions such as "but what does it now?" or "why can't it do X [impossible thing]?"

People have been told AI is potentially going to take over from the human race. Unless that's in your pipeline, they're going to basically treat your work as eaten bread.

Which is precisely how it's supposed to be. Nobody is required to respect your work in progress. The world will either care or...care not.

Your comment is a little hand-wavy and strongly worded ("revolution", "immeasurable", "limitless", "all domains").

Many things have exponential growth - bacterial reproduction, compound interest, certain chemical reactions. It's important to understand that this does not automatically result in miraculous universal transformation, but must be considered in the context of the world we live in. A little humility is always in order.

Your analytical thinking misses the human element. To give an example: Steve Jobs didn't have humility, nor did he shy away from hyperbole. You would have had the same response to his words, but look at what he - literally, in the physical world - achieved. Human motivation - no matter how deluded - is not actually something not to dismiss. It has effects on others, which translates to action. The inputs are not all machines. At least not yet.

While not wanting to downplay the marketing and design genius of Steve Jobs who created a brand identity so strong that for a while, "smartphone" became synonymous with "Apple", Steve Jobs really stood on the shoulders of giants in terms of innovation.

Without the Alan Turings, John von Neumanns, Edgar Dijkstras, Donald Knuths, the researchers at XPARC and Bell Labs, etc., there would never have been a Steve Jobs. He does play a role, but not nearly one as big as all those people that came before him.

Some people like to push for progress for progress's sake; others mostly to make money out of it (not saying this is bad!). The latter get more rich but it doesn't mean they're more important.

> I'm not sure how anyone who's watched the exponential growth of a brand new domain can pick a point today to and say that things aren't as good as we expected.

They do worse than this; one of the articles in the section uses a bad Google translation in Yoruba, from 2018, and uses that to claim it “is therefore still baffled by the sorts of questions a toddler would find trivial”. I'm not sure Yoruba even used neural nets at that time!—‘massively multilingual’ training only really came about in 2019, and was productiviced June this year.

A disappointing show from The Economist, given how little effort it would have taken to check with the newest model.

> ML is already starting to change society.

It already has—look at how much companies try to shunt you through a robot before you speak to a human despite their being mostly useless compared to humans. It’s clear this revolution we both see happening will come with horribly kafkaesque failures that will take 30 years to “discover” so we in the tech industry can “learn our lesson”.

Hell you can already see this with the use of fraud detection rather than implementing basic secure transactions on credit cards. The direction we’re headed in is clearer than ever: technology does not lead to better products but rather larger scale consumption of lower quality.

There's value, the tech works, but applying it is surprisingly hard.

I met someone who dedicates their life to using machine learning to replace/aid/automate pathologists 6+ hour days searching for cancer tumors in lungs. They have been at it for 5 years. There is an insane amount of approvals, red tape, knowing the right people, convincing the hospital to use it - all tasks not related to the tech actually working. Building it is the easy part. They were on the way there, working in the research section of a hospital. But they are still more than 5 years away from me being able to walk into a hospital and get my lungs scanned for tumours.

I personally can't wait for these general purpose function approximators to make all our lives better.

It's not that the technology isn't there. It's not that the technology can't be technically applied successfully. It's that convincing systems of people to change is way harder than we might think.

There is a strong chance that the technology to create digital beings will be available before human society is able to integrate and adjust to the current generation of AI.

So what may happen is that the way that AI really gets integrated is by actually replacing human beings who largely die off.

That's a really interesting perspective and a good point. Reminds of "Age of Em". Interested to see what the effects would be of digit beings before integrating current generation of AI would be.

I am a practicing pathologist and I have seen many attempts and publications to use ML in pathology, which all lack in these aspects: 1. ML is trained on simplified sets (preselected ROIs, limited choice of diagnoses), 2. ML is biased by the experts who labeled learning sets, 3. there is no obvious process of learning from failures after initial training, 4. who is responsible in case of ML error with substantial consequences for patient? The first point is especially for the lack of better word.. wishful. In the daily practice we are used to account for "things unexpected" - non-representative biopsies, parasites in tissue where tumor was suspected, foreign body reaction from previous operations, laboratory accidents (such as swapped paraffin blocks of two patients), and so on (the list is much longer). We deal with it. That ML can discern between 5 most common diagnoses is fine, but it is rather narrow problem to solve.

Thanks for the very insightful response, you're right. All those problems are barriers to using the ML practically. So it seems like the technology actually is the problem.

Ok, consider me intrigued. What is that we are going to see/experience once you folks had some time? Genuine question! Can you give us a basic idea of the things that you are already sure by now will see the light of day?

Mathematical modeling that is 3-6 orders of magnitude faster, we are already talking deployment.

Same for ML powered solutions to data management - I don't want to say enough to identify anything.

My team has been working on a rudimentary humanlike reasoning engine based loosely on what AlphaGo proved: that machines can learn heuristics identical, equal to, or better than those of humans.

And for perspective, AlphaGo was what...4 years ago? Just look at the growth of compute available cheaply in the cloud since then! These next few decades are going going to see a phenomenal acceleration that's already starting...assuming we don't hit some unforseen limit or block. But even with what we've discovered now, we have more than enough in the way of new tools to refine across industry and society for decades.

Your training set must be something unique to make those claims - very intrigued to know if you chose another closed system environment (I.e. any structured game with predefined scoring), or an open system with carefully selected labeling with multiple “scores.”

Also complete novice so be merciful if I’m off base.

since you’re working on the field : i’m still under the impression that modern ML (neural network in particular) are not producing any science, nor are they producing any level of understanding of the phenomenon it’s trying to model. And that as such, we can’t provide a reliable estimate of the limitations of the produced model, other than by feeding it tons of inputs and measure the result.

It would be like trying to understand gravity by throwing thousands of rocks, measuring the motions, and build an approximation on a huge matrix, without ever being able to get the « real » netwonian formula.

Am i correct ?

Some of this would be considered a feature by companies. It's more defensible to have unknowable AI deciding to do illegal things than programmers hard coding illegal things. Which really boggles my mind. When my kid does something illegal I'm held liable. When an ML algorithm programmed by a team of people does, nothing we can do about that!

I don't see this as really a unique situation or one where people are exempt from liability. If someone starts a fire, they are held liable; that's analogous.

On the other hand, the difference between a team of people and a particular person with mens rea, that's not unique to AI either.

I'm not sure if the courts will see it this way. Anyway, in some fields some stuff (e.g. credit scoring in banka in my country) is regulated and it's the humans who have to make the final decisions - algoriths can only provide an input. In practice, the input is "approve/deny", but it's still a human who is making a "decision" based on this "input".

>When my kid does something illegal I'm held liable

This isn't true.

What makes you think Newtonian physics is in any form more real than computation? If Wolfram is correct, physics is inherently a computation in the form of cellular automata.

Intelligence might be so messy that the only way to get to it is to grow it organically, which to me seems to be happening in research. People are trying a bunch of stuff based on intuitions for what may work, and post hoc adding some theoretical justification for why it's working. Human intelligence didn't emerge from some grand theory of intelligence either.

> And for perspective, AlphaGo was what...4 years ago? Just look at the growth of compute available cheaply in the cloud since then

How would we know the difference between a problem that was interesting to solve but was hard versus a problem that just needed Google's money to solve?

Besides, since when was cloud compute was getting cheaper? I'm pretty sure, considering how increasingly profitable it is despite the "discounts," cloud compute has gotten more expensive, in the sense of increasing margins for the seller.

I guess we'll all find out what OpenAI does with its Azure credits.

mtgp1000 says>"mathematical modeling that is 3-6 orders of magnitude faster, we are already talking deployment."<

How seriously can we take that phrase? Please mathematically model the Covid-19 pandemic's next 6 months. The epidemiology community dropped the ball and you can hardly do worse. Take the "Covid Challenge!"

mtgp1000 says>"These next few decades are going going to see a phenomenal acceleration that's already starting ...assuming we don't hit some unforseen[sic] limit or block."< -

Nobody can predict next year's economy and certainly nothing beyond 5 years! You're predicting for the "next few decades"?

mtgp1000 says*>"But even with what we've discovered now, we have more than enough in the way of new tools to refine across industry and society for decades."

Spoken like a true non-English marketeer!

I guess once the field starts mixing probabilistic, deep and logic approaches we will see a bit of progress towards AGI.

This is one example of where things might be heading: https://science.sciencemag.org/content/350/6266/1332

This has been on my watchlist for a while. The problem is the inablity of these techniques to scale to larger problems. That being said, once that happens, it could be revolutionary.

DARPA threw some money into the problem, and stuff advanced from toy problems to things that are still small but often useful.

See e.g. pyro.ai is already working on non-trivial datasets.

Ultimately, I'm convinced it's a hard compilers / static analysis / abstract interpretation problem. That requires lots of resources and a decade or two of work.

"Hi AlphaGo, let's play tic tac toe". Consider me bored of it all. We're not close to anything. The emphasis is on "artificial", not "intelligence". It should be renamed Glorified Calculation. There are some neat tricks, writing your first neural net is great fun, but we're using the word intelligence with spectacular liberty. I'll keep an eye open for something inspiring, but I've seen nothing of the sort yet.

Then you're not looking hard enough

The modesty of AI researchers is a major problem here. So many miraculous discoveries go completely unadvertised.

Proof of AGI progress would be a significant enough achievement (given there hasn't been any for decades) that it wouldn't require looking hard - but please share any links to it you have.

Nobody else seems able to, especially the AI ("more power and money needed! we're just around the corner!") blowhards.

One big one is farming, think harvesting machinery. There's a number of startups trying to get that going. I think most are not at the scale to be successful the way the market themselves, but their collective learning will eventually lead to some consolidation.

Why AI for that, it's vision AI to know when fruit is ripe or vegetables are ready for harvest. Then hand eye coordination to not bruise the fruit/vegetables and adjust force/sensitivity as the equipment wears down.

Scaling that will put likely many workers out and transform the business of harvesting. What I think is most plausible is one large manufacturer will consolidate designs and rent the equipment to do the harvesting much like the contract labor used now, but at much lower prices than humans could do.

Depending on the crop you’re talking about, we are still very very far away from this.

People have been trying to design machines to pick fruit, even in toy scenarios, for decades, and we have little to show for it. I don’t think that image processing is the bottleneck here, it’s literally the mechanism as far as I understand.

I’m skeptical that any mechanical equipment will ever be able to do these tasks as cheaply, quickly, and efficiently as a human in our lifetimes.

Look at olive harvesting machines.

Definitely easier than fruit, but we've made so much progress.

If it's AI to know when fruit is ripe, etc., then bugs and birds and so on must be intelligent.

> What I think is most plausible is one large manufacturer will consolidate designs and rent the equipment to do the harvesting

No way. Everybody wants the machines at the same time. You're going to incur the capital cost of buying/manufacturing the machines, the cost to store and maintain them for 50 weeks out of the year, and then rent them out at a profit for 2 weeks? No, the farmers will just keep hiring human labor.

If you can solve this problem, you can sell a lot of machines for a lot of money for a short few years. After that, they'll be a commodity and be dominated by all the other capital-intensive manufacturers that can run a factory better than you can.

The trick with so-called exponential growth is that it is almost always an S-curve in disguise.

Is it too soon to tell that we start seeing the asymptote of the AI curve?

Spot on!

>> The applications are not only limitless but they're being realized right now and it's insulting to have my work dismissed so shallowly by someone who speaks with such authority.

What exactly is your work? In research, including AI research, it is ordinary and expected for one's work to be subject to the most ruthless criticism. This is as it should be: only work that survives all criticism can really be expected to last.

If you are an AI researcher, you should be used to that by now and not take it personally.

Do you see any irony in your comment complaining about CEOs over promising and then you go on to describe “immeasurable” gains and “limitless” technology?

Are there examples in history where things requiring great time and effort were built then abandoned?

In recent times, due to a relative dearth of high yield investment opportunities, there is a lot of money with nowhere to go. Some believe much of it is "dumb" money. Overfunded startups are one possible symptom. Heavily-funded "AI" may be another.

This is before we even consider the moral and ethical issues of using "AI" as a substitute for human judgment. It may "work" but that in itself may not be an adequate justification for its use.

> Are there examples in history where things requiring great time and effort were built then abandoned?

The space program?

In the US, yeah. Great example. SpaceX is here to get you back on track. /s

> Are there examples in history where things requiring great time and effort were built then abandoned?

The communist countries seen a huge misallocation of funds into giant projects that didn't make sense (e.g. huge steel mills in Poland that were always losing money).

Absolutely. To be perfectly honest, it surprises me the extent to which ML naysaying seems to be popular on HN. The evidence of enormous progress seems pretty obvious to me.

There have been plenty of articles about progress in machine learning, but nothing in genuine (human-like) intelligence.

I would expect that's the issue for most people who are interested in the subject.

I think ML naysaying stems mostly from extravagant claims of pop-sci articles around papers or of the authors themselves. It is always good to have some amount of people that point out that the models aren't some cyborg superheroes or has to be a step towards AGI but be more practical.

Siri still isn’t able to understand « do NOT set the alarm to 3pm », and many image classifier produces aberrations that no human would ever commit.

Many people feel that ML has so far only produced « tricks », but still doesn’t show any sign of « understanding » anything. As in, provide meaning. It may be unfair, but i think that at this point people would be more impressed by a program « smiling » at a good joke than by something able to process millions of a positions per second, or « learn » good moves by playing billions of games against itself.

While I can't speak for Siri, Google's voice recognition continues to surprise and even frighten me. And if you go online and look up deepfakes or AI generated voice of famous people, it also is freakishly accurate.

Sure, there are some issues and most of them still fall somewhere in the uncanny valley.. but we are just starting to fully exploit this technology. AI that people don't tangibly see but make a giant impact in their lives is for example the Facebook algorithms that decide what shows up on your feed.

When a large subset of the population gets majority of their news from Facbeook, Facebook has effectively created a mechanical system that decides what a large chunk of the population is aware of. I don't think people fully realize what effect that has had on our entire society, including elections. The future will only see more of this type of thing.

I'm always disappointed with Google Asistant. When I say "navigate back home" it tries to route me to a company called "Home". Great job. I've included a hint "back". I've never been to a company called "Home". I'm after work. This is a fun example, but I never get good results. I say "weather" and it shows me the weather, just in a different city. FFS. Maybe in the next 5 years I'll be finally able to do something using voice commands. Right now I just want to smash my phone.

Fair enough. Usually when I use "Navigate to the nearest gas station" or "navigate home" or "set an alarm for 8am on Tuesday" it all works perfectly for me, but I concede I don't use the feature that often so perhaps I've just been using it for well-adjusted commands.

That's the convolution of two problems:

a) Good general-purpose automatic speech recognition, which was an unattainable holy grail for DECADES. Previously, you would have to record a lot of your own voice to train the system for you in particular before it was at all reliable. Now it basically just works.

b) Making good use of the results of (a). Your examples are clearly in this category. This is ALSO a very hard problem; voice interfaces are basically brand new creatures, and I expect we'll be seeing 'best practices' form up for a while yet.

Sure, ML is not at human-level intelligence - it is very much a tool-AI where we give a task to the machine and let it get very good at that.

Nonetheless, it seems like progress on that front has been made incredibly quickly. Sure, Siri might not always "understand" what you're asking but the ML is able to very accurately transcribe your voice to text, something not really possible a few decades ago. Image classifiers produce misclassifications, but so do humans - it is a very alien technology to us, so we view some of the machine's misclassifications as absurd, but at the same time the machine might view some of our misclassifications as obviously off-base as well.

None of what you said seems to explain the consistent negativity on HN to what has really been a transformative technology in a lot of ways.

Progress in speech recognition has been slow and incremental, not fast and impressive. Hidden Markov Models did a decent job in the 90s. Now we have more data and more computing power.

In the past three years we have dropped error rates by a factor of 3, I don’t really think that claim holds water.

We have seen huge progress in a number of other fields as well. Anecdotally, voice recognition has definitely gotten way better as well.

Both voice recognition had first working research machines in the 1950s. Of course present models are way better than these, but fundamentally these are the same as those in the '50s, "just" with tremendously better hardware and algorithms. But there is zero intelligence in these, the models have no internal concept of language or the world around them.

This will certainly produce many great specific solutions for specific problems, but generic problems like driving a car are most probably not solvable without an internal world model.

I'm one of the naysayers and have a background in educational science, I'm constantly baffled why AI research seemingly never looks at theories of human learning (though it may very well be that I haven't looked closely enough!). What essentially all AI approaches do is modelling associative learning (https://en.wikipedia.org/wiki/Learning#Associative_learning), but employing more and more processing power for training. This is akin to having a fruit fly brain and copy-pasting that over and over again, in the hope somehow a higher order will organize from that by itself. But most things humans reason about are not learned this way, but rather by infering meaning from things and situations, i.e. one learns how a steering wheel works not through hundreds and hundreds of trial and error cases, but through single Aha! moments in which ad-hoc generated mental models (concepts) are validated in the environment. And GPT-3 has no knowledge organized in a hierarchical system of concepts, just as ELIZA didn't have one.

There are tons of research projects that do exactly what you're talking about, initializing an agent with zero knowledge in an environment which just provide rules of discovery and reward. The agent then takes actions in that space and learns from its own experiences. It's just that it is hard to compare with humans, who have had the benefit of evolution over millions of years. A basic example is that of AlphaGo Zero[1], which learns to play a top level game by being given enough time and just the rules of the game. This is similar to how a child learns to walk, it is just that we only have the capability to model toy situations (a board game in this instance) right now, and access to harder instances (movement in the real world) will slowly come about. There are cases of robots being programmed to poke / move / pick objects to try and learn about their shapes[2], in case you are interested in another such example.

[1] - https://en.wikipedia.org/wiki/AlphaGo_Zero [2] - https://bair.berkeley.edu/blog/2019/03/21/tactile/

AlphaGo Zero definitely falls in the same category as GPT-3 - yes, this is unsupervised learning, but it still is fundamentally the same approach, exactly because of the way discovery and reward work on the model.

This isn't how a child learns to walk at all: A child a priori has the concept of walking, the concept of self, the concept of movement in space, the concept of willing to walk etc. - it just doesn't have the motoric control. The small part of training motoric control through repeated trial and error is indeed similar to what unsupervised learning models, but the important part is missing.

>In the past three years we have dropped error rates by a factor of 3

What are you referring to here?

I wonder if it's because of the name machine learning? It seems a lot of applications boil down to some type of discriminator or pattern matcher.

> It seems a lot of applications boil down to some type of discriminator or pattern matcher

This doesn't seem to imply that no learning is going on or that there is some sort of simple process. If I ask you to say whether something is a cat or a dog, you are essentially functioning as a discriminator there - but there is still a lot of process that goes on behind that.

I recently watched a podcast with Stephen Wolfram (few months old) where he runs a photo of himself through Wolfram Alpha "ImageIdentify" and it classifies him as a plunger with 50% probability and 8% probability it's a human [1]. He then goes on saying how he's working on systems that would allow a self driving system decide what to do based on if the collision detected is a human or inanimate object, etc.

This was just one particularly ironic example of what I'm seeing with ML - we saw this level of accuracy in classification years ago - where is the enormous progress ? There have been pockets where ML shows it's applicable with encouraging initial results but after that it seems to me that we suck at perfecting those system and the progress has been incremental at best.

The only exception I can see is stuff like self-play where the system can generate insane amount of training data.

[1] https://youtu.be/ez773teNFYA?t=8420

Classification accuracy is crazy good, I'm surprised that an obvious photo of him would be viewed as a plunger unless it's an adversarial example.

No offense to Wolfram, but they really are hardly at the cutting edge of ML research nonetheless.

> To be perfectly honest, it surprises me the extent to which ML naysaying seems to be popular on HN.

Maybe because there is a larger fraction of people who know or see how the sausages are made.

There are cool things, but is not magical nor transformative. At least not yet.

I work in “making the sausage” - deep learning techniques have already completely transformed my field and are transforming many others. Most of the people I see naysaying don’t appear to causally work in the field, because their critiques are not the same as the ones people in the field are using.

Can you give a concrete example of deep-learning techniques which have transformed a field? Something not theoretical or confined in a lab? Something which may have changed my life?

Not OP but I can speak with a small example from my previous field - coral reef ecology. It used to be standard to collect video footage and hand annotate small portions of the footage to draw up some statistics about the reef progression. Today we can collect the footage and let ml annotate the entirety of it to get much more accurate and precise statistics about the progression/recession. Not only that but we can do that on an organism basis and also see do highly local analyses. This wasn’t possible 10 years ago. I think the applications for ML outside of consumer tech are far larger than within but everyone here seems focused on that for some reason.

Honest curiosity: do you have some examples of interesting applications? Large and small?

How about translation? Living in a foreign country I possibly find it useful more often than the average person.

Machine generated subtitles on YouTube is another example. I find those useful even in my native tongue, especially when watching at 2x speed.

If you make purchases on the web, there is a good chance that machine learning is behind the fraud detection that you don't even know is taking place.

I work on a production ML platform, so I spend way too much time rabbit-holing on interesting looking projects.

If you're looking for interesting startups/projects-not-from-big-tech:

- Glisten.ai (https://www.glisten.ai/). Recent YC startup, uses a combination of different models to parse product information (actually a huge manual problem in retail/ecommerce) and expose it as an api.

- Wildlife Protection Solutions - Recently deployed a model that can automatically detect poachers in nature preserves. Detects twice as many poachers as previous monitoring solutions.

- Ezra.ai - Uses models to search MRIs for cancers, operational in a few different US cities.

- AI Dungeon - A text adventure game built on GPT-2 (now GPT-3). Super fun, if a little silly.

Now, those are just a handful of smaller companies whose core products are ML. There are a ton of financial institutions using ML for fraud detection, real estate platforms like Reonomy that use a variety of models for evaluating investments, and security companies using ML.

But of course, the obvious answer to this question is "Every popular app you use incorporates ML."

Gmail: Smart Compose, spam filtering, etc.

Uber/Maps: ETA Prediction

Netflix/Spotify/all content platforms: Recommendation engines

Facebook/Instagram/Snap/image apps: A variety of models for recognizing faces, object tracking, etc.

People have this weird "Skynet or it's snake oil" paradigm they use to evaluate ML, ignoring the fact that production machine learning is more or less ubiquitous at this point.

> Maps: ETA Prediction

So THAT's why the ETA given is always too short! If it's based on how often a typical driver makes it, and typical driver is a speeding asshole, then no wonder that the estimates are unrealistic for someone who actually drives under the speed limit. It's a shame that Google is actually normalizing assholiness.

What's wrong with speeding? People generally drive at a speed such that they're taking an appropriate level of risk. The speed limits usually set these risk limits too low. Moreover, the job of google's ETA is to be accurate for the most number of people. If you're in the minority, then too bad.

Moreover, I'm not sure how speeding makes you an asshole.

By speeding, you increase the risk of death/injury of other people, you increase their stress level and you also increase road noise. Moreover, you unilaterally (and self-servingly) decide that the norms established by authorities are stupid and that you know better. This all sounds like a description of an asshole to me.

I'm not much into machine learning myself, but I must admit that ML has been invaluable for good voice recognition and good voice synthesis. Writing programs to interpret speech accurately and synthesizing a "natural" sounding voice by hand is practically impossible, but the perfect application for ML.

Got bad news for you man. 16T in value doesn't come from ML. It comes from inflation and printing money out of thin air. The world has limited resources at the end of the day. When one industry does well, it's at the cost of a different industry.

We haven't even mined the moon yet. Plenty of gains to be made.

Perhaps basing our decisions on general purpose function approximators (whether forced or voluntarily) has not served us so well, ultimately. I think they play an important role in causing, or at least enabling, much of the trouble that we live in nowadays.

How?

Is it not better to engage with the critics and correct their perception in that case? I'm sure the author would not mind a direct response that outlines where they went wrong and why.

I googled "Tim Cross" and he's reachable on Twitter: http://mediadirectory.economist.com/people/tim-cross/.

More generally, I don't understand gnashing of teeth when it is much easier to provide feedback to the author directly.

It sounds like some buzzword speak. Most things that are heralded as ML are nothing but data science idiots from python schools applying some basic math transformations and overselling them.

I was going to say "most ML these days nothing but chains of if-else statements" but yeah same idea.

What's wrong with python?

Nothing wrong with python or JS. But there are many things wrong with bootcamp / workshop people who come out just learning Python with some ML library or JS with some web framework. They don't know even basics of CS and arguably produce bad, inefficient code.

There is a famous word for latter - 'webshit'. The former doesn't have such a title. But there is an obscure one originating from a Slashdot comment IIRC something like: 'you are one of those data science idiots from python schools eh?'

Edit: I didn't mean disrespect to those doing meaningful research. But that's not what most of things sold as 'ML' are.

Can you be a little bit more concrete about the tech you are building right now that is not possible to build using classic ML approaches? You see, I am actually going to work thinking about how should we reduce the model size and still keep it within reasonable accuracy bounds so it would not be worse than a rule-based mixed with classic ML system that we use. And here I learn about immesuarable potential, insults and billions of dollars.

There has been next to ZERO progress towards genuine AGI despite a never-ending deluge of AI articles; that's normally the cause of scepticism.

After several decades and a much-hyped last few years we have fake cleverness - impressively so in both cases - but nothing more.

> After several decades and a much-hyped last few years we have fake cleverness - impressively so in both cases - but nothing more.

Why should I care if my fridge is fake clever or real clever?

You should care if the makers of your fridge are charging you a billion dollars to buy it because its "real clever".

I think the problem is that in a long list of edge cases, it's neither fake clever nor real clever but just "dumb". This is, of course, always an issue with computers, but what makes ML so vexing is that it's inscrutable.

For your fridge that's an annoyance, but when used for more serious applications the ramifications can be large. Example: https://www.technologyreview.com/2019/01/21/137783/algorithm... – there's much more where that came from.

ML works by taking a huge amount of data and finding patterns and is pretty much incapable of judging individual cases. We have ugly words for humans who judge people based on generalisations instead of judging the individual...

Computers and algorithms are devoid of empathy and humanity, I am extremely apprehensive in using them to make any sort of judgement call, especially if the calculation is inscrutable. Humans may not be perfect, but at least they're ... human.

ML is an interesting tool which can be used for many things, but right now it's also being misapplied for many things. I think a lot of opposition comes from that, rather than your fridge or whatnot.

Why do I want my fridge to be clever at all. Just keep things cold please.

For the same reason you should care if another person is conscious or not.

It's even hard to say how "genuine" cleverness in human beings works - and we've had a lot more time to study ourselves - granted that most of that time we've not had the tools to understand how our brains work at the algorithmic level. I am not claiming that achieving some version of "genuine" AGI necessarily involves understanding how human intelligence works, but it is reasonable to expect that knowing more about it will definitely reduce the search space of possible systems.

Is it reasonable? What would you learn about wheels by studying cheetahs? Or about steam engines by studying strong elephants? Or about cranes by studying giraffe? Or about jet engines by studying birds? What if AI is like that to a human brain? Completely different and way better in some dimensions?

>There has been next to ZERO progress towards genuine AGI

I mean, really? In the most pessimistic evaluation of ML research, we still know which techniques and paradigms won't work for AGI. That's not zero progress. Nobody is expecting this to happen overnight.

My comment was in the context of yet another completely unsourced claim of progress.

No evidence - as always. No progress.

Of course, if you - or anybody else - can show evidence of a genuine advance towards AGI, rather than breathless hype and vague assertions based on the impressive but irrelevant narrow-domain "AI" expertise of Alpha-x and the like, I (and many others) would love to read it.

We've been waiting since at least Alan Turing.

I would say the only tangible evidence of a "genuine advance" towards AGI would be the growing amount of computational power available to us every year.

That presupposes we need that power.

Perhaps we do, but if we don't, then the continued lack of progress is in fact evidence of our ineptitude and inefficiency in the face of an embarrassment of riches in terms of tools and power.

It's kind of like a having a searchlight that is more and more powerful and still not being able to find the object in the darkness we are looking for because we have given it to a blind man.

Plenty of practitioners ready to impress us with buzzwords, empty assurances and paper after paper on the brilliance of obscure statistical formulae and their money pit implementation in server farms.

But all this power and cliquey self congratulation is just noise drowning out a very embarrassing truth:

We don't have any new ideas.

Disagree. GPT-2/GPT-3 are able to pass for humans when the reader isn't paying close attention. This article seems insightful to me:

http://www.overcomingbias.com/2017/03/better-babblers.html

A scarily large amount of human speech is essentially word prediction, especially in cases where someone wants to seem impressive without having actually done the work. We're all familiar with the problem of people "bullshitting" about technical topics, especially in the business realm where there are lots of people with no technical background who want to learn and have authoritative opinions on technology, like in this article. The GPT models are quite capable of producing this kind of "sounds legit" speech. And if that isn't a form of intelligence, then what is?

Where in the GPT models is the conscious (or even subconscious) thinking, comprehension, understanding, relating to experience, imagination etc, etc, you know: actual intelligence as a human would feel and use it, general or otherwise?

Tic Tac Toe, Chess, Image Classifying, Translation, Go, Sentence Construction, etc.; We are creating phenomenally impressive calculating worm-equivalents, but nothing human-level, or even remotely so.

Where in the human mind are these things? We judge that they exist based primarily on what people say and do. GPT-style models can't do anything, but they can say. And for better or worse a whole lot of ordinary, everyday speech doesn't have a whole lot of thinking, comprehension or imagination to it. It's just people saying what other people seem to be saying in order to try and stay with the herd. GPT-3 is scarily close to matching that.

So, they're getting pretty close to matching us in mindlessness and stupidity. Call me when they can match us in our highs, not in our lows.

Yep, pretty much :)

It's pretty fascinating that nobody (AFAIK) really predicted that, except perhaps Isaac Asimov. Sci-fi normally presents AI as either superintelligent or non-existent (just basic voice commands). I don't think I've seen one which presents it as an AI ecology of idiocracy.

We've been able to produce that kind of speech before though. I wrote programs using Markov Chains 10 years ago which could produce almost real sounding speech but I didn't think it was intelligent.

The GPT models are essentially massive databases along with the use of probability models for how words are connected. Do you think that means intelligence? Can it generate new knowledge of things outside its database? Can it generate novel theories for things outside its understanding?

The GPT models often generate streams of text which can found word for word in its database, i.e. it's not really generating anything new as just mimicking. When a parrot repeats a human, it doesn't understand what the words mean, it's just making sounds.

But at some level our own brains are just very large databases and very good prediction engines, at least when running on auto-pilot and lacking any deep understanding. Hence the phenomenon of BS artists. Sad to say but I've heard speeches that were less coherent and interesting than what GPT-3 produces.

Reply on news.ycombinator.com