Comment by nopinsight | Hacker News Reader

Comment by nopinsight | Hacker News Reader

A different take by Google’s cofounder, Sergey Brin, in his most recent Founders’ Letter to investors:

“The new spring in artificial intelligence is the most significant development in computing in my lifetime.”

He listed many examples below the quote.

“understand images in Google Photos;

enable Waymo cars to recognize and distinguish objects safely;

significantly improve sound and camera quality in our hardware;

understand and produce speech for Google Home;

translate over 100 languages in Google Translate;

caption over a billion videos in 10 languages on YouTube;

improve the efficiency of our data centers;

help doctors diagnose diseases, such as diabetic retinopathy;

discover new planetary systems; ...”

https://abc.xyz/investor/founders-letters/2017/index.html

An example from another continent:

“To build the database, the hospital said it spent nearly two years to study more than 100,000 of its digital medical records spanning 12 years. The hospital also trained the AI tool using data from over 300 million medical records (link in Chinese) dating back to the 1990s from other hospitals in China. The tool has an accuracy rate of over 90% for diagnoses for more than 200 diseases, it said.“

https://qz.com/1244410/faced-with-a-doctor-shortage-a-chines...

Hi, author here:

Well first off: letters to investors are among the most biased pieces of writing in existence.

Second: I'm not saying connectionism did not succeed in many areas! I'm a connectionist by heart! I love connectionism! But that being said there is disconnect between the expectations and reality. And it is huge. And it is particularly visible in autonomous driving. And it is not limited to media or CEO's, but it made its way into top researchers. And that is a dangerous sign, which historically preceded a winter event...

I agree that self-driving had/have been overhyped over the previous few years. The problem is harder than many people realize.

The difference between the current AI renaissance and the past pre-winter AI ecosystems is the level of economic gain realized by the technology.

The late 80s-early 90s AI winter, for example, resulted from the limitations of expert systems which were useful but only in niche markets and their development and maintenance costs were quite high relative to alternatives.

The current AI systems do something that alternatives, like Mechanical Turks, can only accomplish with much greater costs and may not even have the scale necessary for global massive services like Google Photos or Youtube autocaptioning.

The spread of computing infrastructure and connectivity into the hands of billions of global population is a key contributing factor.

> The difference between the current AI renaissance and the past pre-winter AI ecosystems is the level of economic gain realized by the technology

I would argue this is well discounted by level of investment made against the future. I don't think the winter depends on the amount that somebody makes today on AI, rather on how much people are expecting to make in the future. If these don't match, there will be a winter. My take is that there is a huge bet against the future. And if DL ends up bringing just as much profit as it does today, interest will die very, very quickly.

Because there is a dearth of experts and a lack of deep technical knowledge among many business people, there are still a great many companies that have not yet started investing in deep learning or AI despite potential profits based on current technology. Non-tech sectors of the economy are probably underinvesting at the moment.

This is analogous to the way electricity took decades to realize productivity gains in the broad economy.

That said, the hype will dial down. I am just not sure the investment will decrease soon.

While I agree there is underinvestment in non-tech sectors, I don't see why that would change and they will use deep learning. There are lots of profitable things in non-tech sectors that can be done with linear regression but not done.

There are lots of things in the non-tech sector that can be automated with simple vanilla software but isn't. To use AI instead, you need to have 1) sophisticated devs in place, 2) a management that gets the value added, 3) lots of data in a usable format, 4) willingness to invest & experiment. Lots of non-tech businesses are lacking one if not all of these.

This. And at the end of the day, deep learning is just a more sophisticated version of linear regression. (To listen to some people talking, you'd think if a machine just curve-fits enough data-points, it'll suddenly wake up and become self-aware or something? The delusion is just unbelievable!)

Just like it did for ecommerce? Expectations we're wildly inflated, there was a bust, the market re adjusted and value was created

Only after the dot-com bust killed off all the weaklings ... there was a brief "cold-snap" after 2001 ... then Web 2.0 happened.

So I guess we're waiting for something similar to happen with AI and then get AI 2.0?

> I agree that self-driving had/have been overhyped over the previous few years. The problem is harder than many people realize.

The current road infrastructure (markings, signs) has been designed for humans. Once it has been modernized to better aid the self-driving systems, we don't probably need "perfect" AI.

But current signs designed for humans work well. They're machine readable (traffic sign detection is available from basically all manufacturers) and can (usually) be understood without prior knowledge and don't need much change over decades. I think there are few examples where messages were designed for computers but are easy to understand independent of the system, manufacturer. ASCII encoded text files are the only thing that come to mind.

How can something be biased when it's listing facts?

Those are actual features that are available today to anyone, that were made possible by AI. Do you think it would be possible to type "pictures of me at the beach with my dog" without AI in such as short time frame? Or to have cars that drive themselves without a driver? These are concrete benefits of machine learning, I don't understand how that's biased.

How can something be biased when it's listing facts?

If there are 100 facts that indicate a coming AI winter, and Brin just talks up the 15 facts that indicate AI's unalloyed dominance, that's definitely biased.

First, what are said 100 facts? The article looks at fairly mundane metrics such as number of tweets or self-driving accidents...

Second, I'm not quite sure that's how it works. Like in mathematics, if your lemma is X, you can give a 100 examples of X being true, but I only need a single counter-example to break it.

In my opinion a single valid modern use-case of AI is enough to show that we're not in an AI winter. By definition an AI winter means that nothing substantial is coming out of AI for a long period of time, yet Brin listed that Google alone has had a dozen in the past few years.

> First, what are said 100 facts?

You cannot ask a generic question, then attack the answer based on absence of evidence for a specific example.

Can't speak for the other items, but these

>translate over 100 languages in Google Translate;

>caption over a billion videos in 10 languages on YouTube;

barely even work. Yeah, it's a difficult problem but it's not even close to being solved.

YouTube captioning in English works surprisingly well, the improvement over the last few years is huge. It still chokes on proper nouns but in general it mostly works.

Mh no it does not. It is just a source of hilarity apart from a few very specific cases (political speeches mostly, because of their slow pace, good english and prononciation I guess).

Every time I activate it I am in for a good laugh more than anything actually useful.

I think it's a bit like self-driving cars in the sense that it's good enough to be impressive but not good enough to be actually usable everywhere. Of course self-driving is worse because people seldom die of bad captions.

Google's captioning works well when people speak clearly and in English. Google translate works well when you translate well written straightforward text into English. It's impressive but it's got a long way to go to reach human grade transcription and translation.

I think when evaluating these things people underestimate how long the tail of these problems is. It's always those pesky diminishing returns. I think it's true for many AI problems today, for instance it looks like current self-driving car tech manages to handle, say, 95% of situations just fine. Thing is, in order to be actually usable you want something that critical to reach something like 99.999% success rate and bridging these last few percent might prove very difficult, maybe even impossible with current tech.

What's important to remember, I think, is that we should not compare YouTube auto captions to human made captions, because auto captions were not created as a substitute for human made captions - if it wasn't for auto captioning, all these videos wouldn't get any captions at all. They may never be perfect, but they're not designed to be, they're creating new value on their own. And IMO they crossed the threshold of being usable, at least for English.

It works for general purpose videos. Transcripts of any kind appear to stop working whenever there's domain knowledge involved. That doesn't matter for most youtube videos but is crucial if you want to have a multi purpose translator/encoder.

A. Cooper had a nice example of this kind: a dancing bear. Sure, the fact that bear dances is very amusing, but let it not distract us from the fact that it dances very very badly.

Google Translate is way, way better than it used to be (at least German > English which I suppose is probably an easier task than many languages).

Google Translate of Goethe:

    Have now, ah! Philosophy,
    Law and medicine,
    And unfortunately also theology
    Thoroughly studied, with great effort.
    Here I am, I poor gate!
    And I'm as smart as before

word for word not completely bad, but then it breaks when we have to translate 'Tor'. Google Translate is clueless because it is unable to derive that here the 'fool' is meant.

It's unable to 'understand' that 'I poor gate' makes no sense at all.

Google Translate is the 'poor gate'.

On the other hand Deepl gives translations for news articles that are of such quality that it allows me to read international news as if it were local. Definitely useful.

DeepL is better, but it basically has the same problems. I understand both German and English - and I can easily detect where DeepL also shifts the meaning of sentences, sometimes even to mean the opposite. DeepL like Google Translate has no concept for 'is the meaning preserved?'.

You may think that you can now read German news, but in fact you would not know if the sentence meaning has been preserved in the English translation. The words itself might look as if the sentence makes sense - but the meaning is actually shifted - slight differences, but also possibly the complete opposite.

The translation also does not give you any indication where this might be and where the translation is based on weak training material or where there is some inference needed for a successful translation.

Not sure if I agree. If you have some knowledge of the language is still mostly easier to translate the words you don't understand. Easy texts work, anything more complicated (e.g. science articles) not really.

Hi, why in your analysis you spoke only about the companies that are not doing so well in self driving leaving out waymo success story? They are already have been hauling passengers without a safety pilot since last October. I guess without the minimum problem otherwise we would have heard plenty in the news like it happened for Tesla and Uber accidents. Is it not too convenient to leave out the facts that contradict your hypothesis?

Is this happening inside google campus or in a city?

In Arizona, on the streets: https://arstechnica.com/cars/2018/02/robotaxi-permit-gets-ar...

Where are the Waymo cars running? Everywhere? Are they still veering into buses to avoid sandbags?

Phoenix, and they plan to start a commercial service by the end of the year in several US cities.

" letters to investors are among the most biased pieces of writing in existence. "

Maybe true but they are words that are about things which are either true or not true. Has nothing to do where the words were shared. Saying they are on an investment letter so not relevant seems very short sighted.

But just looking at the last 12 months it is folly to say we are moving to a AI winter. Things are just flying.

Look at self driving cars without safety drivers or look at something like Google Duplex but there are so many other examples.

Of course Google (or any other company) aren't going to blatently lie in a letter to investors (that kind of thing gets you sued) but it's pretty easy to spin words to sound more impressive than they may actually be.

Using the list provided, one example

"caption over a billion videos in 10 languages on YouTube;" - This doesn't say how accurate the captions acutally are. In my experience youtube captioning even of english dialect isn't exactly great. For one example try turning on the captions on this https://www.youtube.com/watch?v=bQJrBSXSs6o

so it's true I'm sure to say they've captioned the videos AI based techniques, but that doesn't mean they're a perfected option.

Also (purely anecodtally) Google translate also isn't exactly perfect yet either...

... I don't think I understand that video even with my own ears. YouTube captioning has actually significantly improved from it's previous hilarious state

Making cars that drive safely no current, busy roads is a very difficult task. It is not surprising that the current systems do not do that (yet). It is surprising to me how well they still do. The fact though that my phone understands my voice and my handwriting and does on the fly translation of menus and simple requests is a sign of a major progress, too.

AI is overhyped and overfunded at the moment, which is not unusual for a hot technology (synthetic biology; dotcoms). Those things go in cycles, but the down cycles are seldom all out winters. During the slowdowns best technologies still get funding (less lavish, but enough to work on) and one-hit wonders die, both of which is good in the long run. My friends working in biology are doing mostly fine even though there are no longer "this is the century of synthetic biology" posters at every airport and in every toilet.

When I saw the Google demo of a CNN using video to split a single audio stream of two guys talking over each other, I became a believer.

Do you have a video of this?

https://youtu.be/ogfYd705cRs?t=6m54s

Edit: also, a blog post with more examples and a link to the related publication: https://ai.googleblog.com/2018/04/looking-to-listen-audio-vi...

is it this: https://www.youtube.com/watch?v=WnzlbyTZsQY perhaps ?

Sorry, I went to bed!

The original google report was discussed here a few weeks ago:

https://news.ycombinator.com/item?id=16813766

Hey, a small advice for the future: never build your belief entirely on a youtube video of a demo. In fact, never build your belief based on a demo, period.

This is notorious with current technology: you can demonstrate anything. A few years ago Tesla demonstrated a driverless car. And what? Nothing. Absolutely nothing.

I'm willing to believe stuff I can test myself at home. If it works there, it likely actually works (though possibly needs more testing). But demo booths and youtube - never.

Your advice would be a lot more convincing if you had a youtube video of a demo to come with it. Just saying. :P

You can't test most of high end physics at home. I hope that doesn't mean you don't believe it!

A lab publishes an exciting result, what does the physics community do? Wait for replication. Or if you’re LHC, be very careful what you publish.

In theory, there are two separate experiments (ATLAS and CMS) on LHC because of this. In practice, they are probably not independent enough.

It's probably a good idea to ignore high-end physics results for a few years, though.

The BICEP2 fiasco is a good example why.

A rigorous evaluation with particular focus on where it doesn't work would be better.

> understand images in Google Photos

This is one of the areas I’m most enthusiastic about but … it’s still nowhere near the performance of untrained humans. Google has poured tons of resources into Photos and yet if I type “cat” into the search box I have to scroll past multiple pages of results to find the first picture which isn’t of my dog.

That raises an interesting question: Google has no way to report failures. Does anyone know why they aren’t collecting that training data?

They collect virtually everything you do on your phone. They probably notice that you scroll a long way after typing cat and so perhaps surmise the quality of search results was low.

Doesn’t that seem like a noisy signal since you’d have to disambiguate cases where someone was looking for a specific time/place and scrolling until they find it?

I’ve assumed that the reason is the same as why none of the voice assistants has an error reporting UI or even acknowledgement of low confidence levels: the marketing image is “the future is now” and this would detract from it.

> understand

what is this 'understand'?

Well, the way I see it: mostly, these are “improvements”, huge ones, but still. They ride the current AI tech wave, take it an optimize apps with it.

For most things, that people dream of and do marketing about need another leap forward, which we haven’t seen yet (it’ll come for sure)

Almost anything that has to do with image understanding is entirely AI. Good luck writing an algorithm to detect a bicycle in an image. This also includes disease diagnostic as most of those have to do with analyzing images for tumors and so on.

Also, while a lot of these can be seen as "improvements", in many cases, that improvement put it past the threshold of actually being usable or useful. Self-driving cars for example need to be at least a certain level before they can be deployed, and we would've never reached that without machine learning.

I agree, the effects can be very impressive. I meant, that what is achievable is quite clear now and that we need a major innovation/steps for the next leap

>caption over a billion videos in 10 languages on YouTube;

Utterly useless. And I don't think it is improving.

I find the auto captions pretty useful.

This is less useless than you think. Captioning video could allow for video to become searchable as easily as text is now searchable. This could lead to far better search results for video and a leap forward in the way people produce and consume video content.

They mean useless in the end result. Of course having perfect captions could potentially allow indexable videos, but the case is that the captions suck. They're so bad in fact that it's a common meme on Youtube comments for people to say "Go to timestamp and turn on subtitles" so people can laugh at whatever garbled interpretation the speech recognition made.

Have you used/tried them recently? The improvement relative to 5 years ago is major.

At least in English, they are now good enough that I can read without listening to the audio and understand almost everything said. (There are still a few mistakes here and there but they often don’t matter.)

Yes I’ve had to turn them off on permanently. Felt I could follow video better without sound often than with subtitles.

I tried to help a couple channels to subtitle and the starting point was just sooo far from the finished product. I would guess I left 10% intact of the auto-translation. Maybe it would have been 5% five years ago; when things are this bad 100% improvement is hard to notice.

It is super cool how easy it is to edit and improve the subtitles for any channel that allows it.

>understand almost everything said...

If by 'almost everything', you mean stuff that a non native English speaker could have understood anyway, then yes.

I'd say the current Youtube autocaptioning system is at an advanced nonnative level (or a drunk native one :)) and it would take years of intensive studying or living in an English-speaking country to reach it.

The vast majority of English learners are not able to caption most Youtube videos as well as the current AI can.

You underestimate the amount of time required to learn another language and the expertise of a native speaker. (Have you tried learning another language to the level you can watch TV in it?)

Almost all native speakers are basically grandmasters of their mother tongue. The training time for a 15-year-old native speaker could be approx. 10 hours * 365 days * 15 years = 54,750 hours, more than the time many professional painists spent on practice.

Not true. The problem with Google captioning and translate is that unlike a weak speaker it makes critical mistakes completely misunderstanding the point.

A weak speaker may use a cognate, idiom borrowed from their native tongue or a similar wrong word more often. The translation app produces completely illegible word salad instead.

I was talking exclusively about auto-captioning, which has >95% accuracy for reasonably clear audio. Automatic translation still has a long way to go, I agree.

To be honest, as the other child comment said, I too have noticed they have gotten way better in the last 5 years. Also, the words of which it isn't 100% sure are in a slightly more transparent gray than the other words, which kind of helps.

I think he is stating that the quality of the transcription is poor.

You don't need amazing transcription to search a video. A video about X probably repeats X multiple times, and you only really need to detect it properly once.

As for the users, sure the translation may not be perfect, but I'm sure if you were deaf had no other way of watching a video, you would be just fine with the current quality of the transcription.

Often you need exactly that. Because it's the unique words the machine will get wrong. If you look for machine learning tutorials/presentations that mention a certain algorithm, the name of it must be correctly transcribed. At the moment, it appears to me that 95%+ of words work but exactly the ones that define a video often don't. But then again getting those right is hard, there's not much training data to base it on.

I disagree, even with the high error rate, it provides a lot of context. Also, a lot of comedy.

Reply on news.ycombinator.com