Douglas Hofstadter: The Shallowness of Google Translate (2018) | Hacker News Reader

Douglas Hofstadter: The Shallowness of Google Translate (2018)

theatlantic.com

95 pointsandrepd82 comments

> that human translators may be an endangered species. In this scenario, human translators would become, within a few years, mere quality controllers and glitch fixers, rather than producers of fresh new text.

Huh? As someone who's worked in translation briefly before, that's already the case and has been for at least a decade -- at least for written material that's not "art" (literature, poetry).

It's far faster for professional translators to run something through Google Translate or similar and then clean it up, than type it out from scratch. And identical quality in the end.

There's little "soul-shattering" about this at all. Someone who needs to churn out translations of UN meeting notes day after day isn't doing it for their soul -- they're doing it because it's their job and people of different nationalities can find out what happened in the meeting to do their job.

I'm a little surprised to learn that GT is used to translate UN meeting notes, because the right kind of mistake in that context can literally lead to a costly, possibly catastrophic, international incident.

Sorry, my experience was at professional translation companies -- I don't know about actual processes at the UN, was just using it as an illustration that translating isn't usually for "art".

But I don't see why it should be any different. It's still being cleaned up.

The translator knows what they want in the end so the result is the same whether they start with automatic translation or not.

Plus, where I worked all translations were always then reviewed by a second translator anyways, because people miss things too -- we're only human.

Automatic translation as a first step just speeds things up. It doesn't change the final quality at all.

That’s true...people are lazy and costs want to fall, but it’s a little disconcerting in areas that require nuance and detail.

I’m by no means a translator, but it isn’t difficult for me to find sentences where Google Translate simply butchers the nuance of Japanese/English conversions. Things like politeness and tone? Gone.

Maybe it’s better for closer language pairs, but if a professional translator starts with automated translations for an important document, they’re taking a huge risk. Even as an intermediate speaker, the things I am good at (concept understanding, idomatic speech, politeness, tone) are exactly the things that translation tools are bad at, and correcting the errors often just requires re-writing whole paragraphs from scratch.

That's why I said written texts. You're right, it's not up to the task for dialog.

And I'm not aware of automatic translation used professionally for dialog (e.g. movie subtitles) because you're right -- there it's primarily nuance and tone.

I’m talking about written text. Even for humdrum stuff like business emails, the lack of nuance and contextual understanding is crippling for JP/EN translation.

Ha, OK -- "prepared" written text maybe? "Formal" written text? I've never come across people paying for translating business emails. :)

I'm talking about the vast majority of what translating companies are paid to translate -- articles, contracts, reports, specifications, meeting summaries, and so on -- whatever you want to call that. Thanks for helping me to clarify.

I actually had a start-up idea circa 2005 for translating business emails between Japanese and English. The idea (and please feel free to use it for whatever it's worth now) was to constrain the emails into the set phrases derived from MINNA NO NIHONGO, where you could only choose the sentence type and fill in the nouns and verbs.

eg: “Do you think <NOUN> will <VERB>?”

would be translated to: “<NOUN>は<VERB>でしょうか?”

and back in the other direction. The emails would be constructed from a sequences of various sentence types.

It would be a sort of highly constrained “business language”, but of course with a nice UI(!)

Google Translate is spectacularly bad at Japanese/English. I received an email yesterday where someone who is visiting me said (in translation) that she was looking forward to eating "my rice" (proper translation: my food), which is such a basic thing I can't believe that Google still gets it wrong.

Definitely. Happens so often that I always warn new students off of using GT as a dictionary (which is a super common thing lately).

I use GT to figure out the most probable mapping for words and simple grammar points when there are multiple dictionary options, or to give me kanji readings. It still screws those up often enough that I don’t trust it.

As a counter example, GT is surprisingly good at English to Russian translation: my sister’s husband knows no Russian, but is able to participate in family iMessage chats with no issues. Most of the time I can’t tell his messages have not been written by a native Russian speaker. He is able to fully participate in a written conversation. Mind boggling!

The laziness is the main harm that Google translate creates. I wish they would just attach the translated words to the original sentence and let the translator assemble the correct version. Then when you see an odd goofball word you click on correct version (if possible) so that way the program might improve slowly. Off course the translator needs to know the grammar of the language itself and that you probably need to reassemble the sentence. You might not always get the grammar right but at least the meaning might be preserved.

It is also faster for professional programmers to run something through stack overflow and paste the top voted answer into their code.

Not being a translator I wonder how good of an analogy this is

Not a very good analogy since the translator knows the correct translation before hand and therefore can identify the incorrect translation produced, whereas someone using stack overflow does not know the correct answer before hand.

Not a translator, but a tri lingual person here. Often the decision what is a "correct" translation is not simple or unambiguous. As to the use of stack overflow, we all have our own experiences.

Its a little hazier than that. Someone using stack overflow might not know the exact answer (else why look) but may know enough to choose between answers and recognize the one most likely to solve their problem.

>In this scenario, human translators would become, within a few years, mere quality controllers and glitch fixers, rather than producers of fresh new text. Such a development would cause a soul-shattering upheaval in my mental life. Although I fully understand the fascination of trying to get machines to translate well, I am not in the least eager to see human translators replaced by inanimate machines. Indeed, the idea frightens and revolts me. To my mind, translation is an incredibly subtle art that draws constantly on one’s many years of experience in life, and on one’s creative imagination.

Well, there's another way to achieve the same effect, by turning anything written or spoken into such trivial BS, that any old translation will do.

When nuance is not appreciated or delivered, then you don't need to translate nuance. When literature is replaced by skimming online articles, there's no much nuance to begin with.

I don't think you're wrong and there is still many valid use cases for machine translation.

There are many situations where text should not be nuanced or contain subtlety, like business signage or instructions. It should be clear and direct, however people often want to convey frustration either actively or passively. Often people want to be playful around awkward or uncomfortable messaging. And there are times when they want to appear humble.

Shallowness is one obvious problem (obvious to any bilingual person). The more troubling problem I’ve noticed is that occasionally it’s inexplicably outright wrong about basic things.

Anecdote time. One day in late 2017 (I remember that because of an event), a friend of mine read a Google-translated page in Chinese, and sent it to me (fluent in Chinese) because a statistics was “amazing”. I looked at it, turned out Google translated 万, meaning “ten thousand”, to “million”. Of course the number was unbelievably good. I honestly couldn’t think of how that happened.

I don't know about Chinese, but in Japanese, 万 is frequently used idiomatically in the same way we might in English use "a million"—just to mean "a whole hell of a lot."

You’re right, except it was a statistic like 2.8万, translated to 2.8 million. Your take could point to a possible cause, though, maybe also putting it under the “shallow” category.

(Additionally, when 万 is used in Chinese to mean a whole lot, I would translate it to “thousands of” or “tens of thousands of”.)

Myriad might be the perfect translation then. It comes from Greek for 10,000 and is now routinely used for "a lot" in English.

I once google-translated a russian technical document and it replaced all occurances of "мкВт" (microwatt) with "mW" (milliwatt). Took me a while to figure out why all stated powers were way off.

I get errors like this from Google Translate all the time. I assume it's because the engine is fully ML (which seems to be good at getting larger grammatical structures but sometimes mistakes individual words), as opposed to some kind of expert system approach (which can easily look up individual words but tends to make soup out of them).

As for the error you saw, here's a guess: a million yen is ten thousand dollars, so an ML agent trained on JP/EN text that discusses money might see those numbers crop up a lot in close proximity.

My favourite glitch in Translate happened around 2008, where in the en/ru pair it translated Bush as Путин (Putin) and Putin as Буш (Bush). The reason was apparently that in the training corpus, where the English version said something like "Bush met with Putin", the Russian version tended to say "Putin met with Bush"

"pelosi met with trump" is translated incorrectly to Russian as "Пелоси встретился с Трампом"

The Russian verb is incorrectly in masculine form. Interestingly both names are capitalised in translation

The verb is "incorrectly" used only if you assume that "pelosi" implicitly refers to female politician "Nancy Pelosi". You might infer that if Nancy was recently in the news, but that's a hard leap.

Don't need to assume anything, it was my text snippet and pelosi did refer to a female polition.

A contemporary human translator would reasonably default to the feminine form without further context.

The trick is not to rely on Google translate for a finished translation, but to use it just as a tool toward reaching that goal.

As someone who is learning French these days, it is an absolute boon.

One of my favorite things is letting Google auto finish a sentence.... because those auto finished sentences usually tend to be phrases or constructs French speakers use a lot.

The other neat thing is that Google shows a check mark near a translation which it feels with high confidence is good — in my experiences those translations really are good and without fault.

> The trick is not to rely on Google translate for a finished translation, but to use it just as a tool toward reaching that goal.

This assumes the use case is a translator fluent in the source and target languages translating a passage. The far more common use cases for Google translate are people with near-zero knowledge of either the source or target language and little chance of completing the task successfully.

He addresses that:

>Let me return to that sad image of human translators, soon outdone and outmoded, gradually turning into nothing but quality controllers and text tweakers. That’s a recipe for mediocrity at best. A serious artist doesn’t start with a kitschy piece of error-ridden bilgewater and then patch it up here and there to produce a work of high art.

For somebody learning a language, they would be even more easily mislead.

It's faster than using a dictionary to look up words one-by-one, and Google's actually integrated a dictionary into the interface. And beginner sentences are well-represented in the corpus so they're relatively unlikely to mislead.

Using Google Translate for learning a language could be likened to using training wheels on a bike.

The training wheels dont randomly fall off of bikes. Either a great analogy counter to your point, or a terrible one.

Did somebody say "shallow"?: https://photos.app.goo.gl/wDy8TQR69iwuGUZb6

The intersection of a donkey and a horse is called a mule or a mule . The mule father is a donkey and the mother is a horse cat, while the mule father is a horse and the mother is a donkey cat. Both junctions are almost always sterile.

At the age of two, stallions are driven away from the herd and form their own herds by collecting oaks.

https://translate.google.com/translate?sl=auto&tl=en&u=https...

There are four species of wild deer in the wild: deer , wild deer , deer and white-tailed deer . Elk are by far the largest of these species. All species of Finnish deer are game animals . Despite its name, spruce deer do not belong to deer but to deer [2] [5] .

It has also been referred to as the deer [4] [5], to distinguish it from the deer ( Cervinae ). The Mammalian Nomenclature Committee has proposed that the sub-tribe be renamed in Finnish to the goats .

https://translate.google.com/translate?hl=&sl=fi&tl=en&u=htt...

The subspecies of Canis aureus lupaster, previously defined as a subspecies of live cabbage in North Africa, originates from South Asian gray matter based on DNA analyzes published in 2011. [18] [19] The North American red wolf ( Rufus ) has also been the subject of scientific debate, for example as a cross between coyote and gray matter.

The same genus of wolves as the wolf canis is the goldfish, the warbler , the warbler , the Ethiopian wolf and the coyote , and the dingo , which is nowadays generally classified as a wolf subspecies.

The survival of young kittens depends a lot on how many adult wolves and babysitters remain in the herd.

Less than half of the wolf catching attempts are successful. A deer or a creepy male stuck in a threatening position is likely to survive, but the fugitive gets the wolves to his feet.

The wolf can carry loose pieces of pregnant mother and puppies in the nest or vomit the contents of their stomachs into the puppies.

In Central Finland, for example, the formation of new worms has been observed to have led to a reduction in foxes and raccoon dogs and, as a result, to an increase in woodland chicks. [40] This behavior, on the other hand, exposes wolves to infectious diseases in small beasts , especially the wardrobe .

https://translate.google.com/translate?hl=&sl=fi&tl=en&u=htt...

Whales are also reared as fur animals, known as the blue fox.

The bean is most active in the twilight, when it comes up in search of food. Its main food is the fells and, to some extent, the moles . In years when small rodents are scarce, the bean has to deal with birds, their eggs and chicks, and berries and carrion, for example.

The situation of the species is particularly poor in Scandinavia and Finland. However, globally, the whale is not endangered. According to a report published in 2008, the stock of nuts is gradually gaining ground in the Nordic countries .

In Finland, the whale is classified as extremely endangered . [5] As a popular fur animal , in the twentieth century, wild white fox was hunted almost extinct in Finland. Even in the early 20th century, the species was abundant in Lapland, and during the winter of 1908-1909, whales migrated all the way to the southern coast of the country.

Attempts have been made to improve the livelihood of Naal by hunting foxes and providing the nipples with nourishment and other food. [12] [13] In Norway and Sweden, dogfish are fed on dogfish, which has contributed to the growth of the stock. [7] The survival of nails is also hampered by the fact that they are no longer provided with enough reindeer kills for wolves and wolves.

https://translate.google.com/translate?hl=&sl=fi&tl=en&u=htt...

fi->en translations are so funny that I read them when I'm feeling down. Few other things make me laugh out loud so much.

Part of me hopes they never invent a better way to translate.

It's like an algorithmic "English as she is spoke" :p

https://en.wikipedia.org/wiki/English_As_She_Is_Spoke

I’ll equate this to being critical of any early technology as if no further development will happen.

Translation, good translation, is not easy. If it were, it would have been a solved problem years ago. And, yes, some languages pose more challenges than others.

I often have to write the same text in multiple languages. I usually write in English and use Google Translate to quickly generate a rough draft translation in other languages, say, Spanish. I then copy-paste the translation, proofread and edit. It is a huge time saver. Aside from not having to re-write the material, the cognitive load of the proofreading and editing process is significantly lower.

I guess the right things to take from the article are (1) the validity of the criticism, as a legitimate face value observation, but also as a way of sketching the kinds of problems and areas where we hope for progress and (2) the fact that Google Translate is nevertheless useful in its current form.

I think his overall thesis isn't a declaration that no further progress will be possible, but that it may have to draw from resources that are more than just statistical associations of words - things like interior human experiences. So it would not just measure words against surrounding passages, but against some understanding of concept generation that emerges from human subjectivity.

And, this is where I think a fuse blows for many people, things get complicated, circuits fry, and people find themselves filled with an urge to declare"well, computers just CAN'T do X." But I think what Hofstadter is saying is that it is a hard problem but also the kind of thing that in principle could be done to make progress on translation.

Achieving human-level understanding is the canonical problem in AI. I think I can say we don’t yet know how to encode and represent this thing called understanding.

We are excellent at classification and sub-par at understanding, with translation being just one of the many application domains that will benefit when we go through the understanding inflection point.

[edit: see ~gleinstein's sibling comment – it's what i was trying to say, but articulated much better]

> I’ll equate this to being critical of any early technology as if no further development will happen.

i read it as saying that the current approach to translation, while a useful approximation, can't ever really work, not because it's "early in its development", but because it's fundamentally not enough – the software needs some actual "understanding of what is being said" to accurately translate. from TFA:

> The practical utility of Google Translate and similar technologies is undeniable, and probably it’s a good thing overall, but there is still something deeply lacking in the approach, which is conveyed by a single word: understanding. Machine translation has never focused on understanding language. Instead, the field has always tried to “decode”—to get away without worrying about what understanding and meaning are.

[after showing some instances where Translate fails:]

> We humans know all sorts of things about couples, houses, personal possessions, pride, rivalry, jealousy, privacy, and many other intangibles that lead to such quirks as a married couple having towels embroidered “his” and “hers.” Google Translate isn’t familiar with such situations. Google Translate isn’t familiar with situations, period. It’s familiar solely with strings composed of words composed of letters. It’s all about ultrarapid processing of pieces of text, not about thinking or imagining or remembering or understanding. It doesn’t even know that words stand for things.

> Let me hasten to say that a computer program certainly could, in principle, know what language is for, and could have ideas and memories and experiences, and could put them to use, but that’s not what Google Translate was designed to do.

https://deepl.com seems to be better at this. I've long been looking for differences, dreading that Google (with its superior amount of money, larger dataset, and perhaps also a larger talent pool) would outperform it on something. It has always been on par so far, often giving word-for-word the same translation. When trying the author's example, however, Deepl actually does it correctly (also outperforming Microsoft Translate, which I threw in for good measure): the manually-translated french turns into correct English. (Not speaking French, I can't judge the french translation it makes.)

Now I'm reading this article: https://news.ycombinator.com/item?id=21559633 and this made me curious again:

> However, in German, there is no expectation at all that the subject must come first (although it often does). These two German sentences share the same meaning: "Der Hund hat den Ball." and "Den Ball hat der Hund."

It totally trips up Google (Den ball hat der Hund -> "the ball has the dog") but Deepl has this sort of silliness for breakfast (Den ball hat der Hund -> "the dog has the ball").

Of course, this is not negating the point of the article: translating is interpreting (or "art", as the author puts it), so deepl still can't be as good as a human. But for edge cases, deepl does seem to be better, at least in these two examples. In real-world cases that I tried so far, as mentioned, I've found it to be 100% equivalent so far (n=20 or so).

From my own experience, I’ve found that Google can be a useful tool. But you have to understand the difficult bits of each language you’re translating between and specifically write around them, making the translations tool job a lot easier. E.g. specifically avoid idioms, words with double meanings, etc.

Hofstadter is trying to draw a conceptual distinction between "understanding" (generating language by drawing from a full interiority of human subjective experience), and mere "translation" (based on looking at words).

But analyzing how words are used is a great way of working backward into something that gets closer and closer to human subjectivity. Some neural net trained on how words are used would start to model concepts that reflect "under the hood" human experiences that generate language.

Maybe it's not 'enough', and a robust translation engine would have a sense of irony, jealousy, favorite foods, etc that it would need to rely on to translate things, and maybe that set of concepts to be obtained via something other than training on words. But I don't think there's a clear conceptual line to be drawn. Studying the way we use words can get us a loose approximation, maybe even a good one.

The Chinese example at the end where he had to look up the phrase himself (on Google) is particularly revealing. Humans don't have any more insight into words than the machine. They're just better at interpreting explanations written by humans and intended for humans.

I lately think the auto translation on youtube is better than on google.

If I'm not wrong, this is a continuation of Hofstadter's critique of the "new AI" (i.e., AI boom since 2006 based on machine learning) and how it is at odds with his conception of what needs to be done in order to move towards AGI based on ideas surrounding his book Godel, Escher, Bach [1].

For context, please read this in light of the 2013 piece in The Atlantic [2].

[1] https://en.wikipedia.org/wiki/G%C3%B6del,_Escher,_Bach

[2] https://www.theatlantic.com/magazine/archive/2013/11/the-man...

After reading this I can't help but think (due to my fascination with the subject) of the parallels one can observe in modern day polarized political discourse on social and other forms of media.

> Whenever I translate, I first read the original text carefully and internalize the ideas as clearly as I can, letting them slosh back and forth in my mind. It’s not that the words of the original are sloshing back and forth; it’s the ideas that are triggering all sorts of related ideas, creating a rich halo of related scenarios in my mind. Needless to say, most of this halo is unconscious. Only when the halo has been evoked sufficiently in my mind do I start to try to express it—to “press it out”—in the second language. I try to say in Language B what strikes me as a natural B-ish way to talk about the kinds of situations that constitute the halo of meaning in question.

In any communication between two or more people, verbal or written, there is an ever present translation process going on. Political (and other similar) conversations, despite the communication taking place in the same language (or so we conceptualize it), are subject to the very same AI translation problem Hofstadter notes: the translation of words into ideas.

In his case, Hofstadter takes his time, reading the text carefully, "internalizing the ideas as clearly as I can" (~steelmanning). He is explicitly aware of the mind's behavior (ideas, triggering (an incredibly complex and poorly understood process) all sorts of related ideas, creating a rich halo of related scenarios), and that "most of this halo is unconscious". He takes his time, letting the subconscious mind do it's thing. And only then, "when the halo has been evoked sufficiently in my mind", does he put pen to paper to finalize the task of performing an as accurate as possible translation.

You can see this same sort of thing happening all over the place in the real world, except unlike Hofstadter, most people are unaware of what is happening under the covers, and lack the explicit and conscious intent to perform an accurate (as possible) translation before setting their fingers on the keyboard to have their say, which will then in turn be inaccurately interpreted (translated from words to ideas, and then processed) by readers, each in their own unique way due to the nature of their personal heuristics and mental model. "Round and round she goes, where she stops, nobody knows."

I'd love to find someone of Hofstadter's background and capability who is focusing precisely on this phenomenon. The closest I've come across so far is Jonathan Haidt, and a few people like Eric Weinstein (who often touches on these ideas, but is focused on them to a much lesser degree than Haidt). Any recommendations of others to look into would be appreciated, as would any words of encouragement from people who believe there is actually something important going on here. I believe this, a lack of true mutual understanding of the ideas and beliefs of others, and the lack of intent or desire to do so (if not revulsion toward the very idea (in my perception, speaking of sloshing ideas)), is one of the fundamentally important and mostly overlooked problems in the world today.

Douglas Hofstadter is a pompous buffoon. His style is infuriating, pushing his undoubtedly fine intellect in the reader's face. He writes for people who want to believe themselves smarter than most, and the only cost of admission is that they validate Hofstadter's high self-opinion.

We all know by the end of the article that Hofstadter is a better translator than Google Translate, but nobody believes that Google Translate is going to match the work of someone who understands the text they are translating. Hofstadter claims he's not cherry-picking difficult examples, but he has chosen two passages where gender is critical, and moved them between languages where gender is expressed in very different ways. It's a tricky part of translation, as Hofstadter knows, and nobody expects a computer to speculate on the intentions of the author of a passage, as you must to successfully translate his examples.

Google Translate, plus some critical thinking and a bit of work with a dictionary, will get you a rudimentary understanding of a text in a foreign language --- and that's a great advance. Hofstadter expresses an expectation that no-one holds for an automatic translation tool, shows that Google Translate does not live up to it, and then is satisfied with himself, as usual.

I got something different out of the article other than offense at education envy.

Instead of an ad hominem, would you care to address the specific examples sighted in the article?

I did a bit, but honestly, my main point is that I can't stand Hofstadter's style, either of writing or thinking. It's not an ad hominem attack to prove some other point; it's intended as a criticism of the man himself.

I skimmed the article, but it seems the purpose of it would be better served as series of issue reports in the Google Translate bug tracking system than a full article.

I think it's not really that interesting to diss the state of the machine translation art in 2019, instead of looking where it could be in 2020, 2025, 2030.

>issue reports

You miss the point. The point is that the "deep" learning approach is fundamentally flawed and will not lead to proper translation no matter how many racks of GPUs you throw at it, since it lacks the "understanding of ideas" required to do it.

It comes off as quite arrogant of Hofstadter to imply that people believe translation has now been solved by deep learning. You don't even have to be bilingual and know the other language to detect the subtleties lost by machine translation. It would have been interesting to hear the reflections of the Danish speaking friend. I can imagine Frank finds Google Translate practical for simple words that actually are more or less 1-to-1, things you would otherwise look up in a dictionary. And probably his Danish friend can pick up any idioms and such that Google Translate distorts anyway.

I utterly fail to see the point. In all likelihood I would take offense and cut short any further communication, should someone attempt to talk to me through Google [or whatever] Translate (but then, I can barely stand plain Gmail).

For what it's worth, Danish actually is my native language.

When I visited Thailand last year most of the apps like Uber and Grab were using Google auto translate in the chats. Made communicating with the car drivers painless. Many people knock these auto translations but after knowing people that can't read English the major language of the internet. I can't look at the translations with the kind of disdain many here seem to feel. As these auto translations open the world to lot more people that were sidelined previously.

The practical utility of Google Translate and similar technologies is undeniable, and probably it’s a good thing overall, but there is still something deeply lacking in the approach, which is conveyed by a single word: understanding.

It all boils down to this paragraph, and no one can say it is not true.

You certainly can, and that is the biggest problem with this article: a completely lack of interest in understanding what deep learning understands (ironically).

Hofstadter, having completely missed the boat on connectionism, committed himself to a failed paradigm, and been wrong about AI time after time (remember when Hofstadter claimed beating humans at chess would require computers to have emotions and be able to write poems?), totally fails to grapple with what NNs do do. For starters, Google Translate is a free service and not necessarily SOTA, even when he wrote this. For example, if you were going to ask 'what do multi-lingual translation seq2seq RNNs (or Transformers) understand?', the immediate obvious thing to discuss would be questions about the 'interlingua' which their embedding yields, things that word embeddings learn (including surprising things like relative locations of cities or sizes of objects), or about transfer learning to tasks that require grammar or reasoning or common-sense like Winograd schemas like in GLUE or SuperGLUE, and how they improve massively over earlier approaches and how they are scaling (one of the most important trends in AI). There is a great deal which could be said here, demonstrating why it works so well and why it doesn't work sometimes, and an AI researcher is just the sort of person to ask.

And unfortunately, Hofstadter does none of this and instead bangs on about some errors he found, like it's a total blackbox and understanding is a binary where you either like the same poetry Hofstadter does or you are a cretin, and takes the attitude that any error shows that the approach has failed, is failing, and always will fail because it is 'deeply lacking' (what, exactly? And to think, he complains about 'deep' being abused for rhetorical purposes before and then goes and produces a shallow exercise like this).

I thought this was a lousy neo-luddite essay in 2018, and it's only gotten worse since then. In another 10 years, it'll look like Cliff Stoll.

I am not a specialist on AI, just had course on neural networks at university. Feel free to discard my opinion and do not reply if you find my way of thinking silly.

From what I see and understand, even if you have multiple layers of perceptrons and back propagation. It is still really "deeply lacking" because you are not able to train networks with all context there is. You can still overtrain network, so more training is making it actually worse. You can train NN only so much, just like you cannot teach dog to speak, it can perfectly fine understand your commands but you will never communicate with the dog on a level of "this painting is beautiful, you agree?".

I also do not believe in possibility of creating AGI.

> From what I see and understand, even if you have multiple layers of perceptrons and back propagation. It is still really "deeply lacking" because you are not able to train networks with all context there is.

I don't know what you're trying to say here. NNs can have plenty of context. Let's take just the Transformer architecture for concreteness, ignoring all the others. Something like GPT-2 uses a fixed window, but it can be expanded arbitrarily widely if you're willing to pay the compute price; the scaling can be improved within that window by not computing attention over the full window, per Sparse Transformers; recurrency can be added to make it more RNN-like, as does TransformerXL; compressed versions of the memory can be stored and fed back in, allowing lookback arbitrarily far, per Compressive Transformers; multi-modal input, like images + embeddings + text, are of course completely possible and can represent arbitrary metadata and context; there are many different ways to add in external memory stores (DeepMind regularly experiments with these, like the DNC); and so on. What exactly do you think is impossible in principle in the connectionist paradigm to make NNs do (and why do you think your brain is exempt from this)?

Ok you just skipped the second part where I say about overtraining NN's. You can put all the context you want from other sources and expand arbitrarily, but still artificial NN's are a lot more constrained than biological ones. It is because artificial NN's are still only approximation of biological NN's. We still don't know a lot about functioning of brain and other stuff.

Maybe some parts of image recognition require spinal cord neurons and muscle tissue to interact? So is it just count of inputs to neuron, or there are ways where inputs to neuron are modified by other chemicals in a body? Are we making weights on inputs stronger because of good reasons, are making some other inputs weaker because of good reasons?

Regarding the "failed paradigm" (i.e: of logic- and knowledge-based AI) and how neural nets scale, specifically on language modelling tasks.

I study a class of machine learning algorithms that learn logic programs (as in Prolog, or ASP) from examples. The field is Inductive Logic Programming. ILP algorithms are characterised by their ability to incorporate background knowledge and impose strong inductive biases on the hypothesis space to learn accurately from a handful of examples.

My favourite example of this is learning a grammar of the aⁿbⁿ language. My implementation of one of the algorithms that my research group studies (I'm a Phd research student) is Thelma:

https://github.com/stassa/thelma

In Thelma's README on github you will find an example of Thelma learning a grammar of the aⁿbⁿ language from three positive examples and no negative examples. The learned grammar generalises to any n. In typical CFG notation it's this grammar:

  S → AB
  S → AS₁
  S₁ → B
  A → a
  B → b

(The nonterminal S₁ is invented, i.e. it was not given in the original learning problem defintion. The preterminals A and B are background knowledge.)

By contrast, neural networks can only learn fragments of this grammar up to some limited n and that only if they're given tens of thousands of training examples.

For instance, a classic paper by Gers and Schmidhüber [1] claims LSTM "generalisation" from training sets of 22 to 42 thousand aⁿbⁿ strings, but their average generalisaion is, e.g. from n in [1,50] (i.e. that's the value of n in training strings) to n in [1,430] (n in testing strings) and their "best" generalisation is at most to n in [1,1000].

On Thelma's README on github I have a small testing query that tests how the aⁿbⁿ grammar it learned from 3 positive examples with n in [1,3] generalises to an aⁿbⁿ string of e.g. n = 100,000:

  ?- _N = 100_000, findall(a, between(1,_N,_), _As), findall(b, between(1,_N,_),_Bs), append(_As,_Bs,_AsBs), anbn:'S'(_AsBs,[]).
  true .

Well, it's a correct grammar so it generalises perfectly. You can bind _N to any number your computer memory will allow and it will still parse.

So, besides the plug of my work (sorry) I would say that the earlier paradigm that you say has "failed" with respect to recent connectionist success is alive and well and it still gets many things right that the connectionist paradigm struggles with mightily, in particular robust generalisation from very few examples (without expensive pre-training etc), and of course any task that requires reasoning [2].

______________

[1] "LSTM recurrent networks learn simple context free and context sensitive languages": https://ieeexplore.ieee.org/document/963769 See table 2 on page 7 for the results I quote above.

[2] The examples of good performance on reasoning tasks you bring up have been strongly criticised, e.g. in https://arxiv.org/abs/1907.07355 (Probing Neural Network Comprehension of Natural Language Arguments) or Gary Marcus' recent paper on a critical appraisal of deep learning etc.

P.S. I'm very sorry to see you're being downvoted.

The fact that logic programming is still looking at "languages" like a^nb^n while neural nets can generate at least somewhat sensible English only serves to underscore how utterly the field has failed. If you think that such simple toy problems are impressive, you really need to get out of your bubble.

Apologies if this sounds rude, but that sounds like sour grapes to me. It's not that learning anbn is not a toy problem. It is. But it's a toy problem for ILP. Neural nets can't solve it. And neither can any other statistical machine learning algorithms.

I'd say that it's rather statistical machine learning research that is in a bubble, satisfied to solve the same old classification problems again and again, apparently oblivious to the fact that there are many problems that their algorithms can't touch because they are neither differentiable, nor reducible to classification.

Unfortunately, since statistical machine learning is the current dominant paradigm, success is measured in terms of what it can do- and anything it can't do is either completely ignored or discounted.

Or, like I say: sour grapes.

> It's not that learning anbn is not a toy problem. It is. But it's a toy problem for ILP.

I don't see why the distinction you are making matters.

> Neural nets can't solve it.

They can, it takes additional training to do so from scratch. Also, the work you cited is also nearly two decades old, I have no idea how SOTA would perform.

> solve the same old classification problems again and again

I truly have no idea what you're talking about, please point me to where, for instance, [0] has been solved "again and again"?

The claim I am making is not that Deep Learning can solve all problems, it cannot, the claim is that logic-based AI has failed to come up with anything of value whatsoever, whereas statistical ML has had sweeping results across many real problems -- for instance: AlphaZero solving Go, Chess, and Shogi; GPT-2 creating readable stories given a prompt; cars that can drive themselves under a wide range of circumstances. If you think a^nb^n is even remotely comparable to those feats, I don't think we can find any common ground.

[0] https://www.youtube.com/watch?v=Wwwyr7cOBlU

Neural nets can still not solve aⁿbⁿ. This is the latest attempt of which I'm aware:

On the Practical Computational Power of Finite Precision RNNs for Language Recognition

https://arxiv.org/abs/1805.04908

Typically, it claims generalisation but fails to demonstrate it.

As we agreed, it's a simple problem. Neural nets can't solve it. Do you understand why?

Re: classification, AlphaZero is a great example of what I'm talking about. The game-playing logic is provided by MCTS, a GOFAI algorithm (it's a variant of good, old minimax). The neural net component is only used to identify promising board positions, i.e. for classification.

As to chess, it was already solved by DeepBlue using alpha-beta minimax and an opening book of moves- another example of GOFAI. Even AlphaGo, AlphaZero's predecessor used domain knowledge in the form of example games played by humans.

But I'm getting the feeling that, while you are making this extraordinary claim that "logic-based AI has failed to come up with anything of value whatsoever", you are not very familiar with the history of AI research.

As a quick reminder, logic-based AI was the dominant paradigm in research for some 60 years. The big success of course were expert systems. As an early example, look for information on MYCIN, the first system to beat human experts at medical diagnosis (of infections, in partiuclar).

Even today, after the recent meteoric rise of deep learning, logic-based algorithms are state-of-the-art for many applications: in particular, planning, constraint solving, game playing (as in minimax variants) and anything that requires reasoning. In applied work, decision-tree learners, a class of machine learning algorithms learning propositional-logic models, are still widely used in data science work, more so than brittle and expensive to train deep neural nets.

>> But I'm getting the feeling that, while you are making this extraordinary claim that "logic-based AI has failed to come up with anything of value whatsoever", you are not very familiar with the history of AI research.

Ugh. This sounds like a personal attack. I apologise unrservedly. I really did not mean to insult you or disparage your knowledge of AI research. There's much better ways to say what I wanted to say here.

Apologies, again. I can't edit my comment now.

Pretending that recognizing "a^n b^n" is somehow better achievement than generating natural English creative writing from prompt is the real sour grapes here.

Since I don't like to explain comments I haven't made, can you show me where in one of my comments I say that?

> So, besides the plug of my work (sorry) I would say that the earlier paradigm that you say has "failed" with respect to recent connectionist success is alive and well and it still gets many things right that the connectionist paradigm struggles with mightily, in particular robust generalisation from very few examples (without expensive pre-training etc), and of course any task that requires reasoning [2].

I'm going with dooglius here. You are presenting a simple regular grammar as a success story of the GOFAI paradigm (which doesn't have that much to do with Hofstadter's paradigm, anyway), while the DL approach has been producing real-world successes on the hardest of natural language tasks. Your example is "not even wrong" in terms of criticisms like Marcus's, because it is so many orders of magnitude away from being useful or reaching the point where it even could fail or be criticized on those terms.

aⁿbⁿ is a context-free language, not regular. The fact that it's simple is exactly the point: it is so simple, yet neural nets cannot learn it. They can only learn limited subsets of it and even those, from tens of thousands of examples. I showed how the same language can be learned in full from three examples. It's a task in which neural nets, despite all their successes that you cite, fail dismally. You dismiss it easily, but it's an ebmarrassing failure and it makes clear the weaknesses of the connectionist paradigm.

Despite myself, I am still surprised by the unwillingness of connectionists to admit this simple fact: neural nets generalise very poorly and have absolutely awful sample efficiency.

As to more complex, real-world tasks, this is a recent example on a language task using the same algorithm (but a different implementation):

Bias reformulation for one-shot function induction

https://dspace.mit.edu/handle/1721.1/102524

There's more of that in the literature.

So I think I really screwed this one up, by assuming a common background. Your comment about aⁿbⁿ being a "simple regular grammar" stuck with me and I realise that we do not have any common background and I should have done a better job of putting my example in context.

It was my mistake to agree that aⁿbⁿ is a "simple" grammar. It looks simple and it's simple for a human.

But, aⁿbⁿ is a context-free language and as such it can only be learned "in the limit", i.e. by a number of positive and negative examples approaching infinity. It cannot be learned by positive examples only, not even in the limit. This is part of Gold's result [1], a famous result from Inductive Inference, the field that essentially predated Computational Learning Theory. Gold's result was that anything more complex than a finite language can only be learned in the limit and anything above regular languages requires both positive and negative examples.

Gold's result had a profound effect on machine leargning [2] and was a direct cause of the paradigm shift that came with Leslie Valiant's paper that introduced PAC Learning [3]. Valiant's paper placed machine learning on a new theoretical foundation where it's acceptable to learn approximately, under some assumptions (particularly, distributional consistency between examples and true theory). This is how machine learning works today and this is why machine learning results are accepted in scholarly articles, not because they are useful in the industry or make nice articles in the popular press.

Where aⁿbⁿ comes into all this is that it's one of a group of formal languages that are routinely used to assess machine learning algorithms. Keeping in mind that grammars for CFLs are impossible to learn precisely from finite examples, the point of the exercise is to show that some learning algorithm can learn a good approximation that generalises well from a small number of examples.

This is how formal languages are used in neural networks research also. For a recent example from the literature see [4].

The example in my comment shows instead that our algorithm can learn aⁿbⁿ precisely, not approximately, from only three positive examples. This should be surprising, to say the least. Gold's result says that this should not be possible, at all, ever, until hell freezes over. Why it is possible in the first place is another long discussion that I was hoping to have the chance to make here, but instead I allowed myself to be drawn into an adversarial exchange I should have avoided.

To put an end to it: neural nets have produced some spectacular results, and I don't want to discount them, but it is important to understand what those results mean, in the grand scheme of things. The fact that there is a machine learning paradigm that outperforms neural nets on a very hard problem (simple as it may look) means that the same paradigm may outperform neural nets on those other things that they do so well, like image recognition and speech recognition. Or it may just mean that different learning paradigms have different strengths that must somehow be combined (a more popular view for sure).

Anyway, sorry to spread noise and confusion in this thread when my goal was to do the opposite. I have a rubbish way of communicating.

_____________________

[1] Language identification in the limit:

https://www.rand.org/pubs/research_memoranda/RM4136.html

[2] And a few other fields besides. Noam Chomsky used Gold's result to argue for a Universal Grammar enabling humans to learn natural languages. This all is really groundbreaking stuff and a real shame that it's not more widely known in machine learning circles.

[3] A theory of the learnable:

https://web.mit.edu/6.435/www/Valiant84.pdf

[4] LSTM networks can learn dynamic counting:

https://arxiv.org/abs/1906.03648

I’m sorry to question the point of your work, but what is the point, exactly? How is what you do at all useful for any practical purpose? What can you do for me that I might want to do but can’t currently do with DL?

>> What can you do for me that I might want to do but can’t currently do with DL?

I'm sorry but I don't understand this question. I'm not trying to do anything "for you". It sounds as if you think I'm trying to sell you something. I'm a researcher, I don't sell stuff.

But I'm curious: what are you currently doing with deep learning?

Gotcha. No practical use of your work. You're like those pure mathematicians who just do math for the sake of math. I don't see a problem with that, it's just amusing you're trying to compare your field with deep learning.

I'm a researcher too - I look for ways to build faster hardware for deep learning. But to answer your question - using deep learning I can classify objects, recognize speech, and translate language, as well as generate plausible text, nice sounding music, or beautiful pictures. Those are just few examples of what I can do with DL much better than with any other methods.

>> Gotcha. No practical use of your work.

I find this unnecessarily provocative and it's certainly not what I said.

>Google Translate, plus some critical thinking and a bit of work with a dictionary, will get you a rudimentary understanding of a text in a foreign language --- and that's a great advance.

For some languages it will, they targeted the official working languages of the UN at the time. I hear for them it's pretty good. Personally, I live in Japan and rely on Google Translate every day and my experience has been generally crap. I've seen it get some simpler phrases in Japanese dead wrong and have been noticing it more often the more I learn the language. The part that infuriates me the most is how translations for simple phrases seem to change on an almost daily basis. Not long ago it had a pretty good translation for the formal (keigo) greeting used in business correspondence but as of a few months ago it utterly mangles it. Yes language changes but not this quickly, a lot of these phrases would've been used by the great grandparents of my colleagues.

>Google Translate, plus some critical thinking and a bit of work with a dictionary, will get you a rudimentary understanding of a text in a foreign language

Which is precisely what he says.

> Hofstadter expresses an expectation that no-one holds for an automatic translation tool,

As demonstrated by the quote at the start, about Russian being just English coded in strange symbols, and also by the unreal hype around "neural networks", plenty of people do on fact hold unrealistic expectations of automatic translation. How many times have you heard someone say translator jobs are going to be automated away in 5 or 10 years? I've heard and read it plenty of times.

> but he has chosen two passages where gender is critical, and moved them between languages where gender is expressed in very different ways.

So what, if the program can't handle these simple sentences, it can't claim to be a translation program. The machine 'translation' throws away the entire point of the sentence.

Why don't we just evolve towards the machines and start speaking to each other in JSON (We should come up with a more efficient way to pronounce '}' as soon as possible!)

Most of the AI revolution is just parlour tricks and marketing puffery. Fourth grade children could translate the French sentence better.

Make a point but better keep the name calling and gross generalizations regarding those who read Hofstadter's output for readers that enjoy that kind of pointless abuse. There are plenty of other outlets for it.

would you prefer that he hid his light under a bushel ?

I really hate pieces like this. Hofstadter should know better than anyone how state-of-the-art machine translation works. Only absolute laymen would use words like “understanding” to describe what’s going on. It’s really good, and we’re a lot further than we were 10 years ago but honest to god people should just drop using anthropomorphic laden terms like “understanding” and “intelligence” into these discussions. The I in AI is unfortunately taken too literally. Maybe just do yourself a favor and watch the videos to CS224 [1] and then you’d be less surprised that these systems do not “understand” (whatever the hell that even means, unless rigorously defined).

[1] http://web.stanford.edu/class/cs224n/

Reply on news.ycombinator.com