Hofstadter, having completely missed the boat on connectionism, committed himself to a failed paradigm, and been wrong about AI time after time (remember when Hofstadter claimed beating humans at chess would require computers to have emotions and be able to write poems?), totally fails to grapple with what NNs do do. For starters, Google Translate is a free service and not necessarily SOTA, even when he wrote this. For example, if you were going to ask 'what do multi-lingual translation seq2seq RNNs (or Transformers) understand?', the immediate obvious thing to discuss would be questions about the 'interlingua' which their embedding yields, things that word embeddings learn (including surprising things like relative locations of cities or sizes of objects), or about transfer learning to tasks that require grammar or reasoning or common-sense like Winograd schemas like in GLUE or SuperGLUE, and how they improve massively over earlier approaches and how they are scaling (one of the most important trends in AI). There is a great deal which could be said here, demonstrating why it works so well and why it doesn't work sometimes, and an AI researcher is just the sort of person to ask.
And unfortunately, Hofstadter does none of this and instead bangs on about some errors he found, like it's a total blackbox and understanding is a binary where you either like the same poetry Hofstadter does or you are a cretin, and takes the attitude that any error shows that the approach has failed, is failing, and always will fail because it is 'deeply lacking' (what, exactly? And to think, he complains about 'deep' being abused for rhetorical purposes before and then goes and produces a shallow exercise like this).
I thought this was a lousy neo-luddite essay in 2018, and it's only gotten worse since then. In another 10 years, it'll look like Cliff Stoll.