1,904 karma · joined November 18, 2012
www.xuanji.li
[ my public key: https://keybase.io/xuanji; my proof: https://keybase.io/xuanji/sigs/YyOVzG7h-Ip1eEcZdfnsvWckOKvnM-6b09PfP3q-9Y0 ]
hi
FWIW I'm sympathetic to that position (in some form), it would just be better to be more precise... people are acting like "water is a basic human right" is a principle that can straightforwardly be used to derive the rest of our water policy.
Even after defining what the difference is... I'd be interested in understanding what makes trading in "water-delivery-and-treatment-infrastructure-and-operational-costs" OK, but trading in "water" not OK
Do you pay a water bill where you live? If so isn't that already "trading with water"?
If we do get to a point where this is true, it will primarily be because of decreasing amounts of water sources, increasing population, etc. Banning water futures won't magically solve these problems.
No, I can implement the exact same weird functions with option types, with signatures
sum: Array<Int> -> Optional<Int>
count_elements: Array<Int> -> Int
FWIW, I personally prefer both functions to not return Optionals, but the point is that the mere presence of Option types in your type system (and the absence of null pointers) doesn't magically eliminate bad library design
A midi file is designed to contain different kinds of information than a score, so it can absolutely contain less information (of a certain kind) than a score.
Imagine a group of notes that are attacked very close together in time in a midi file - is it meant to be a chord played without arpeggiation? A chord arpeggiated as fast as possible? A chord arpeggiated to a specific tempo?
When a composer writes (say) a chord with an arpeggiation sign next to it, it specifies that the notes are to be played in a certain order, and quickly. There are multiple possible performances that comply with this "specification". MIDI cannot encode this information (the information being the set of acceptable performances).
You're looking for a proof of Bezout's identity, or something in the spirit of that theorem :) to be honest, I haven't seen a proof of it that I "really really like", although you might find one
I don't think that "NMT and similar techniques" ignore meaning by design though. What they do do by design, compared to expert systems etc, is avoid having explicitly encoded knowledge (of the kind SHRDLU had). Take word2vec for instance, it's not NMT but fits into the "statistical NLP" description - its purpose is to find encodings of words that carry some semantic content. Now, of course it's very little semantic content compared to what an expert could plausibly encode, but it is some semantic content, and this content improves the (subjective human) evaluation of NMT systems that do use word2vec or something similar.
Also, we should carefully distinguish "prior knowledge" as in "prior common-sense knowledge" and "prior linguistic knowledge". The end-to-end trend eschews "prior linguistic knowledge", while current NLP systems tend to lack "common-sense" knowledge, for rather different reasons.
End-to-end training tends to eschew prior linguistic knowledge because it improves (subjectively evaluated) performance in real-world tasks - I believe this is true for MT as well, but an easier example if you want to look into it is in audio transcription. I don't think there's a consensus about why this happens, but I think it is something like - the previously way people were encoding linguistic knowledge was too fragile / simplified (think about how complicated traditional linguistic grammars are), and if that information can somehow be learned in the end-to-end process, that performs better.
Lacking "common-sense" knowledge - that's more in the realm of AGI, so there's a valid debate about to what extent neural networks can learn such knowledge, but the other side of that debate is that expressing common-sense knowledge in today's formal systems is really hard and expensive, and AIUI this is also something that attempts to generalize SHRDLU run into. But it is definitely incorrect to say that it's ignored by anyone by design...
BTW, the biggest improvements (as subjectively evaluated by me) I've seen in MT on "dissimilar languages" have come from black box neural nets and throwing massive amounts of (monolingual or bilingual) data at it, rather than anything from formal systems. I use deepl.com for Japanese-English translation of some technical CS material, and that language pair used to be really horrible in the pre-deep-learning days (and it's still not that good on google translate for some reason).
OK, I hadn't realised we had such different implicit views on what the "goal" of AI / AI research was. Of course, I agree that the goal of "having machines think like humans" is a valid goal, and "generates value by serving production traffic" is not a good subgoal for that. However, this is not the only goal of AI research, nor is it clear to me that for e.g. public funding bodies see it as the only goal.
I use MT (at least) every week for my job and for my hobbies, mostly translating stuff I want to read in another language. I love learning languages but I could not learn all the languages I need to a high enough level to read the stuff I want to read. The old non-NMT approaches produced translations that were often useless, whereas the NMT-based translations I use now (mostly deepl.com), while not perfect, are often quite good, and definitely enough for my needs. Without NMT, realistically speaking, there is no alternative for me (i.e., I can't afford to pay a human translator, and I can't afford to wait until I'd learned the language well enough). So how can you say that AI "remains completely useless"?
Basically, you have implicitly assumed that "make machines that think like humans" is the only valid goal of AI research. And, from that point of view, it is understandable that evaluating NMT systems for how well they approach that goal using human evaluations, has many downsides. However, while some people working on NMT do have that goal, many of them also have the goal of "help people (like zodiac) translate stuff", and in the context of that goal, human evaluation is a much better benchmark target.
More to the point, you still have yet to show that NMT "serves no other purpose than to show how well modern techniques can model large datasets", given that they do well on human evaluations and they're actually producing value by serving actual production traffic (you know, things humans actually want to translate) in Google Translate. If serving production traffic like this is not "serving a purpose", what is?
There were two ways that performance can be evaluated that I had in mind:
1. Commercial success - what approach does popular sites like Google Translate use? 2. Human evaluation - in a research setting, ask humans to score translations - this is mentioned in your link
On the subject of benchmarks, how about speech transcription? I was under the impression that those benchmarks are pretty reliable indicators of "real-world accuracy" (or about as reliable as benchmarks are in general)
How do you know if no one has attempted it, or if all attempts so far have failed to achieve their goals? One of the claimed downsides of "hand-crafted systems in limited domains" (i.e., something like the SHRDLU approach) is that they would take too much effort to create when the domain is expanded to something even slightly bigger than SHRDLU's domain, so a lack of successful systems could be evidence of no one trying, or it could be evidence that the claimed downside is indeed true.
The fact that a working system for the limited domains of e.g. customer support or medical diagnosis would be worth a lot of money suggests to me that they must have been tried, but that nothing useful could be built, and we didn't hear about the failed attempts, meaning that those domains (at least) are too big for hand-crafted systems to work.
> Yes, it is faffing about. Basically NLP gave up on figuring out how language works and switched to a massive attempt to model large datasets evaluated by contrived benchmarks that serve no other purpose than to show how well modern techniques can model large datasets.
It is inaccurate to say that all benchmarks are useless. For instance, language translation (as in Google Translate) is a benchmark NLP task, but it is also something I personally use at least every week, and deep learning based solutions beat handcrafted systems by a lot for this particular task (speaking as an end user who has used systems based on both approaches). The same comments apply to audio transcription (e.g. generating subtitles on YouTube) as well.