Researchers have identified new elements of whale vocalizations | Hacker News Reader

Researchers have identified new elements of whale vocalizations

vice.com

127 pointsmanicennui78 comments

As usual with these kinds of things the title doesn't match the article.

There is no breakthrough in understanding. Instead they identified a thing in the sound that changes.

They do not understand what it means, they just see it. It could be something, or nothing, or just music.

Indeed, they don't yet understand meaning.

But a new understanding of the components of the code and their relationships is certainly a breakthrough towards that understanding.

Yes, the headline had me hoping that there was some full understanding (it'd be nice to broadcast to the Orcas off Spain & Gibralter to please not attack the boats), but I can't say that it's clickbait-level.

After harvesting the ozone layer using airships that produce eardrum-splitting noise for decades, the aliens achieved a communication breakthrough and were finally able to communicate to the pesky humans: “Please stop attacking our airships. They are very convenient for us.”

Perhaps, but the scientists involved went from "whale songs are like groans" to "whale songs are like talking". The 'breakthrough' they were describing is that visibility into a complex, and repeatable, substructure to the songs shows that those songs could contain more complex "words." If you have ever tried to communicate by groaning only (which an acquaintance of mine was forced to do while recovering in the ICU once) there are very limited things you can say, and it often reduces to you asking questions and them producing a "yes" groan, or a "no" groan. They can't 'tell' you a whole lot.

As a result of their research they are getting closer to the underlying structure (which others will look for in other species of whales) and with that they may be able create situations where they could start testing the concepts that might be in that structure. That sounds kind of breakthroughish to me :-)

Having read the article. I don't know where one would get the idea that it said anything meaningful about the songs being like "talking", which implies a whole host of features that were not even mentioned. They made analogies with human language phenomena such as suprasegmental features but really this was about there being newly discovered patterns in the sounds. These were not correlated to any specific meaning or purpose. We need those if one is going to talk about the whale song as a communicative system at all, much less a language. With reference to something like Hockett's design features, we have a modality, but despite the existence of a pattern, we don't even know if there is discreteness, necessarily. It is kind of like how children will be taught about atoms and think about the solar system. The analogy may be meaningful or not. The article relies on the readiness of the reader to anthropomorphize the noises made by the animals and misleads them.

This from the paper's abstract: Finally, we control for whale movement and present several pieces of evidence suggesting that the observed patterns are not artifacts, but are actively controlled by sperm whales. We also show that the two coda vowels (the a-vowel and i-vowel) are actively exchanged by sperm whales in dialogues. The uncovered spectral properties suggest that codas are highly compositional, more informative, and more complex than previously thought.

This from the article: “If our findings are correct, it means that the communication of sperm whales is much more complex and can carry more information than previously thought,” the researchers concluded.

The researchers here make two observations, the whales intentionally use specific forms in their songs and the specific forms are sequenced differently but intentionally. For biologists this is significant because it differs from a common form of "singing" or "calls" that each have a specific meaning but they are not internally variegated like these calls are. What the researchers propose (or hypothesize) is that this intentional and varied voicing within a "song" suggests the whales are exchanging much more detailed information than "threat" or "help" or "food here" etc.

Morse?

Impertinent question: why not ask ChatGPT to talk with them? It doesn't have understanding either. I think this would be a great experiment in the meaning of consciousness. Can we train an LLM to communicate with whales, birds, etc.?

It’s being done right now:

https://blog.padi.com/talk-to-whales-with-ai/

One difficulty is that there isn’t nearly as much whale chatter available for training data as there’s human chatter.

And one important question is, why is this useful if we don’t know what the LLM says to them? But the post above touches on that too.

Won't they need some documents that combine whale speech noises with human words to bridge the gap? Otherwise they are making comments that are just word-like or sound-like fragments.

If you had enough recordings, you could (I think) build weights based _solely_ on whale speech. Humans wouldn't be able to understand the weights, and the word vectors in that model wouldn't match the word vectors in an English model, but I suppose there's a chance that vectors might be similar? I don't know. I think you'd have to be very good at both linguistics and also AI to know.

I don't think this is strictly needed. An English dictionary may seem pointless because it defines every word using only other English words. But the meaning is contained in the _relationship_ between the words.

I'm sure you've seen the example of word vectors that captures some of this meaning. king - man + woman = queen

In Spanish, rey - hombre + mujer = reina

The _relationship_ between "king" and "queen" in English may look close enough to the _relationship_ between "rey" and "reina" in Spanish, allowing you to bridge the gap between the two languages, even if they are entirely disconnected and you've never seen a direct translation between them.

>And one important question is, why is this useful if we don’t know what the LLM says to them?

There's no reason to be sure we couldn't know.

It's not like there are examples in the training set for every lang to lang combination modern models are capable of translating.

A llm can write a program to compute factorial or whatever, but that doesn't mean it has consciousness. Same for writing a poem about any topic. It is of course also not evidence of no consciousness. We just don't know but the likely-hood seems low.

For whales or even dogs or great apes, I think the chances are much higher, but we just don't know. We can't even agree on a definition for what consciousness is.

What does it matter if they’re conscious if they’re dumb?

Just because a consciousness exists staring out of a pair of eyes doesn’t change any ethics if that consciousness is exceptionally limited.

I thought about this for a few days. I eat meat, animal products like milk too. I feel some guilt about it, an animal died to get me this chicken sandwich. We'll face decisions about whether to continue ending the consciousness of animals to feed ourselves when we have alternatives. I'm far from the first person to think of this, but when we have some machine consciousness that compares to a chicken (I don't know how to measure consciousness but we'll face that one day), will there be a common understanding of what it means morally to be ending that consciousness?

I think one day we'll probably almost completely stop killing "intelligent" animals (in 100s of years after we develop safe and tasty alternatives). What about a software consciousness? I don't know. What about if we are able to "upload" our consciousness, will it have civil rights? Probably not at first.

In theory, we could train an LLM on whale "song" or bird chirps. We'd need to have lots and lots of it to feed it. But I don't think it would tell us what the animals are saying. The animals would theoretically recognize the output of the models and be able to interact with it but I don't think there is a clear path from that to our understanding.

The LLM accidentally declares war on all whales... :/

It would, best case, just be able to talk about concepts that whales and birds talk about, right? “Ahh, a predator” “I found food” “weird trickster-monkey spotted, expect bizarre tests or treats or mysterious death.”

I’m not sure what a ML algorithm could do better than a tape recorder here.

We don't know what concepts whales talk about

Let them cook. That's how many scientific breakthroughs begin.

They identified a thing that changes, but in a consistent way across the sample population. That's a very important distinction, because synchronizing vocalizations is not easy. Humans spend a long time learning it. And failure to do it "correctly" can be everything from an accent to a speech pathology, depending on how "wrong" it is. (Scare quotes purely because the definition of right and wrong changes over space and time.)

Can't we use an AI model to decipher different animal noises and correlate to what it could mean?

Sounds like they took the first step.

There is a big effort to do that actually, called Project CETI: https://www.projectceti.org/

If nothing else it has a fantastic name

Yes. And this result is comming from that project.

Gašper Beguš who is the linguistics lead of Project CETI is the first author of the paper this article talks about.

I’m not sure how you would go about attributing meaning to the sounds. Usually you need labels for that sort of thing. With current techniques we might be able to predict which sounds come next, but we still wouldn’t know what they meant.

Link to paper: https://oa.mg/work/10.31219/osf.io/285cs

Here is a bad dramatization, er "extrapolation", starring Meryl Streep, of where this could lead if we ever do gain real understanding of whalespeak.

https://tv.apple.com/us/episode/2046-whale-fall/umc.cmc.2gda...

Spark notes...

Scientists studied how sperm whales communicate underwater.

They found that whale clicks have similarities to human vowels and diphthongs.

An AI model helped by imitating whale sounds, providing useful information.

The researchers discovered specific patterns in whale vocalizations, like unique "coda vowels."

Whales seem to control the frequency of their calls, making their communication more complex than we thought.

No, we can't have a casual conversation with whales yet. Serious ones either.

> The researchers explain in their paper, published as a preprint online this week, that the first clue that so-called spectral properties could be meaningful for whale speech was provided by AI.

> Not only did the AI predict elements of whale vocalizations already thought to be meaningful, such as clicks, but it also singled out acoustic properties.

Hakai magazine had an interesting article/podcast episode about the project a while ago: https://hakaimagazine.com/features/are-we-on-the-verge-of-ch...

Could a multimodal LLM finetuned on whale sounds be used to translate whale language to human language?

Only if it was trained on a corpus of existing translations.

Just training on existing sounds might give us a whale sound generator. But it would be the equivalent of Prisencolinensinainciusol: https://youtu.be/-VsmF9m_Nt8

Not necessarily -- the embedding into semantic space is largely unsupervised; LLMs do not learn to translate through explicit examples of parallel text. Largely it's just a matter of having a large enough corpus of whale speech in a format that can be encoded into a token sequence.

    > LLMs do not learn to translate through explicit examples of parallel text.

Of course they do, lots of that is in the training data.

For general-purpose LLMs, there is no indication given during training that texts are parallel -- while there are parallel texts in the training data, the model is not trained on generating one text from another. That is, it does not get any feedback during training about translation, only prediction in the current token stream, not parallel ones.

There are several completely unsupervised language translation systems that predate LLMs, but the performance is middling.

For LLMs the translation behavior is largely an emergent property that is not completely understood; if we can tokenize whale language in a useful way, it is entirely possible that the LLM can derive a weak or approximate translation of some of the language structure.

If you have a parallel corpus in your training data you don’t need to explicitly give feedback about translation. The loss function is the feedback.

Maybe a better example is in the multimodal context. Drawing images from words is a type of “translation”. But for this task we need captions, i.e. a parallel corpus.

It's still not necessary to have bilingual data

https://news.ycombinator.com/item?id=38574712

First of all, thank you for finding this paper, it seems to be addressing the question directly.

However, I believe you are overstating the results of the paper. If anything, the paper demonstrated massive reductions in zero shot translation capabilities after removing translation. And for languages with no cognates with English, BLEU scores are pretty abysmal. So the paper suggests that parallel corpuses still very important.

For whale languages, you won’t have any cognates with English, and we don’t even know if they have a grammar that is remotely human.

Language Models trained at the size and data they tested are pretty terrible to begin with, translation or not.

Like I said, the crucial thing is that it looks to be a diminishing gap. I'm not saying parallel corpora is doing nothing or is unimportant but that's a clue that monolingual competency of the languages in question is the biggest factor here. The extreme positive transfer language models exhibit in terms of multilingual competency is another clue. https://arxiv.org/abs/2108.13349

Parallel corpora may be a crutch with fading relevance.

> The loss function is the feedback

Please expand -- as I understand it the LLMs use cross entropy on next token prediction as their loss function and I fail to see how this gives any feedback about translation, even given parallel texts. Predicting the next token in a French language text and predicting the next token in an English language text are not obviously more interesting than dealing with non-parallel texts in both languages.

It's totally possible that these parallel texts are critical in a sense to the translation capabilities but this is not obvious.

It would be incredibly difficult to establish empirically because reducing training data reduces the effectiveness at all tasks. Translation, like almost all of the capabilities of LLMs, is an emergent behavior that we don't fully understand yet.

>a large enough corpus of whale speech

To add, large enough here is also a potentially shifting target. LLMs exhibit extreme positive transfer of language competency. So 5B tokens of Korean will take Korean competency much further if trained alongside 50B tokens of English than if alone.

There must be a underlying commonality for that to be possible.

What would a whale call a Quarter Pounder with Cheese?

>Only if it was trained on a corpus of existing translations.

Not Necessary.

Searching for Needles in a Haystack: On the Role of Incidental Bilingualism in PaLM's Translation Capability (https://arxiv.org/abs/2305.10266)

ablation studies performed on the effects of removing tiers of multilingual corpora (parallel corpora, bilingual (not necessarily parallel), and english only)

Removing Parallel corpora or even just bilingual data in general reduces quality but does not make the model unable to translate. Crucially, the gap or reduction in quality also seems to diminish with scale.

The paper also shows models may not need any multilingual corpora at all. The 8b Model can still translate latin scripts with only english training data, adding evidence for larger models being better equipped to leverage either sparse signals (i.e., language-identification failures during ablation) and/or weak signals (i.e., language similarities from shared scripts).

Yes, but all the translations would be wrong or made up.

Whale: "Holy cow, the humans are talking . . . wait, WHAT did they just say about my mother?"

At least a single person understands LLMs in here.

Is HN getting dumber? What's with all these threads where people basically assume the "AI" we currently have is some kind of superintelligence?

Recent and related:

Vowels and diphthongs in sperm whales - https://news.ycombinator.com/item?id=38541217 - Dec 2023 (29 comments)

Just for fun some years ago, I wrote an Onion-style satire article about, "Whale language decoded; actually a cipher for Spanish".

The gist was a lot of the researchers with names like Jones and Smith were disappointed. And the one guy on the research team with a Spanish last name was getting a lot of weird comments about it which made him uncomfortable.

75% of the whale song was decoded to just mean, "Oy mira! Camaron!"

Dolphins were discovered to be speaking a cipher for English, but with a Chicano accent.

Some of my friends found it funny, others didn't get it.

---

I'm really interested to hear what they're actually talking about!

Whale done!

With porpoise!

Felicetaceans!

Sometimes I'm not if I have malware installed or if this is just the state of the web right now: https://imgur.com/a/wzb8fAT

I can't imagine browsing sites without an adblocker.

Sounds like a good statatement to print on t-shirts.

Ads Are For Suckers

reads more succinctly. (And no offense to OP.)

No offense taken. Unlike my childhood where I used to navigate via webrings and questionable search engines, these days I rarely visit anything but HN links. So I haven't had much of a need for an ad blocker.

Occasionally I'm surprised links like these get to the front page, even with all the ads and horrible UX. I must admit, I too wanted to believe so badly we can communicate with whales, that I managed to scroll down to see the first paragraph below the fold. But quickly closed it shortly after to read the HN comments.

Yeah, it is too bad this crap is the new normal for web sites. It saddens me that Gen Y (or what have you) never knew an internet before it became a marketing landscape.

Related: there's a snazzy uBlock Origin logo T-shirt for sale online, but I won't link it right now, because unclear whether it's approved by Raymond Hill, and I don't see a mention of what the proceeds support.

my guess based on his strong (and extremely venerable) principles against monetizing uBO is that it's not approved by him, although I think him selling merch would be an awesome way to fund development. I would gladly buy some uMatrix t-shirts if it meant a resumption of uMatrix dev! :-D

A vanilla browser is genuinely malware that needs an adblocker to act as an old school antivirus.

I think the problem is that you have one population of readers who go with the ad blocker, and they don't know that they are about to send everyone to this eyesore. The other population is people who don't use ad blockers, but also who would never visit this site, because it's an eyesore.

For that second population: https://osf.io/preprints/osf/285cs

The overwhelming remainder of people are in neither: they don't block ads, and they don't care. And their TVs are on all day long.

If you define a browser without a content blocker as 'malware' the answer is 'yes, you have malware installed'. Get a content blocker - uBlock Origin works fine in Firefox and still works in Chromium - and take the road to sanity. As far as I'm concerned the commercial web is unusable without at least a single layer of defence against advertisements and advertisers, preferably several layers - DNS or hosts file, content blocker, etc.

how'd you make the Memegen overlay text?

I use CleanShot - an OSX screenshot utility. Highly recommend if you spend a good portion of your day in issue trackers or doing project management work. Besides quick annotations, it's easy to capture gifs, videos, scrolling screenshots, etc.

Ironically that imgur link on mobile is essentially a page of visual noise

About 40% of the screen is pestering me to download their app to look at an image. That I always find the most annoying. I kind of wish product managers would start from the assumption that users don’t care about your company at all. Impress me and give me a reason to care before pestering me!

Yeah, it's 2023 and there's still no great way to share images besides self-hosting or the hacks mentioned here: https://news.ycombinator.com/item?id=31631144

I strangely forgot the screenshot tool I'm using, Cleanshot X, actually has a cloud-share feature built in. I've never used it since 99% of the time I'm pasting these images into issue trackers, slack, etc.

I use vanilla firefox on Android and dont se any shit like that. So i say you have maleware

Great news! Now Kirk and crew don't need to slingshot around the sun and go back in time to save the Earth.

Proof we're on a separate timeline right there. Slightly bummed tho I must admit.

This knowledge was likely lost during the Eugenics wars, so the time continuum is safe for now.

This knowledge applies to Sperm Whales. The space probe was specifically looking for Humpback Whales. The continuum is doubly safe for now.

And we can now translate their warnings about the incoming extinction level CME and save Atlantis. These are exciting times

While reading the post, I kept thinking how would it sound to pronounce “Adaris” using whale a-vowels and i-vowels.

Apparently whales have been doing podcasts for a while now

Reply on news.ycombinator.com