Hacker News Reader

Comment by retrac | Hacker News Reader

> Also, while it's easy to calculate word frequencies for stuff like newspaper articles, there aren't any good statistics (last I checked) around just normal everyday conversation. Because that stuff isn't getting recorded and transcribed.

Linguists have resorted to spying on people in parks and shops and restaurants, to get "natural" language unperturbed by observer effects, audience tailoring, the filter of writing, etc. (In most cases that would not be considered ethical today!)

The gold standard is something like: get acculturated to the language community you wish to study, make some acquaintances and friends, somehow finagle an invite to their homes or workplaces or social venues, and then with their permission, record their conversations as unobtrusively as you can.

There is so little data on how language is really used it is quite bedevilling. As soon as people know you are listening they change how they speak.

View on HN

> Linguists have resorted to spying on people in parks and shops and restaurants, to get "natural" language unperturbed by observer effects, audience tailoring, the filter of writing, etc.

Hence the common phrase, found in most language phrasebooks: “is that a linguist hiding behind that bush?”

Im working this into some conversation this week. I feel its something Douglas Adams would have wrote.

> “is that a linguist hiding behind that bush?”

No, a FAANG employee. /s

That’s what makes ethnolinguistics fascinating. As a native French speaker, I started getting interested by linguistics when I learned Italian, and now I keep noting things I never thought about when I talk with other native French speakers. As someone said, “You don’t know your language until you know another one”.

Why is it unethical to record what people say in public settings, like a cafe, transcribe it, and then throw away any identifying information? You could go even farther and only store the individual set of words spoken without any ordering, so it would be impossible to reconstruct the actual content of anyone's conversations.

Universities in certain countries have strict ethical standards that require consent and crediting informants. These standards were often inspired by working with indigenous informants, who were viewed as having been historically exploited in the collection of data, but they apply now to anyone eliciting data from anyone.

What is the practical harm that is actually being avoided here?

When people speak in public, they assume that other people around them can overhear what they're saying. This is just gathering anonymous statistics about word use. It's hard to imagine how there's any real downside for anyone here.

The word set is enough to identify some people based on speech patterns or interests. The inclusion of other words in the set — cocaine, divorce, abortion, merger, lymphoma — would make many people uncomfortable.

In a perfect world, it wouldn't be. Personally, if I knew an infallible person was recording my private conversation and anonymising it to an acceptable degree, I wouldn't care. For example, by removing personal identifiers and aggregating the data across a topic such as "conversations at cafes in central Paris in 2026".

But the second there is doubt about security across any link in the chain from raw recording to anonymisation and storage, I would no longer consent.

That's what I discovered religion is for, understand the language as it was created.

Ever wonder who decides what's self-explanatory and what resonates?

What?

This is one upside of the digital panopticon, I guess.

Its um like you know whatever

Reply on news.ycombinator.com