Hacker News Reader

My favourite example of bad data in for machine learning is the tragic tale of Scots Wikipedia: https://www.theguardian.com/uk-news/2020/aug/26/shock-an-aw-...

It turned out an enthusiastic but misguided US teenager who didn't actually know the Scots language was responsible for most of the entries on it... and a bunch of natural language machine learning models had already been trained on it.

View on HN

Reminds me of: https://en.wikipedia.org/wiki/English_as_She_Is_Spoke

Scots pretty much is a dialect of English that is phonetically spelt out - it's not surprising that a US teenager could write it.

>"Scots pretty much is a dialect of English that is phonetically spelt out - it's not surprising that a US teenager could write it."

No, there are a number of distinct linguistic features of Scots, and it has its own regional dialects, e.g. Doric, Orcadian, Shetland (which is also in part based on the extinct Norn language). See e.g. https://dsl.ac.uk/about-scots/history-of-scots/ (and sub-pages such as https://dsl.ac.uk/about-scots/history-of-scots/grammar/ ) for further information. Simply doing a dictionary-lookup word-replacement completely misses all of this nuance.

Reply on news.ycombinator.com