Hacker News Reader

Comment by willis936 | Hacker News Reader

Language isn't wasteful. All of that extra information allows for error correction. You can infer what

  *r* **u *k**?

means. Longer messages can be easier to decode, for similar reasons stated above in this thread. A lot of information is contained in the language outside of the character set, which is necessary for decoding. It asks a lot of the decoder to know everything necessary, but allows for relatively robust communication.

View on HN

That's a pretty atypical example, you've put the errors in exactly the right places. Imagine that string was

  *r* *o* ***y

  a** y** *k**

and you can see these are indecipherable and this is not a good example of "error correctable language".

Error correction in language typically refers to understanding through context (i.e. other sentences), which would not be lost in this sort of compression.

They aren’t indecipherable though. Context is inseparable from language. How many three letter words begin with “A”, followed by three letter words that begin with “y”, followed by four letter words that have a “k” in them? This is all well described in information theory and the redundancy of language is not invalid because I picked an obvious example.

There are many three letter words that begin with A, followed by 3 letter words that begin with Y, followed by 4 letter words with a K in them. Here are some examples.

  All you skim.
  And you skip.
  Ate yak skat.
  Add yak skin.

If you're telling me I can randomly hide 2/3rds of characters in any English sentence and you can still derive the meaning, I'm going to need a citation beyond "information theory" in general.

But this is tangential to the point. I'm saying the context is preserved with compression, hence the meaning is preserved. I'm saying we can compress the amount of letters used and still get the same meaning.

Elements of Information Theory (2e), Cover. Section 6.4. I also suggest following the authors suggested reading at the end of the section.

Also there is a difference between having information and having perfect information. English can’t guarantee perfect information of arbitrary statements with 2/3 loss.

Y cn ndrstnd wrttn nglsh vn wtht n vwls!

Random errors would be unlikely to only remove vowels.

Still counts as error!

Reply on news.ycombinator.com