Hacker News Reader

Comment by diego | Hacker News Reader

The thing is, 90% coverage is not that great. What happens is that you understand common words that make up for a lot of structure, but when an uncommon word appears it's probably important to the sentence. For example, "son, if you go to the plumbf tomorrow morning don't forget to pick up some zlonks."

98% is closer to what you need in order to read a text and have an idea of what's going on. See this article:

https://www.sinosplice.com/life/archives/2016/08/25/what-80-...

View on HN

As someone who barely scraped by the Kanji/readings of my N2 but have to do a chunk of my work in Japanese, I gotta disagree.

Sure, reading isolated sentences with only 90% coverage is really hard sometimes. But usually you're reading whole blocks of text, so you have context. Also, maybe you don't know some kanji but it has similar radicals to other, so you can make educated guesses (only sometimes of course).

I mean it's just like English! There are radicals/etymological clues, there are context clues, there's so much that you can work off of to figure things out. And exercising "figuring things out with partial information" is a valuable skill that can compensate for a lot of missing kanji knowledge.

An aside: I've heard that when learning a language, you should be able to understand roughly 80% of what you're reading/listening to, because then you'll have enough of a base to not feel overwhelmed, while still having new stuff to absorb.

I don't disagree with your point in general but the irony is that your first sentence is an excellent demonstration of the parent comments point.

I have no clue what a "N2" is and I'm not entirely sure what you meant by "Kanji/readings". There are only six other sentences of context and they didn't really help to understand the first one.

In case someone else wants to know:

From a quick Google I gathered that there are 5 levels of proficiency you can certify with. They are N1 to N5, N1 is the highest rating.

And still you're fluent in English (I assume) - it's not an issue with the English language but your understanding of the context. Japanese (and Chinese) work a bit differently than latin languages. Even if you know 100% of the kanji (very few Japanese people do), it doesn't mean you know 100% of the words - and vice versa. Since the characters are idiomatic, it also makes it easier to guess the meaning of a word or character you don't know than if they would have been written out phonetically.

Kanji: The Chinese characters in question

N2: The second highest level of the Japanese Language Proficiency Test (JLPT). N1 is by many seen as a very high bar and many Japanese employers demand it if you'll be using Japanese as your main working language. N2 is generally attained after 1-2 years of Japanese studies.

> Since the characters are idiomatic, it also makes it easier to guess the meaning of a word or character you don't know than if they would have been written out phonetically.

Ideographic, as in trying to portray ideas graphically, not idiomatic, as in using expressions like a native speaker

I had a feeling I was not using the right word, thanks for pointing out :)

Surely no one knows all the Kanji, since some of them have open academic investigation whether they even are a valid character or not?

https://www.japantimes.co.jp/life/2018/10/29/language/ghost-...

Indeed. Which means if your goal is fluency, it's silly to strive for 100.0%.

No one knows "all Kanji" much like no one knows "all English words" even in native English-speaking countries.

Do kanji handwriting-recognition models even “know” (i.e. are trained on) all in-use kanji, or do they just not bother with some of the rarest ones because their presence in the dataset would decrease the likelihood of correctly matching more-common characters, which it’s overwhelmingly-likely are what you’re writing?

Presumably it's the same as words in English, where you'd never make a word suggestion model that just blindly considers all possible words. You swipe jow or gow and it will change it to "how" despite the other two existing in arguably correct English in some very obscure case.

> "N2 is generally attained after 1-2 years of Japanese studies"

From my own experience, this would only be true in the most favorable of the situations. 2 years studying fulltime while living in Japan sounds about right. 1 year studying as a hobby few hours a week, no way.

Oh yeah, I definitely meant full-time studies coupled with practice outside of study-hours.

I think your estimate for N2 is a bit too optimistic: passing it in two years would be very difficult without living in Japan.

For context, it took me 3 years of university (outside Japan) to pass N2.

I think even if you reach the parent comment's 98% point, "N2" will not appear there. But 90% English is probably good enough to be able to reason "N2 must be some kind of level".

I've personally always advised my students to increase difficulty once they've reached 87.5% accuracy.

(I'm just amused of all the % thrown around I this thread)

Exactly. You need to be in the zone of proximate development or learning isn’t very efficient. I remember as a teen I had a fixed mindset and would get 100% on every calculus test. The problem became that I liked getting a perfect so much that ended up actively avoiding harder material and just sticking with what I was taught in school. I was afraid of posing interesting problems to my self out of fear that I wouldn’t be able to answer them.

This is a dangerous and corrosive mindset that did a lot of harm to me and other people I know. It took me maybe until 25 to actually start to accept my lack of knowledge and incompetence and start to do something with it. I sometimes wonder what my life would be like if I had the emotional maturity to challenge myself at a young age instead of pursuing destructive perfectionism.

> the irony is that your first sentence is an excellent demonstration of the parent comments point

And allow me to claim that you've actually proved your parent's point! I have no idea what an "N2" is either but from the context was able to accurately guess that it's some kind of proficiency test.

I thought his first sentence made his point. I didn't know what an N2 was either, but from the context I thought it could be some kind of test he barely passed.

Maybe whether one can make a reasonable guess depends on more than the context - perhaps background, experience, and point of view. I know I've missed points that are obvious to others because of our differing perspectives.

Tend to agree with you. Context is important.

If I hear something that amounts to "you idiot, get the schmonck out of my way", I'll understand even if I have no idea what "schmonck" really is.

>> An aside: I've heard that when learning a language, you should be able to understand roughly 80% of what you're reading/listening to, because then you'll have enough of a base to not feel overwhelmed, while still having new stuff to absorb.

Funny as this is how I learned (and have similarly described) to read language X and it felt ___almost___ like zero resistance. I learned first just to identify each character by name, then somewhat how they were pronounced, then vowels, then how there were pronounced, then special rules for X, then special rules for Y.

All of the learning material clump them together from start, a few consonants, a few vowels, some of their properties, and rules on how to combine them. All at once, this made me ignore writing for years until I tried it my way which I think worked well (although it took some time, still).

When I was learning English, the most memorable/epic lesson of advanced English I ever took was trying to understand what we were told was an "advanced" poem; the Jabberwock!

We were a class of 6-8 students, in pairs making our guesses of what the words that we didn't "understand" meant. We all got pretty advanced ideas and all got to similar conclusions. The teacher would tell us afterwards how those were made up words, and we were all in awe at how much we still understood and agreed upon the meaning of the words.

https://en.wikipedia.org/wiki/Jabberwocky

Sidequestion, is this poem taught in schools in the US by any chance?

Australian here. I studied it in school and so have my kids. They were equally amazed at how much meaning they could bestow on the words.

GEB by Douglas Hofstadter has an interesting section on translations of the Jabberwocky http://www76.pair.com/keithlim/jabberwocky/poem/hofstadter.h...

Yes. We even did a similar exercise in my class.

I agree with rtpg on this one - and disagree with the Diego.

I also read/type Japanese (my writing out of lack of continuing to write on a daily basis has gone to the dogs). With four years in Japanese universities and 20 years living in Japan I had the opportunity to experience what it is to get reasonably proficient in a second language.

The synaptic connections you can make by understanding 'most' of a compound kanji (multiple characters joined together) along with parts of the kanji makeup for a specific kanji you do not know puts you in an incredible position to figure out what the word is likely to mean.

Regardless, once you Japanese gets to a point, you begin to realise it is less about the meaning of words and more about the context and usage of language. You find yourself understanding how to use some words you have not studied without having said to yourself 'so what is this word equivalent in English' - and at times there is not a word equivalent in English. That is when you realise you are thinking in Japanese not just translating.

Synaptic connections, context and knowing what the relevant responses to how you want to respond are the key to fluency in my opinion. This is also the case for writing emails etc.

This also explains why people who do homestays where they get to watch people interact with each other progress faster than someone who goes to live with their significant other half. If you are involved in half of the interactions you get less opportunity to mimic the common responses.

Sorry for digressing a little on this one. Once you get that 777 kanji mark you would be well on your way to being able to contextually understand much more for the kanji/sentences you do not fully comprehend - which in turn leads to more kanji being 'gut-felt' understood if not fully formally learnt.

An additional benefit once you get some proficiency is that you can hear a word you do not know - but you can guess the half of the kanji being used in the compound word (and confirm by asking) and that gives you some context of what the word means. Very much like using latin/greek roots in English to breakdown words.

Time to go back to my coding before the day gets away from me. I just needed to respond to this one because I thought Diego's opinion on language learning was too far off the mark from my experience with it.

Being fluent in Japanese as a second language, I agree with this. It's Zipf's law and sounds great, but 90% isn't as useful as it sounds. You'll mostly recognize a single common Kanji of compound words consisting of 2-3 characters, or common structural words. It's a far cry from being able to understand content you find in the wild.

Also, "understanding" a Kanji is an ill-defined term. Most Kanji have multiple meanings based on context, and many different readings. So for each Kanji you're not learning a single character, but possibly a lot more. Especially the Kanji that correspond to more abstract concepts you cannot learn by themselves. They don't have a concrete meaning like "eat" or "drink". You essentially have to memorize all of their word compounds, a single character does not help.

EDIT: I also started out learning Japanese by memorizing the top 2000 Kanji using Spaced Repetition. While it definitely helped, it wasn't nearly as useful as some of these marketing-driven sites want you to believe. Kanji meaning are too complex to be captured like that. Even if you "know" all Kanji in a word you'll likely not understand the word's meaning unless it's something simple and concrete. I think you are much better of memorizing and studying word compounds. Over time you will automatically "pattern match" the Kanji you see often to their abstract concepts.

Example from 1min of browsing a JP text: Take "可能" which is a very common word that usually means "possible". Knowing the two Kanji (tolerant and ability) it not going to help you. It could mean dozens of other things based on those simplified Kanji meanings. This is not an exception, the majority of words are like this. On the other hand, let me give you a bunch of words containing "可": 許可、可決、可能性、不可欠 (permission, approval, possibility, essential) and you start to pattern-match that "可" corresponds to something like "positive possibility", but it's hard to translate.

Afterthought: Kawaii (cute) is actually 可愛い, also containing "可", which literally may mean something like "a thing that can be easily loved", or simpler, cute. But you wouldn't be able to guess that if you just know the Kanji.

Perhaps not the best example. Knowing that 可 is usually read 'ka' in compound words, and that 愛 is read as 'ai', you get to 'ka-ai-i', at which point the meaning of 愛 (love) will likely push you to かわいい (kawaii).

Of course, but you are assuming that you already know the meaning of "kawaii". My point is that if you don't know the word, you can't infer the meaning from the Kanji, except in very simple cases. That's why memorizing Kanji alone really isn't as helpful. Knowing 90% of Kanji meaning and reading doesn't help you much in understanding compounds/words.

> but you are assuming that you already know the meaning of "kawaii".

Have you ever been around non-native students of Japanese? This is the one word that is pretty much guaranteed to be known by all with an interest in anything remotely Japanese.

Your point is valid, but kawaii is perhaps not the right example.

Actually, I suspect it's a pretty decent example.

Most non-native students of Japanese will almost always encounter that word in hiragana, katakana, or even Romaji--encountering that as a kanji is actually remarkably rare.

The fact that the kanji for kawaii is an ateji makes it one of those odd ducks.

> Most non-native students of Japanese will almost always encounter that word in hiragana, katakana, or even Romaji--encountering that as a kanji is actually remarkably rare.

Interestingly, it's very common for students of Chinese. The word 可爱 (kě'ài, "cute") is dirt-common, but it isn't native to Chinese -- it originates as a loan from Japanese.

This isn't clear to the Chinese themselves, who use 卡哇伊 (kǎwāyī) if they want to refer to the Japanese word.

There's another modern word for cute, 萌 méng, which is also a loan from Japanese, though I think popular awareness of it as a weird loanword is higher, since its literal meaning ("sprout") is so far removed from the concept "cute".

The counterpoint is that if you are fluent in spoken Japanese you can get by with a fairly minimal number of kanji (and complete understanding of the kana as well, obviously).

Which is the trend with native Japanese people too.

I don't know for Japanese as meaning sometimes shifts from Chinese, but in Chinese the standard definition of 可 is "can, may, be able to".

You obviously learn it by itself but as Chinese words are mostly a combination of 2 characters, you immediately also have to learn e.g. 可以 (can, may, be able to), 可能 (maybe), 可爱 (cute) etc.

So someone who's learning characters in order to get 90% coverage (or whatever) would not simply learn characters but learn actual words. Learning characters in isolation would not be that helpful, indeed.

When you don't know a word (i.e. a combination) but you know the individual characters it is much easier to learn the new word either by guessing or checking.

In context, the meaning of 可爱 would be fairly straightforward to guess, for example. Even in English 'lovable' is a synonym of 'cute'.

> the meaning of 可爱 would be fairly straightforward to guess

To be honest this whole thread about 可愛い is more or less bonkers, because it's an ateji. The word's meaning doesn't derive from the characters, the characters got arbitrarily attached to an existing word because they were similar in sound and meaning.

As such, the whole thing is about as meaningful as talking about how easy it is to guess that 珈琲 means "coffee"...

I must say I don't know much about ateji in Japanese.

In this case, though it does seem that the characters where chosen at least partly because of their actual meaning.

It seems that it is both an ateji and a jukujikun [1] because the word does not come from the characters but the characters do have the correct meaning.

[1]https://en.wiktionary.org/wiki/%E5%8F%AF%E6%84%9B%E3%81%84#J...

> the characters where chosen at least partly because of their actual meaning

Sure, didn't I say they were in my post?

The point was, in a discussion of how well X predicts Y, it's not very useful to examine a test case where Y came first and X was chosen post-hoc to match it.

> Sure, didn't I say they were in my post?

No, quite the opposite actually ;)

> the characters got arbitrarily attached to an existing word because they were similar in sound and meaning

It does not seem arbitrary in this case because the meaning does match.

I do take your point that using that word in the discussion above, which is about Japanese was not the best example. On the other hand, it is a good example in chinese.

(A) In Japanese the meanings don't match that closely. The word didn't originally mean cute, but rather pathetic or pitiable, and evolved over time. More info: http://gogen-allguide.com/ka/kawaii.html

(B) By arbitrary here I mean that there is no linguistic connection. "Arbitrarily chosen because they are similar" => "chosen for no reason other than their similarity".

The characters of 可愛い do not have the “correct” meaning. Just like in Chinese, 可 means “can”, or “possible”, and 可愛い is a weird exception. (It’s also nearly always written in hiragana.)

Nearly every word involving the kanji 可 in Japanese has something to do with permission: impossible (不可能), possible (可能), permission (許可), approval (可決), etc.

That said, the kanji-centric view of the world is ineffective, and one primarily adopted by beginner learners who have not spent any significant amount of time studying words in Japanese. It is always better to just learn words.

If the characters were chosen because they fit the meaning, what does that change?

The thread was about how words' characters and meanings relate, and with ateji that relation is highly atypical, is the basic point.

Actually, no. 可 is like 'the be able to' prefix in Japanese/Chinese, so 可愛 is can be seen as a compound word because of that, literally means 'is able to be loved'

In Chinese, it can be further compounded into many common phrases:

可悲(pathetic, 可+'sad')

可气(annoying/irritating, 可+'anger')

可怜(pitiful, 可+'pity')

可恨(resentful, 可+'hate')

What would be a more suitable example would be 可(ke)乐(le) in my opinion, in which case it means Coke in Chinese, it is a transliteration.

Yes, Most of the time 可 means 'able' in '__able'.

But it could also be 'a thing that can be easily loved'. Rr simply, 'pleasing'. For example, 可口 means tasty. So, 可爱 could be understood in both ways actually.

I know there are a lot of ideas that there isn't a "right way" to use SRS, and I'm sure that someone would argue that what I am about to say is a no-true scotsman or whatever, but the simple fact of the matter is that you were using SRS wrong. In this instance 'wrong' means 'inefficient', 'suboptimal', and 'in a severely defective fashion'.

You don't just load single words up into it and study them on their own, you load words, compounds, sentences, sentence fragments, radicals, etc. etc. and everything else you can, and the SRS system helps you study and remember it. I can understand SRS not being as useful to you if you were only using it for single characters.

Just to clarify, I did use SRS for all my learning materials, including words and sentences. But I also had decks for single characters (you can find a lot of these online). Over time I found these pure character decks to be the least helpful or nearly useless, so I dropped them. Using SRS for other things like grammar and words/compounds is great.

> Even if you "know" all Kanji in a word you'll likely not understand the word's meaning unless it's something simple and concrete.

I don't think anyone passingly familiar with Japanese thinks otherwise. That is, I think you're arguing against a position here that nobody actually holds. The argument for memorizing kanji is that it makes it easier to learn compound words, not that you'll just know them without learning them.

I agree that people with some Japanese knowledge likely already know this. But out there you will find a lot of marketing material/posts targeting absolute beginners promoting some kind of "Top X% Kanji lists", as if they are a huge shortcut and secret to quickly learning Japanese. So I'm just saying they aren't.

If you talk to people who don't know much about Japanese they often believe that memorizing the characters is the difficult part and doing so will help you understand a large fraction of written text. I think it's quite a common misconception.

I feel like a lot of this perception comes from a tendency for people to equate Kanji characters with words. I always explain to others that a character is kind of like a root, e.g. Sub-optimus-al = suboptimal. It would be crazy to say one can learn English by memorizing just a few hundred Latin roots and suffixes, and in fact such a claim is so irrelevant nobody even keeps track of the statistics.

This is why I created an Anki deck that contains both kanji and common words that use those kanji [1]. It even includes statistics on how much each reading is used, so you can estimate if it's worth learning this reading or treat it as an exception.

[1] https://ankiweb.net/shared/info/831167744

Let me add that just as it's not enough to focus on learning Kanji, even learning compound words made out of Kanji is not enough. There are many Japanese words that are never or only rarely written with Kanji, and depending on the text there can be many sentences with no Kanji at all. I recognize 4000+ Kanji by knowing Mandarin, so I can somewhat read e.g. software manuals (many Kanji, some English loanwords) but children's books are too advanced for me.

The "777 Kanji for 90% coverage" figure is probably more relevant for illiterate native speakers. Based on a corpus of 167,281 Japanese sentences I had segmented and lemmatized some time ago, you'd need 2,685 words for 90% (up to "故郷", "birthplace", usually written "ふるさと"), 6,564 for 95% (up to "すらり", "slender, smooth"), 14,098 words for 98% (up to "鼻先", "tip of the nose") and 20,657 words for 99% (up to "恐慌", "panic"). Obviously the exact numbers depend a lot on the diversity of the corpus, so don't mistake them for fixed targets to aim for.

When did you start studying Japanese?

I agree, and that's why I find wanikani so useful. After 7-8 years trying to learning kanji and always failing after 3-400, I have been using wanikani for ~ 6 months, and the system of building vocabulary on top of kanji on top of radicals really did it for me. I learnt 600 kanji in 6 months, ~ 2000 words of vocabulary, and I am quite confident I will reach 1000+ EOY.

Even at the beginning, learning the simple kanji, having the vocabulary I knew/heard living in Japan associated to the kanji made it very useful from the start.

For anybody learning kanji with anki, etc. and failing, I recommend giving wanikani a try. I am just a happy user. Disclaimer: I live in Japan, work in a mixed English/Japanese environment and speak simple Japanese w/ my wife every day

There’s some level of proficiency (whatever the percentage is) where each sentence is understood save for one word, and that word because of a single character that you can then look up in a dictionary. To me, that’s a useful level to achieve, because it means that you don’t have to constraint-solve sentences by guessing at the meaning of one character to decode the meanings of others, but instead can just hold onto the complete sentence minus one “hole” in your mind while you go look that “hole” up. Much less mentally-taxing!

> 90% coverage is not that great ... when an uncommon word appears

TFA muddles this somewhat, but the research referred to has to do with coverage of kanji characters in a corpus text, not words. Someone who can read 90% of the kanji in a Japanese text might understand significantly more or less than 90% of the words, depending on the text.

Also notable: the research behind this was based on newspaper text corpora, which one could expect to have a lot of kanji and kanji compounds that are uncommon in casual usage.

This is why picture books are so important for improving listening comprehension for young children. It gives them something to focus on so that hearing words they don’t understand or new grammatical constructions is less distracting, it provides visual cues related to some of those words and helps guide understanding of the story, and it gives the children and whoever is reading to them something to point at in discussion. It also makes the stories more appealing and makes repeat readings more rewarding.

I would guess (based on not-too-rigorous anecdotal experience) that the optimal level of word familiarity for most rapidly improving listening comprehension when read-to 1:1 is closer to 90% than 98% (but with the caveat that books should be re-read several times). The big difference is that having someone to talk to makes it very quick and easy to ask what words mean and discuss other aspects of the story.

When read to for an average of 1+ hour per day, young children improve at listening comprehension very quickly.

why picture books are so important

Any references on this? Interesting.

I am just talking about my own experience. But there are thousands of papers about early language development (which I have not read).

You could start with a google scholar search like https://scholar.google.com/scholar?q=storybook+OR+%22picture...

OK.

Was wondering if that was research based or just personal experience. The hypothesis makes sense, but I'd love to see it substantiated.

I’ve been looking into the research on second language acquisition a bit, and the primary mechanism for learning a language appears to be hearing/reading understandable sentences for content, rather than structure. When learning, some sentences in the text will make sense to you and others will be out of your reach.

I suspect that the role of pictures (or any other context) is to help correct your mental state of the narrative world so that the sentences you don’t understand don’t present an impassable barrier, preventing you from getting to the later sentences that will help you learn.

'Tosche Station' and 'power converters' obviously - see 90% is all I need!

I noticed that it takes very little to understand what a conversation is about, but understanding the specifics is a different story entirely.

For example if you just understand the word "car" and the name of a place, it most likely is related to traffic. If that's coming from someone arriving late you can bet he is finding excuses ;)

That's a reason why you shouldn't bad mouth foreigners if they can hear you based on the premise that they don't understand the language. They may not be able to converse with you but they can still get the general idea.

> son, if you go to the plumbf tomorrow morning don't forget to pick up some zlonks

As a counterpoint: “son, if you go to the jawn tomorrow morning, don’t forget to pick up some jawn”

https://www.atlasobscura.com/articles/the-enduring-mystery-o...

We use 'vaina' for the same jawn in Bogotá.

Vaya mañana a la vaina esa y me trae la vaina.

Sabe qué, dejémonos de vainas.

In fact, we also have another word that serves both as a multipurpose substantive and an multipurpose adjective:

"¿Yo dónde dejé esa chimba?"

"Está muy chimba su carro"

"Esa chimba estaba muy chimba"

But in what order will you learn all the 99% (10k? 20k?) words?

I don't see anyone claiming: 'learn this 777 characters, and then STOP LEARNING'.

You learn the most frequent first. Then you start reading. Much easier than when you only understand 10% of the words.

So, you get to 90%. Then you read some text and you start to find new words from time to time. That's when you start using a dictionary.

Heck, I'm an avid reader and I am still finding new words (or old words with new meanings) in my native language!

It is easy to lookup or ask the meaning of the 1-3 words you don't know in a sentence. It is best to always have some percentage of the content not be what you know, otherwise you will plateau.

If you are speaking/writing, much easier to form a sentence and search for the one word you don't know than search how to generate an entire sentence structure.

For Japanese it can be different, as most structural words are in Kana, a phonogram, instead of in Kanji.

This. I peaked at about 1000 kanji in active memory. It was still not enough to read newspapers easily. Joyo kanji ("regular use Chinese characters") have about 2100 characters, which any high school grad should be able to read. Anything much below that level and you are guessing/looking up a lot of content.

You are better off trying to cover specific domains. I was able to read technical articles related to hardware and software topics pretty easily.

I would agree with that. I know about 1000 German words, and that covers the bulk of words people use when speaking German, but not the words that matter.

You say that like it's a bad thing. I wish I could decipher that much of a random sentence in Japanese. At least then I can ask about the parts I don't know. And for most people learning Japanese, they'll probably already know the word in hiragana or katakana, knowing kanji is just icing on the cake.

I don't have sources for this, but I am quite sure that 90% coverage of kanji occurrences will give you >98% comprehension coverage.

(Here I make the assumption that you'd know all the readings and all but obscure or archaic words for a kanji to count)

Your example is too convenient. How about "son, if you go to the butcher tomorrow morning don't forget to pick up some zlonks."

With a bit of context it's usually easy to figure out the missing parts

Just out of curiosity, how many kanji would you need to know for 98% coverage?

Edit: Doh, the OP link actually includes this statistic.

７７７ characters: ９０.０％ coverage １４７７ characters: ９８.０％ coverage ２４７７ characters: ９９.９％ coverage

Japanese learn 2000 or so kanji at school so I’d say that’s a good ballpark

https://en.wikipedia.org/wiki/List_of_jōyō_kanji

I laughed so hard at your great example and read it to all my friends sitting in the room.

Your example is more like 85% coverage. Not that it changes your point much.

>The thing is, 90% coverage is not that great. What happens is that you understand common words that make up for a lot of structure, but when an uncommon word appears it's probably important to the sentence.

If you have 90% you can more often than not guess the uncommon word...

That's if you know 90% of the words.

If you recognize 90% of the kanji and know what they mean, that's not the same as knowing words.

>That's if you know 90% of the words

No, I'm taking about the premise of the article: if you know 90% of the words most widely used (that is, the above 777 characters for example). Not 90% of the words in general.

If you know the most likely words, you can more often than not guess the unknown words in a phrase from the context.

I'm also talking about recognizing 90% in whatever text you're reading, not about knowing 90% of the lexicon!

My point is that those characters aren't words.

Well, some of them are, sometimes.

especially since it's the highest entropy words that are missing, by definition... so you always get hit the hardest

Reply on news.ycombinator.com