Hacker News Reader

Comment by jonplackett | Hacker News Reader

I once heard about a meeting where someone did a presentation for the best part of an hour to a room full of Japanese people. They all nodded along, smiling as the presentation progressed. It wasn’t until the end that the presenter asked them what they thought. They all just kept smiling, but no answer. Someone at the front turned and talked to the room (in Japanese) then turned to the presented and said “they do not speak any English, but they are sure from this presentation that you are very, very intelligent”.

I feel like the Japanese people. I don’t understand what you’re doing here but I am convinced it is very clever.

View on HN

Anecdotally, based on experience with shipping software with an annoying bug to Japanese consumers, the Japanese people are way more patient than Americans.

I worked at an American company that made utility software for DOS/Windows. A major Japanese distributor liked our stuff, and made a distribution deal with us. They also helped with localization, and suggested feature changes to better fit what Japanese consumers wanted. Later this expanded to where they would also suggest new products they wanted for Japan and we'd develop them.

One product under development was very rarely causing a system lockup during installation. By "lockup" I mean it appears completely dead. No mouse motion. No keys worked. Hitting "CAPS LOCK" would not even toggle the light on the keyboard. Hard reset seemed to be the only way out.

It was rare enough that we always saw the install attempt after that hard reset work.

We just could not figure out what was causing this. The Japanese distributor decided we should go ahead and release, and their tech support people would deal with any complaints.

So we released...and their tech support got a lot of calls. People were hitting it a lot more than we had seen in testing.

...except they were not calling to complain about their system locking up. No, they were calling to complain about a slow install.

Apparently, when it became completely unresponsive all you had to do was wait 20-25 hours and it would complete the install.

I cannot imagine any American consumer whose PC has become completely unresponsive going 20-25 hours without giving up and resetting the thing. Not only were there such people in Japan, there were a lot of them. We didn't get one complaint about a lockup--all of the many many people who called called about it taking around a day to finish.

(Oh, and the knowledge that it was not a complete lockup was enough of a clue to let us figure it out. It suggested it was some sort of timeout, not a lockup. Our product needed to know what optical drives were available, and it turned out that the way it scanned for them did not get alone well with one particular brand of optical drive controller card, which could trigger about an hour timeout per drive checked for. We switched to a much more cautious drive scan, and the problem went away).

Lore from an old company of mine says that we had the opposite of this happen. We had an opportunity to expand from the US into China. So a dedicated team spent months localizing our English-only software into Chinese.

Our sales people traveled to a key meeting with execs at the Asian division of a multinational corporation (the potential customer). They did a big demo to show the software working in Chinese, hoping to impress the execs with how the software was ready to go.

The execs awkwardly said essentially, "Hey, that is probably great, but even though we handle the Chinese market, we don't speak Chinese; it's only our customers that do. So we couldn't tell what your software does, and we wouldn't be able to use this. Do you have an English version?"

You need to watch his video, very cool, it really helps to understand how this works: https://www.youtube.com/watch?v=f9EbD6iY9zI

That video is really, really awesome! And it won't leave you feeling "Japanese" either. (Which is a great people, btw. I'd really like to go there someday, mostly for the food and language and history. And Anime also, I'm forced to admit.)

There's a few things in there that are factually incorrect -- in particular, the false notion that "every input has a unique output" can be quite dangerous in some cryptographic settings.

That said, the purpose of this talk is about the mechanics of the function, and not its properties or how to use it safely. So don't let that detract from what is, really, an awesome presentation.

I'm sorry, could you please elaborate? I was always under the assumption that hash functions have to be deterministic, and thus, that "every input has a unique output" was a correct statement.

AFAIK the contrary is invalid, so that "not every output is the result of one and only one input".

There are 2^256 potential outputs for SHA-256, while the number of potential inputs is infinite. Therefore, the same output can be generated with different inputs, although finding such "collisions" by chance is extremely unlikely

The claim is not that every output has a unique input, which would not be correct, and seems to be what you are addressing.

at 1:08 in the video, that is exactly what he claims:

"So every piece of data in the world has its own unique hash digest."

This is false for the reasons apeescape describes: every piece of data in the world has its own hash digest, but these hash digests are not unique.

On the other hand, if we can count "every piece of data in the world" then we can estimate the probability of having a collision.

Yes that sentence is technically incorrect, but practically correct. We've never found a collision and though we expect it to be theoretically possible, even common if you consider "all possible inputs" and the pigeonhole principle, for practical purposes hash outputs are unique because nobody considers "all possible inputs" when evaluating probabilities.

I'm saying that for a layman explanation, it's reasonable to say that hash outputs are unique. Because following that with "technically, it's more 'practically' unique, theoretically there are collisions but you won't encounter them with probability > 2^-256" (or whatever it is) just confuses the topic to them more than just summarizing. You have to admit that most people won't go on a 200h adventure to learn about the state space of 256+ bits and how to conceptualize tiny statistical probabilities, so there must be a point where you have to cut the explanation to an approximation of the truth. This is true in every field.

I don't like to leave holes like this in people's comprehension. It's OK if people don't end up with an intuitive feeling for how relatively unlikely different things that don't actually happen are, but I want them to be aware of that category as distinct from things which can't happen because the type of argument needed is different.

The air molecules in the room you're in can't all gather in one corner because that's not possible, it's forbidden by conservation rules.

But they won't gather in two opposite corners only because that's so tremendously unlikely, it would be allowed by conservation but statistically it's ludicrous.

The same is true at the opposite end of the spectrum. Almost all real numbers are normal (in all bases) but the nature of "Almost all" in mathematics is different in an important way from "All" and I want people to grasp this difference when I'm discussing properties of numbers. It definitely is not true that all real numbers are normal, you probably rarely think about any normal numbers at all.

> I don't like to leave holes like this in people's comprehension.

I agree. I think this wording would be better than in my previous comment, what do you think?

    it's reasonable to say that hash outputs are *almost surely* unique

Kind of agree w you, but still feel adding a few words by way of a disclaimer about collisions is much better than presenting as plain truth something that merely approaches it.

> I'm saying that for a layman explanation, it's reasonable to say that hash outputs are unique. [...] theoretically there are collisions but you won't encounter them

You could have said exactly the same thing about MD5 right up until you couldn't. Then you could have said "oh yeah well MD5 is broken, but it's safe to assume you'll never find one for SHA-1", right up until we did. So if you say "oh yeah well SHA-1 is broken, but it's safe to assume you'll never find one for SHA-256", I disagree.

It would be one thing if collisions in hash functions were found by just repeatedly hashing things until you find a collision. If that were the case, then yes, I'd agree with you on those 1-in-2^256 odds, at least for a while. But by and large, that's not what happens. Over time, weaknesses are found in algorithms which allow you shrink the search space, which significantly changes your odds.

I see what you mean, but it sounds like the output is unique, and we probably agree that in this field you need to use sentences that cannot be easily misinterpreted.

A function being deterministic means that any input will have a single output. But it is not unique for any hash function, SHA-256 included. The definition of a hash function is any function which takes an arbitrary length input and outputs an n-bit output for some fixed value of n. By virtue of the fact that you have infinite inputs and finite outputs, the outputs cannot be unique.

Generally when people make this claim, what they're actually referring to is what's called Collision Resistance (CR) and/or Weak Collision Resistance (WCR), which instead make claims on difficulty of finding such collisions (of which infinitely many exist).

WCR, necessary for almost any cryptographic use, states that for any given input it should be difficult to find a different input which hashes to the same value. CR, generally desirable for cryptographic hash functions, states that it should be difficult to find two different inputs such that their hashes are equal. CR implies WCR, but WCR does not imply CR -- for example, SHA-256 (currently) exhibits CR but SHA-1 only exhibits WCR.

Did you see the README.md? It explains each of the steps.

Thanks for pointing that out - I kinda missed it until seeing your post. I went through each step - it's still a bit convoluted, but definitely helps me appreciate hashing functions. It's probably very easy for a poorly coded hash function to accidentally wipe important data or accidentally and a bunch of data with zeroes, so it's pretty cool to see a lot of ROT and XOR usage which does more interesting things with the original data.

Had no idea about the prime roots and multiplication, that's pretty clever too.

> “they do not speak any English, but they are sure from this presentation that you are very, very intelligent”

I kind of hope there's some nuance to this story and it didn't quite happened in the way you described.

If they think it's "polite" to let a person talk for an hour without understanding a word of it (while nodding and smiling), and then have the nerve to call it "intelligent", that's an insult to both the presentor and the concept of intelligence.

Cultural differences notwithstanding, the idea of "completely wasting someone's time while pretending to pay attention" shouldn't be too difficult to empathise with.

It's probably mixed with avoiding confrontation and thinking that the other people understand it. I'd assume that calling it intelligent didn't have a belittling tone, but rather a 'still awesome, sad I didn't understand it' one

i just felt like i was in the matrix for a little bit and that was ok for me :)

You made my day

Reply on news.ycombinator.com