When compared to the reality that almost an entire race of people in North America were stolen from their homes and brought to a new land as slaves to benefit their white owners... any potential wrong you are focusing on here is only visible with a microscope.
Your bad behavior doesn't need intent. You didn't need to intend to be a racist when you put him under a microscope like that -- but that doesn't mean you weren't racist when you decided it was important enough to write about. Especially considering the exact behavior is frequent among all races.
And it's the top comment at HN. I can't help but think this says something important about what is really going on.