You also have to take cultural and linguistic differences into account. In my experience, Japanese people tend use terms like "atheist" and "non-religious" less in the sense of a personal system of belief, and more in the sense of "not being a member of a formal, established religious institution."

I've had variations of the following conversation with several Japanese people:

  Q: Are you religious?
  A: No I don't have any religion.
  Q: Oh, I see. By the way, what did you do on New Year's Day this year?
  A: I went to the neighborhood shrine to pray, like I always do!