Take the problem of Integer Factorization, something that I have personally been fascinated by for years. The problem is so ridiculously simple that a 5 year old could understand it. With that said, google for a few wikipedia articles and see the notation and the way it is presented. The non-mathematician does not care about difference of squares or triangular numbers or what.have.you.
It seems to be the case that mathematicians are unable to talk about math without requiring the person they are talking to to have extensive knowledge of the topic. That's IMO why Carl Sagan and Feynman were so great. They could take complex problems and reduce them (read communicate) in a good way that most could understand and reason about without needing to traverse the entirety of the knowledge pyramid. Similarly, that is what Numberphile does so well.