Even if you remove those, look at the bottom of the data, across the horizontal axis. There is still a downward trend, although that is an eye judgement as well. That is, if you bin the kids according to empathy (x-axis), the lowest math achiever in each bin is worse as emphathy increases. The max in each bin doesn't seem to get better, however. The slope would definitely be milder if those offending points were removed.

Moral of the story: higher N is always better.