Is that a problem? Panpsychists don't seem to think so. Why must human beings believe that, in all the universe, they are a particularly special arrangement of energy?
As to whether there is something special about it, I personally feel that the difficulty of explaining it is enough to regard it as such.