This is a confusion between the gravitational force that the core feels from the layers above, and the pressure that the layers above push down onto the core.

That is, the core doesn't feel gravitational pull towards the outside, because it's balanced on that other side, but the outside is being pulling in towards the core, which pushes on the core.

It might be easier to simplify the mental image by picturing just a slice of the whole thing.