If you are using GA code that is at all recent, it should have no impact on the page speed as it loads async. All the parts of the page that Google cares about will already have rendered by this point.
I've still gotten a perfect score on PSI with GA enabled, but it is one piece of the puzzle.