No, I don't see any reason to think that. If anything, it will improve your Page Speed Insights score, which will improve your rankings.
Google collects all they need and more from users using Chrome. They don't need GA anymore, and I'm pretty sure they said a while ago they were going to stop looking at the data (again, because they collect far more data from user using Chrome).
Good on you!