Of course it's ruining skills. Anyone who has a really well honed mental skill knows that we get rusty really damned fast (e.g. musician at a professional musician level, which happens to be my reference point). You keep shedding, you stay sharp. You take more than a week off, it's going downhill.

There's no way this isn't happening with other skills. No goddamned way. Anyone who tells you otherwise is grifting.