I genuinely thought everybody did the same until went to a clinic by Pepi Taveira, an argentinian jazz drummer, where he admitted doing the same - that unlike everyone elde, he didn't kept a count but kept the measure reference by learning the melody - specially on odd time metrics.
Ask me to do anything beyond add/subtract/divide (forget algebra entirely lol), though, and my brain shits itself. Even at a certain point, division becomes complicated for me. But I will hit that pocket and never, ever lose the beat every single time, and that's always been interesting to me.
If you're in the pocket anything you play will be on time.
Pretty much anyone can stay in a 4/4 time if they have a lick of musical ability.
With the weird time signatures like 13/8 or 7/4 it can be tough to find the pocket, so in those cases the vibe tends to divide into 2 sections that fit right at first (like one section of 7/8 and one of 6/8, or one section of 4/4 and one of 3/4) and you bounce between them until you sew them both together and then you can stay in the pocket for all of them.
It's harder to explain via text than it is in music practice.
What's weird is there are bands that deliberately play odd meters in their time signatures to match some sort of math concept that is just no where near the same likeability as what Tool does with their Fibonacci sequence and other things. I don't know if they are just lesser musicians, but it definitely feels forced. This is what I notice a lot about some jazz where they are doing it for the sake of doing it not because it sounds good.
It's definitely a case of learning the rules so you know how best to break them. If you just start breaking them without knowing why they are there in the first place just sounds bad to me.
I think the art with that sort of stuff is making a rhythm and melody for the vocal, or lead instrument that feels natural, yet fits over the odd time signature.
There was a (slightly obscure) branch of Jazz funk in the late 80s early 90s called M-BASE that had a lot of this kind of stuff.
This track is called the X Format, by Steve Coleman, and i think it's in 13/8 time. But its always boggled my mind. It must be devilishly hard to play, but it feels quite groovy.