> I think it's the different point size (or whatever that word is for the horizontal height lines that fonts live within).

Agreed! The x-height [0] (among other things) differs jarringly between the serif and sans fonts used.

0: https://en.m.wikipedia.org/wiki/X-height#:~:text=In%20typogr....