Your ruler is absolute distances, like the speed of light. You can inflate or deflate the balloon and the number of marks on its surface won't change. But the time required for the light to travel between them will.
Relativity probably doesn’t work this way at all, but if light did actually slow down, would it be perceived by us as light moving at the same speed across a greater distance?
As to the previous question about relative size. If the universe doubled in size, the objects within it such as atoms don’t also double in size. Gravity and the atomic forces don’t change, so eg the size of stable electron orbits or planetary orbits don’t double in size.