It wasn't that far-fetched as it sounds now: using Maxwell's equation, one can easily derive the speed of light, c. But then the question is: speed relative to what? Because Maxwell's equation doesn't tell exactly what it is relative to.

But, as we all know, speed is relative, so it must be relative to something - then the something ought to be the medium for light. So people started looking for it - and it refused to show up even when it should! The famous Michelson–Morley experiment.

Of course, it was resolved by theory of relativity: it turns out the speed of light is relative to everything! You just have to slow down time itself, no big deal.