Then indeed there would be nothing to take issue with. However to make the analogy extremely tenuous (at best), one needs to show that there is no baby, rather than just claiming there isn't.
Galileo, whose name is invoked frequently in these discussions, speculated broadly and was often wrong.