It gets worse. Whales and dolphins are more closely related to certain fish than those fish are related to some other fish. (As are we by the way.) There are also some fish, such as lungfish or certain eels which can spend more time out of water than whales and dolphins can. Some fish can even breathe air, which undermines another assumption about whales etc.
For some reason, I rarely hear fewer people objecting to jellyfish or shellfish being referred to as fish, even though these are more distantly related to fish than mammals are.
So biologically, the term "fish" doesn't make much sense.