If the galaxy really was full of intelligent life it's likely we would see evidence of it in some form. For one, it seems probable that any expansionist intelligent life in our galaxy would eventually notice our Earth is likely to be an interesting place. Even if they didn't visit us personally it seems strange they wouldn't send a simple probe or try to make their presence known to us in one way or another.
But to your point, perhaps intelligent life is quite common, it's just rare for it to be expansionist - at least beyond it's own solar system. In which case yeah, there could be a lot of Earth-like civilisations out there, but we'd be unlikely to see evidence for them.