I'm not against colonizing space itself, but early seagoing explorers didn't attempt to colonize the ocean. We need to establish ourselves in environments that allow easy production of oxygen, water, and food.
In the end, it is about money. It takes a lot of money to get setup on Mars, but after it is setup, it is a HUGE resource, which means much, much more money. The moon could also be a huge resource, but it is expensive to colonize because it can't hold atmosphere easily. Colonizing space itself without mining a resource in a way that eventually will pay off the initial investment is useless and a drain on our society.