When the event occurs that wipes out the ability of humans to live on Earth, no amount of money spent on space exploration will seem like it will have been too much.
Such an event is coming sooner or later, from one source or another; probably not in our lifetimes but we're so far off from good solutions that "we have plenty of time" is not a good excuse to not be investing in space.
As an "average (tax paying)" person there is a list a mile long of shit we spend tax dollars on that I'd rather cut over space exploration, with the military (over $500 billion a year) being an obvious start.