Consider if filmmakers were advised not to go to film festivals or authors were advised not to go to book festivals until they're noteworthy enough to get invited. I'm not sure that would be a winning approach for most.
I oversimplified a bit for readability and I'm thinking specifically of technology entrepreneurs.
I don't believe the value of conferences is absolute zero but the point would be that spending your time building a notable product is far more valuable than spending your time at a conference.
I've spent a fair amount of this year attending conferences (most notably SXSW) and this post was a reaction to my experiences.