You of course are right. Tail between legs is embarrassing but in the end you won't go hungry or homeless.
Problem is in popular culture there are also stories about the 1 out of 1000 guys (just like you) that essentially gambled and came out ahead. Let me say that again. Gambled and won. You hear those all of the time. What you don't hear is about the ones who lost or hit the skids as a result of those chances that they took. Or maybe you do (there are always counter stories floating about) but the shine of the winners typically totally blocks those out.
For example I am sure in response to this thread someone has already mentioned the type of family that Steve Jobs came from.