I spent one year working on an open source “side” project full time. I had savings and my wife continued to have income so I had plenty of safety net (thanks honey), with a goal of seeing whether I could sustain doing it indefinitely. My experience doesn’t help with the social security part of your question (we also were in the US) but maybe can help with the rest.
tldr; it didn’t work out financially but I’m super glad I tried, if only to not regret not trying. The biggest thing I didn’t expect was for capitalist instinct to take over: I liked working on open source because I could work on what I wanted to; but to work on what I wanted to indefinitely, and play the latest video games and go out to eat sometimes or whatever, I needed the thing to make some money.
So an unplanned, surprising amount of time went into revenue generation. I previously had a Patreon page that brought in ~$20 / mo, and during my year I got to ~$500 / mo (Patreon + Stripe). I also picked up some contract work for a few hours a week. No matter what happens the bill comes due, in more ways than one—I grew a new respect for founders, who have more than just themselves riding on this.
Even so, it was freeing to turn my nights-and-weekends fun work into full-time work, to spend stupid irresponsible amounts of time on small things like polish, to be able to drop what I was doing at any time to go hang with friends or walk the dog or go get eggs, and to dip my toes into every trade from design to swag to marketing to customer support to user research. I also used my free nights and weekends to learn a foreign language at a local community college.
Wouldn’t trade it for the world, but probably also won’t be doing it again until retirement.