I've never commented before so I had to make an account for this. Hopefully it doesn't get flagged, but that's the risk of lurking I guess. The rest of this post is anecdotal and perhaps fairly unusual, so forgive the lack of citations.
I'm 1995 I moved from Canada to the United States. Up until then I had been in the public school system since kindergarten. After 1995, I was "unschooled" from late 1995 to 2001, and I have to take exception to your idea that it is a privileged experience. I'll spare you the details but the summary is that my mom was more worried about the physical and mental abuse I encountered in public school (U.S. 5th grade, around 10 years old average and the first year of "middle school") at the hands of my supposed peers, primarily based on physical appearance and "foreign" + "new kid" status.
So after this very brief stint in American public school I was told by my mom that I would be doing "homeschooling" instead of getting my face rearranged for being white (whoops, spoiler).
Interesting fact: my mom had no idea how to homeschool. We had food stamps, sometimes, but usually it was just living off of combinations of flour, water, and salt. She became "religious" in order to get food boxes and a hope of networking to get work. We lived in rooms most of you would consider closets.
Her version of "unschooling" was to tell me that I need to learn math, science, history, and English, that way I could get a job when I grow up. She walked with me to the nearest library, and got me a library card, and told me (paraphrased due to memory): "read until you figure out what to do" her tone implying that I should learn the topics she mentioned.
I did. I also worked; I didn't have a computer at home (yet) but I knew how to use VCRs to record cable, and I could move furniture (even as a 10-11 year old), so my mom's church network got me a few dollars a week to save for myself. I saved up and bought a Commodore 64 from Goodwill after a big fight about "useless shit."
I taught myself BASIC, at first from a library book that showed how to draw Christmas trees and math problems (original leetcode I guess), then from the help files. When I wasn't doing that, I was reading science fiction. Absolutely no educational material.
My mom soon made enough to get a 386 Acer PC on credit, and I learned QBASIC with its infinitely more helpful help system. The computer had a modem and by late 1995 my mom had learned about ISPs from her friends at church. She called the owner of the local ISP and essentially traded me for internet access; I would go to their "office" (a repurposed shipping container) to answer tech support emails, and at home we could dial in for free.
This is where "unschooling" truly began. I had Microsoft Encarta that came with the PC, I had a book called Internet For Dummies, and I had virtually unlimited access to information.
I also ate once every 2 days, and it was rice and shitty meat more often than not.
I won't say where I am today, but I will say that it all worked out for the both of us.
Hope that helps someone.