I can relate; I've been a lone wolf for quite sometime - I'm a bit of a social anomaly in that I don't "fit" into
any one cultural category. I don't separate my life into "personal" and "professional" therefore I have (for the most part) just one persona most people see.
Note: I am about to give you a rather long preface with a lot of personal detail to give you an accurate picture of "where I am" so that my answer to your question can make more sense and (hopefully) be helpful!
I dance for a Salsa team, I sail, I've been on a number of survival trips, I'm an avid snowboarder, I lived and worked on a commune for six months (this was a life changing experience), I backpacked through India on a low budget for two months, I love Dubstep and the electronic music scene, and I consider myself a rogue intellectual.
Did I mention I grew up on a 6 thousand acre working ranch?
I dropped out of highschool in my senior year, got my GED, and tried "college" twice. I dropped out twice. Around that time I chose to move out and live on the commune I mentioned earlier. There were only three of us but we were active: building structures, digging trenches with a Backhoe, rebuilding the foundation of a 100 year old house, clearing forest, and building a 100% off-grid solution complete with solar panels and composting toilets.
The man that was the accepted foreman for the project was (still is) an amazing friend and teacher to me. He was first and foremost, a powerful intellect; he was also skilled in any number of blue-collar activities. He was the man that ironed into my head that college is not necessary, that one can instruct themselves on anything. He sparked my interest in programming, psychology, metaphysics, physics, mathematics, logic (as the topic), personal development, introspection, and a long list of other intellectual subject areas.
He taught me that there is never a crowd on the leading edge.
I've had a self-compiled curriculum for the last six years of my life since then. I am an autodidact in every sense; I've read Ovid's Metamorphoses and critically interpreted it using what I learned from Jungian Psychology (I don't read passively); I taught myself PHP, then Python, then Erlang, then Scheme, then Objective-C, and others. I've pursued Mathematics with a passionate heart (whereas in Highschool I loathed the subject), I read books on computer science, books on logic and formal reasoning, books on how to write eloquently (don't use this comment as an example though!), and outlier books that make modern day physics look like a jumbled mess.
Presently (since it is an American tradition) I make my money from independent contracting; but, I started out waiting tables. I then moved on to building my mom's website, to programming for an entertainment company (at $10.00 per hour), to developing for a large startup (bit of an oxymoron I know, but that's why they don't exist anymore), and a string other 9-to-5 jobs.
Note: this part may be of interest to you, as it pertains more to your question.
I got sick of 9-to-5 and decided to quit the working world so that I could focus more on my self-education and personal projects. This lead me to an eight month stint in which I did only that. I paid for things with passive income and lived a slim life style. Until I decided to take up contracting work and to work for myself (it's been great ever since). I also had been working on an open-source project for the last two years as well.
This super long comment has (more or less) lead me to the present day me. I've found solace in coworking spaces, my Salsa team, hacker meetups, and my online friends. I still have yet to find an actual cluster of intellectually deep and potent people (hackers, you could say).
My present idea has been what I call "The Enclave" as a sort of hackerspace for thinkers and builders to come together. A place where these people can interact without the disturbance of "Normals" (most fringe intellectuals have quirky personalities and strange interests), a place where they are not just "accepted" but "allowed" to be what they are, a place in which deep collaboration can occur on topics ranging from the technical to cooking and gardening to philosophy or pseudoscience.
This is how I get my juices flowing, not by just working on OSS or working (I do all of that), not just by starting my own little pet software project (I do that too), but by dreaming of how we might solidify the etheric space most individuated people interact in (the internet, online spaces, etc...). Hope that is at all interesting.