I personally have never looked to work to satisfy my passion (for computers). No matter the job I am working at, I have a steady stream of side projects and try my hardest to keep up with a small bit of the massive firehose of change that is modern computer science/programming.
Don't worry about being average. Many of the people who have created great things weren't that far from average. They just seem like geniuses in retrospect. Also, it's about 5 billion times (give or take) easier to do software these days, you have a huge leg up on geniuses of the past.
Find a project to do that makes you happy, and do it until it doesn't make you happy anymore (hopefully, after it's finished)