1) You should have a rating system for experts and award different level of badges and users should write reviews also experts with high scores should have a higher pay rate
2) interviewing experts is not going to scale not only you will not have the time but you also need internal experts to interview the Become an expert" applicants (you need some fast vetting process or rely on the rating system)
3) You should collect data from StackOverFlow and other sites and see which questions novice folks are having a hard time getting answers to and try to get experts on those topics (this could become your competitive advantage)
4) You can scale this to businesses - they would certainly appreciate the on-demand help aspect of this thing. Businesses specially small ones may have support account subscriptions with vendors like Red Hat, IBM, etc...which can be costly for them. If your experts are reliable they would rather pay you only when they need help instead of paying the vendors monthly/yearly.
5) As your experts are working the problems you should keep a very organized knowledge base of the issue and resolution (maybe you can monetize this data later on or it maybe useful somehow)