Hey there! Here is my perspective (Freelance Usability Engineer, Degree in Software-Engineering and Human-Computer-Interaction) on your situation...
Quick answer:
Your hunch is right. If you actually NEED quantitative research with scientific findings in the end you would need a lot of data (> 1.000 evenly distributed participants, no bias, no research background, ideal target group) and a bullet-proof testing set up with control groups etc.
But, from reading what you intend to do - this is not actually what you need. My assumptions are as follows (tell me if I am wrong):
- You don't need to publish anything "scientific" in the end
- You don't have a unlimited budget or large enough expected $$ outcome to justify spending lot of money into making it "statistically correct"
- You want to test/build/validate hypothesises quickly to come up with a viable product solution within reasonable effort and timeframe (Usual Entrepreneur / Intrapreneur limits)
Long answer:
I love the idea of approaching "idea generation" from a analytical + user centered point of view. Too often the engineers or founders give birth to a new feature or something "we should try" without checking if there is any possibility the users need it too. Because? Most of entrepreneuers and engineers would rather DO and IMPLEMENT stuff they like instead of waiting and talking to users - only to find out "their idea" is not needed at all.
I have done several projects in different niches where we did extensive user observation (contextual inquiry in their real life working environments [1] ) and requirements engineering interviews. What people tell you != what they actually do and feel! Nowadays this might be called "idea extraction" but uses the same concept of asking about their work process, good, bad, annoying and time consuming tasks. Trying to talk and ask without suggesting anyting and paying attention to their answers...
User interviews should be structured and you should try to be neutral + ask the same questions over and over again. BUT you dont need 1000 people. Usually it is far more effective to talk to (or observe) 7-10 people about one specific topic because you will still catch 80% of their issues and there will be patterns you can recognize.
So my personal approach would be to "scientifically" form 3-5 hypothesises which can be validated or invalidated by analyzing data, observing users and asking specific questions. Do your homework before and after, check analytics, get more insights from your databases, pre-validate your ideas and come up with observation-scenarios and questions which will validate or invalidate your hypothesis. Usually a few will already drop dead. Now as you are suggesting you would like to validate by using a product or prototype of some sort? For early validation and testing I would recomment to read "Sketching user experiences" from Bill Buxton [2] - lot of low fidelity ideas to test a product before writing any code at all.
You can either use live users to test some hypothesises, use remote usability testing tools or invite 5-10 people into your office for conducting a actual user test. I would highly recommend to go step by step. Only do a in house ux test if you are actually convinced something will work and you still have hypothesises on real users behaviour. The effort is far to high for just trying out stuff on an idea-level.
I hope this added some insights to your current challenge?
Patric
[0] http://uxstepbystep.com
[1] http://www.usabilityfirst.com/usability-methods/contextual-t...
[2] https://www.amazon.de/gp/product/0123740371