It might be more useful for harder problems but there are already a number of sites addressing those : TopCoder, kaggle, Innocentive, etc.
Turns out everyone who writes a cucumber test is a programmer writing their own unit tests or a whole batch for everyone else.
I do think there is mileage in wufoo / VWO style webstie testing - record a series of macros and replay them each day against the site - let people know when something changes / does not work. ie a regular scheduled sign up with specific credit card then a cancellation - test that the whole sign up / email thing is still working.
Instead, I created "Amir" which is effectively a Selenium/CasperJS runner performing pre-recorded tasks each day and checking for outliers. Once it fails, emails go out.