It would be cool to have a site where you submit a business plan (along with web prototype, if it's a web business) and the plan is given to 3 random people from a pool of entrepreneurs, who write a formal review of your business (and perhaps sign a digital nda).
Maybe Ycombinator can sponsor something like this.
Consider some YC companies that are still fine-tuning their product. They've been reviewed by the likes of pg and co and had the benefit of their advice for 3 months as well as the benefit of experience from the large pool of YC alumni. But even they need to fine tune based on what the larger market thinks.
Still, as soon as you go live you'll be hanging out in the wind for all to see anyway.
Don't think it translates well to startup culture (people with good ideas tend to just do them) and besides, when you look for funding you often get peer-review of sorts from angels/VCs/Banks.
Lastly, although VCs are a good source of feedback, usually going and pitching your idea means a significant commitment (quit your job already, developing the idea). Furthermore, this form of feedback doesn't work for bootstrapped startups and startups in small markets.
Flickr for example, have groups that require you to comment on the previous 2 photos to post in that group. That doesn't mean all photos in the group have only 2 comments, many have much more because it creates a culture of critique.
As much there has been talk about "Not going dark" for programmers I think this is equally true of businesses. If you believe in what you do enough to tell other people about it then they can help keep you going in the right direction. If you stay in stealth mode forever you could be digging your own grave without knowing it.
1) rigor; comments have a tendency to be whatever pops into a person's head rather than a rigorous assessment of a business model. I think rigorous feedback would provide an opinion on each component of a successful business.
2) reputation; I know people here are very smart, but if you have a selected pool of entrepreneurs you can guarantee a certain level of reputation for your feedback. On ycombinator, you can still get a scathing review from a 17 year old startup wannabe and then sit there wondering whether they really know what they're talking about.
3) non-disclosure; when you post on ycombinator, your business plan reaches a public audience and gets indexed on google. With a random selection of 3 reviewers from a selected pool of advisors, this narrows the audience down, and furthermore you can get NDAs from them. Also, if your business magically appears the next day, you have some legal recourse if it's one of the reviewer's best friends.
But surely there must be some sort of solution to post it on the web and not get indexed. Google will surely not index flash videos. Or why create a locked page and then specify the password here. Google then cannot index your file :)
I hope I am wrong and what your looking for can be developed. It really would be an excellent service.
I guess the innovation here is creating a randomized pool of alpha-testers.