The very existence of 'things of value' is what motivated people to build crawlers and search engines in the first place. There are many reasons to create 'a great piece of content', even if nobody else is likely to see it (e.g. for my personal reference, consolidating my understanding, just for fun, etc.). Filling it with fluff simply reduces its 'greatness'.

I think the problem is figuring out how we systematically identify things of value.

My personal feeling is that the solution lies in having good 'seed' URLs with which to build indexes. This might be done using curated lists, as a number of people here suggest, or perhaps through crawling and scoring the content.