So what are your goals here? Are you trying to monetize your writing, or are you trying to prevent others from doing so?
So what are your goals here? Are you trying to monetize your writing, or are you trying to prevent others from doing so?
Right now I'd rather not have my comments aggregated at all, anywhere. But, if I change my mind, I want to be in control of the aggregation--who does it, what content is aggregated, what details are disclosed, and how much I am compensated for it.
I think eventually all search engines will have to do revenue sharing with content producers. Because, while search engines provide a useful service (even Backtype might be useful if it had a bigger index), they cannot exist without content to index. Plus, search engines are increasingly competing with content publishers (e.g. Google Knol, everything on Yahoo).
So, really what I am opposed to here is the opt-out (instead of opt-in) nature of the republication. I would be less opposed to it if there was an effective and convenient way to opt out of it. It seems right now the only ways to opt out are to file DMCA complaints and/or lobby websites to exclude the spam bloggers' robots in robots.txt.
You realize there is no way something like google could have been created in your proposed environment?
SEs are already becoming opt-in for some sites. Extranets are a great example. Some people are already building up robots.txt files that either exclude by default, or they have long lists of blacklists that they add to whenever they notice a new spider. For example, look at http://news.ycombinator.com/robots.txt. It disallows /reply? and /user?, which is basically all the content on the site. I expect that a large percentage websites will become more like opt-in instead of opt-out, especially as the number of robots increases--eventually, there will be so many robots spidering sites so frequently that bandwidth caps will be exhausted just from the robots.
I don't believe in "forceful" anything. But, I do believe that large, popular sites will make by making (exclusive) revenue sharing deals with search engines in the future. And, robots.txt can be used to advance some social policies too--for example, you could cut Yahoo off if you don't like its human rights record. Copyright is what will enable those kinds of things to happen.
I also think that Google's domination of search will come to an end and there will be more incentive for publishers to listen to offers that other search engines make. Google is probably not going to offer revenue sharing (from ads off SERPs) first, but I bet somebody else will. By this, I mean that the search engine will share revenue from clicks with the sites that appear on the SERPs that the clicked ad was on. It is the naturally evolution of SE advertising, and it is the only thing that will make SE's increasingly agressive use of publishers' and authors' content bearable for some people.
User-Agent: * Disallow: /x? Disallow: /vote? Disallow: /reply?
user is not disallowed anywhere, vote? and reply? and x? are, but they are not content, they are actions, basically all of the content of this site is indexed.
Your idea of site owners being selective about SE makes no sense to me. Would users then have to use 5 or 6 different search engines to accommodate all possible webmaster preferences?
I think that SE revenue sharing will happen. Only the SEs that share revenue will survive--a SE that doesn't share revenue will get locked out of the most valuable content and it will be forced out of business.