Hacker News Reader

Full thread

elchief

What about a system like this:

You crawl your own site every day (or generate the same content into files)

Put the content into files with file names representing the URLs (or HAR format?)

Zip em up

Put them on bittorrent

Tell the search engines to look there for your content

Wouldn't that save everyone a lot of work?

View on HN

That was almost the original intent behind sitemap.xml, (http://sitemaps.org). Except of course, that they were being used erroneously and changes published when no content was changed, and so the spiders stopped using them as canon.

This would be another dream come true for SEO. Too easy to game.

Reply on news.ycombinator.com