We (meanpath) released[1] some crawl data which may be of interest to those of you hacking away on a search engine project.

[1] http://blog.meanpath.com/meanpath-january-2014-index/