Tell HN: I'm giving away a sample dataset -- URI scheme frequency
I've released a small, sample dataset based on a 7.5MM page crawl. The dataset lists the frequencies in which URI schemes appear in the anchor element's href attribute. The list was truncated at 300 URI schemes (for this sample) and is available in TSV format. The columns are: uri_scheme, frequency, unique_pages, unique_domains.
http://webscaled.com/static/samples/uri_scheme_part-00000.tsv
Please keep in mind that this is rough data. This file came straight from a MapReduce job. The values have not been classified (as they will be in the full version of the dataset), and a few are just people attempting to write "http:" but failed.
You are free to do whatever you please with this dataset. I just ask that you reference http://webscaled.com/ somewhere. Also, send me an email from the Webscaled contact form if you're interested in the sample dataset for top Doctypes.
Edit: Probably best to release under a license. Creative Commons Attribution-Share Alike 3.0 United States License.
Thanks for your time