Appreciate the input Damon, ethical concerns are definitely a consideration and we'll want to be respectful of mechanisms like robots.txt