Naive questions:
How does your data center look like? What is your web crawler written in?
I assume that your startup is fairly small - how difficult is/was it to get enough resources (funding) to scale up? (esp. datacenter wise)
I see that you do not just cover English and other (co.?? org.??) domains are covered fairly well.