Not GP, but from having run a small/niche search engine that got hammered by a crawler in the past:

Webserver was a single VM running a Java + Spring webserver in Tomcat, connecting to an overworked Solr cluster to do the actual faceted searching.

Caches kept most page loads for organic traffic within respectable bounds, but the crawler destroyed our cache hit rate when it was scraping our site and at one point did exhaust a concurrent connection limit of some kind because there were so many slow/timing-out requests in progress at the same time.