From what I've read, the answer is "yes", stackoverflow was crawled. EDIT: I looked and stackoverflow is included in the Common Crawl dataset, which is one of the datasets on which GPT-3 was trained. Having said that, it's not clear to me the degree of coverage of that domain guaranteed by that crawl... looks pretty comprehensive, though. http://index.commoncrawl.org/CC-MAIN-2020-24-index?url=*.sta...
The training data was so large that I don’t think a single epoch was used. I.e. it’s not a guarantee that any specific site is known by GPT-3 just because it’s in the data.
But, with the nature of random samples, it’s probably likely that it saw at least a little bit of SO.