I don't trust Facebook at all, but we need more than just "THE EVIL AI WAS MADE BY AN EVIL COMPANY" what is the concern? They're going to shove ads into your AI?
Facebook has been weaponized numerous times, why would this be any different? Why would they be given any benefit of the doubt?
> Dataset Sampling prop. Epochs Disk size
> CommonCrawl 67.0% 1.10 3.3 TB
> C4 15.0% 1.06 783 GB
> Github 4.5% 0.64 328 GB
> Wikipedia 4.5% 2.45 83 GB
> Books 4.5% 2.23 85 GB
> ArXiv 2.5% 1.06 92 GB
> StackExchange 2.0% 1.03 78 GB
At no point do they use information from Facebook customers