Hacker News Reader

Could there be any merit training this on a common-sense dataset such as Cyc?

Probably not, most common facts (sandcat is a type of feline) are already known by transformers. Maybe some obscure ones.