When I read this one obvious explanation would be that as parents gets older their gametes accumulate more mutations, especially men, and the children they have later on in life suffer a bit. But that effect should be very small compared to the effect sizes reported here.
The bit about fish eating makes me think this might relate to omega-3/omega-6 fat balances in our diets?